Tag Archives: 2017-paper

Paper: WritLarge: Ink Unleashed by Unified Scope, Action, & Zoom

Electronic whiteboards remain surprisingly difficult to use in the context of creativity support and design.

A key problem is that once a designer places strokes and reference images on a canvas, actually doing anything useful with key parts of that content involves numerous steps.

Hence, with digital ink, scope—that is, selection of content—is a central concern, yet current approaches often require encircling ink with a lengthy lasso, if not switching modes via round-trips to the far-off edges of the display.

Only then can the user take action, such as to copy, refine, or re-interpret their informal work-in-progress.

Such is the stilted nature of selection and action in the digital world.

But it need not be so.

By contrast, consider an everyday manual task such as sandpapering a piece of woodwork to hew off its rough edges. Here, we use our hands to grasp and bring to the fore—that is, select—the portion of the work-object—the wood—that we want to refine.

And because we are working with a tool—the sandpaper—the hand employed for this ‘selection’ sub-task is typically the non-preferred one, which skillfully manipulates the frame-of-reference for the subsequent ‘action’ of sanding, a complementary sub-task articulated by the preferred hand.

Therefore, in contrast to the disjoint subtasks foisted on us by most interactions with computers, the above example shows how complementary manual activities lend a sense of flow that “chunks” selection and action into a continuous selection-action phrase. By manipulating the workspace, the off-hand shifts the context of the actions to be applied, while the preferred hand brings different tools to bear—such as sandpaper, file, or chisel—as necessary.

The main goal of the WritLarge project, then, is to demonstrate similar continuity of action for electronic whiteboards. This motivated free-flowing, close-at-hand techniques to afford unification of selection and action via bimanual pen+touch interaction.


Accordingly, we designed WritLarge so that user can simply gesture as follows:

With the thumb and forefinger of the non-preferred hand, just frame a portion of the canvas.

And, unlike many other approaches to “handwriting recognition,” this approach to selecting key portions of an electronic whiteboard leaves the user in complete control of what gets recognized—as well as when recognition occurs—so as not to break the flow of creative work.

Indeed, building on this foundation, we designed ways to shift between flexible representations of freeform content by simply moving the pen along semantic, structural, and temporal axes of movement.

See our demo reel below for some jaw-dropping demonstrations of the possibilities for digital ink opened up by this approach.

Watch WritLarge: Ink Unleashed by Unified Scope, Action, and Zoom video on YouTube

WritLarge-CHI-2017-thumbHaijun Xia, Ken Hinckley, Michel Pahud, Xioa Tu, and Bill Buxton. 2017. WritLarge: Ink Unleashed by Unified Scope, Action, and Zoom. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, USA, pp. 3227-3240. Denver, Colorado, United States, May 6-11, 2017. Honorable Mention Award (top 5% of papers).

[PDF] [Watch 30 second preview on YouTube]

Paper: Thumb + Pen Interaction on Tablets

Modern tablets support simultaneous pen and touch input, but it remains unclear how to best leverage this capability for bimanual input when the nonpreferred hand holds the tablet.

We explore Thumb + Pen interactions that support simultaneous pen and touch interaction, with both hands, in such situations. Our approach engages the thumb of the device-holding hand, such that the thumb interacts with the touch screen in an indirect manner, thereby complementing the direct input provided by the preferred hand.

For instance, the thumb can determine how pen actions (articulated with the opposite hand) are interpreted.


Alternatively, the pen can point at an object, while the thumb manipulates one or more of its parameters through indirect touch.

Our techniques integrate concepts in a novel way that derive from radial menus (also known as marking menus) and spring-loaded modes maintained by muscular tension — as well as indirect input, and in ways that leverage multi-touch conventions.

Our overall approach takes the form of a set of probes, each representing a meaningfully distinct class of application. They serve as an initial exploration of the design space at a level which will help determine the feasibility of supporting bimanual interaction in such contexts, and the viability of the Thumb + Pen techniques in so doing.

Watch Thumb + Pen Interaction on Tablets video on YouTube

thumb-pen-thumbKen Pfeuffer, Ken Hinckley, Michel Pahud, and Bill Buxton. 2017. Thumb + Pen Interaction on Tablets. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, USA, pp. 3254-3266 . Denver, Colorado, United States, May 6-11, 2017.

[PDF] [Watch 30 second preview on YouTube]

Paper: As We May Ink? Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences

This work sheds light on gaps and discrepancies between the experiences afforded by analog pens and their digital counterparts.

Despite the long history (and recent renaissance) of digital pens, the literature still lacks a comprehensive survey of what types of marks people make and what motivates them to use ink—both analog and digital—in daily life.


To capture the diversity of inking behaviors and tease out the unique affordances of pen-and ink, we conducted a diary study with 26 participants from diverse backgrounds.

From analysis of 493 diary entries we identified 8 analog pen-and-ink activities, and 9 affordances of pens. We contextualized and contrasted these findings using a survey with 1,633 respondents and a follow-up diary study with 30 participants, observing digital pens.

Our analysis revealed many gaps and research opportunities based on pen affordances not yet fully explored in the literature.

As-We-May-Ink-CHI-2017-thumbYann Riche, Nathalie Henry Rich, Ken Hinckley, Sarah Fuelling, Sarah Williams, and Sheri Panabaker. 2017. As We May Ink? Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, USA, pp. 3241-3253. Denver, Colorado, United States, May 6-11, 2017.

[PDF] [CHI 2017 Talk Slides (PowerPoint)]