Category Archives: paper-based interaction

Paper: As We May Ink? Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences

This work sheds light on gaps and discrepancies between the experiences afforded by analog pens and their digital counterparts.

Despite the long history (and recent renaissance) of digital pens, the literature still lacks a comprehensive survey of what types of marks people make and what motivates them to use ink—both analog and digital—in daily life.

As-We-May-Ink-fullsize

To capture the diversity of inking behaviors and tease out the unique affordances of pen-and ink, we conducted a diary study with 26 participants from diverse backgrounds.

From analysis of 493 diary entries we identified 8 analog pen-and-ink activities, and 9 affordances of pens. We contextualized and contrasted these findings using a survey with 1,633 respondents and a follow-up diary study with 30 participants, observing digital pens.

Our analysis revealed many gaps and research opportunities based on pen affordances not yet fully explored in the literature.


As-We-May-Ink-CHI-2017-thumbYann Riche, Nathalie Henry Rich, Ken Hinckley, Sarah Fuelling, Sarah Williams, and Sheri Panabaker. 2017. As We May Ink? Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, USA, pp. 3241-3253. Denver, Colorado, United States, May 6-11, 2017.
https://doi.org/10.1145/3025453.3025716

[PDF] [CHI 2017 Talk Slides (PowerPoint)]

Book Chapter: Input/Output Devices and Interaction Techniques, Third Edition

Thumbnail for Computing Handbook (3rd Edition)Ken Hinckley, Robert J.K. Jacob, R., Colin Ware, Jacob O. Wobbrock, and Daniel Wigdor. Input/Output Devices and Interaction Techniques. Appears as Chapter 21 in The Computing Handbook, Third Edition: Two-Volume Set, ed. by Tucker, A., Gonzalez, T., Topi, H., and Diaz-Herrera, J. Published by Chapman and Hall/CRC (Taylor & Francis), May 13, 2014.  ISBN 9781439898444. [PDF – Author’s Draft – may contain discrepancies]

Paper: Informal Information Gathering Techniques for Active Reading

This is my latest project, which I will present tomorrow (May 9th) at the CHI 2012 Conference on Human Factors in Computing Systems.

I’ll have a longer post up about this project after I return from the conference, but for now enjoy the video. I also link to the PDF of our short paper below which has a nice discussion of the motivation and design rationale for this work.

Above all else, I hope this work makes clear that there is still tons of room for innovation in how we interact with the e-readers and tablet computers of the future– as well as in terms of how we consume and manipulate content to produce new creative works.

Informal Information Gathering Techniques for Active ReadingHinckley, K., Bi, X., Pahud, M., Buxton, B., Informal Information Gathering Techniques for Active Reading. 4pp Note. In Proc. CHI 2012  Conf. on Human Factors in Computing Systems, Austin, TX, May 5-10, 2012. [PDF]

[Watch Informal Information Gathering Techniques for Active Reading on YouTube]

Informal Organization and the Emerging Class of Casual-Use Devices

A while back Bill Buxton put up a brief quotation on billbuxton.com that I like a lot:

Just a thought: Slate computers and e-readers represent a new class of digital appliance– one targeted for casual use. With the growth of this market will emerge  a new and long overdue approach to interaction– one that is in keeping with the casual intent and context of such usage and which will complement, rather than replace, interfaces that support more formal and structured activities. 

He goes on to make a nice little plug for my InkSeine Tablet PC application, but that’s only a very small part of why I like his quotation.

To me Bill’s comment gets at the essence of the new approach to design and interaction demanded by this emerging class of casual-use devices.

And it’s one that’s already deeply rooted in human behavior, even back in those dark ages before our twilight dreams were set afire by the eerie blue-white glow of our iPad portals to never never land.

I like to call it informal organization. It’s a lot of why paper has remained so pervasive in the so-called “paperless” office of the modern information worker. (You could do a lot worse than to spend a weekend reading The Myth of the Paperless Office by Sellen & Harper to understand this more deeply).

But now that e-reading and tablets are gaining more and more traction, we’re seeing at least some of those paper-based behaviors and expectations smash head-on into the limitations of current-generation readers and tablets.

I can send my documents to my Kindle to read them away from my desktop; I’ve never really liked doing much reading on my computer. And printing seems wasteful and moreover has a higher transaction cost than just sending the document to my reader— hitting print, selecting the right printer, remembering to pick the auto-staple function, walking down the hall to pick it up, coming back to my office and actually starting into reading before I get interrupted and forget why the hell I printed out the document in the first place…

But on the Kindle or iPad, I can get right into reading my document quicker, but then when that moment comes that I want to mark up some copy-edits, jot an annotation in the margin, highlight a passage, slap a post-it on it with a note about what to do next, thumb back and forth quickly between multiple documents, or god forbid casually toss the document into a pile with other like-minded papers– the workflow breaks down and I am into clumsy workarounds and tricks and little idiosyncratic habits I’ve developed to make the tools work for me as best they can.

This is where the frontier in e-reading experiences currently lies.

The e-ink reader and yes even our fancy-pants multi-touch iPads are going to look like antiquated glowing green-phosphor displays to our eyes a few short years from now.

Because these unmet needs are all about the interaction experience, figuring out what the appropriate digital analogs of these behaviors are, and figuring out what the right input modalities and interaction techniques and workflow are to allow users to express these notions as quickly and casually as they can conceive of them.

I talk about some of this in my recent paper Pen + Touch = New Tools. And a few techniques are shown in the video that goes along with that, although to be honest they are clumsy and have a long way to go to get to where I really think things need to be.

As another example, take a look at the Collections mechanism on the Kindle for organizing your e-books. My friend and former colleague Jeff Pierce wrote about the rather limited Kindle book organization mechanisms on his blog, which was actually what tipped me into this little rant. Just gathering together a half-dozen short stories, or documents that you’ve emailed to yourself, or your read vs. unread books and managing that on an ongoing basis is a huge pain in the ass.

But it doesn’t have to be that way. With the massive shift to electronic reading that the publishing industry is tipping towards, and as more and more “real” work versus casual recreational reading moves to such devices, there’s a huge incentive to get these reading devices and tablets right.

And making something that’s even half as good as the stacks of books and papers and post-its scattered around my desk wouldn’t be a bad start.

Book Chapter: Input Technologies and Techniques, 2012 Edition

Input Technologies and Techniques, 3rd EditionHinckley, K., Wigdor, D., Input Technologies and Techniques. Chapter 9 in The Human-Computer Interaction Handbook – Fundamentals, Evolving Technologies and Emerging Applications, Third Edition, ed. by Jacko, J., Published by Taylor & Francis. To appear. [PDF of author’s manuscript – not final]

This is an extensive revision of the 2007 and 2002 editions of my book chapter, and with some heavy weight-lifting from my new co-author Daniel Wigdor, it treats direct-touch input devices and techniques in much more depth. Lots of great new stuff. The book will be out in early 2012 or so from Taylor & Francis – keep an eye out for it!

Paper: Pen + Touch = New Tools

Pen + Touch = New ToolsHinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H., Buxton, B., Pen + Touch = New Tools. In Proc. UIST 2010  Symposium on User interface Software and Technology, New York, NY, pp. 27-36. [PDF] [video .WMV]

Watch Pen + Touch = New Tools on YouTube

Paper: Direct Display Interaction via Simultaneous Pen + Multi-touch Input

Simultaneous Pen + TouchHinckley, K., Pahud, M., Buxton, B., Direct Display Interaction via Simultaneous Pen + Multi-touch Input.  In Society for Information Display (SID) Symposium Digest of Technical Papers, May 2010, Volume 41(1), Session 38, pp. 537-540. [PDF]

This paper had no accompanying video, but you can see the system in action in this YouTube video:

Watch Simultaneous Pen + Touch video on YouTube

Paper: Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input

Manual DeskterityHinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H., Buxton, B., Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input. In CHI 2010 Extended Abstracts:  Proc. CHI 2010 Conf. on Human Factors in Computing Systems, Atlanta, GA, pp. 2793-2802. [PDF] [video .WMV]

Watch Manual Deskterity video on YouTube

Paper: Codex: a Dual-Screen Tablet Computer

The Codex dual-screen tablet computerHinckley, K., Dixon, M., Sarin, R., Guimbretiere, F., and Balakrishnan, R. 2009. Codex: a Dual-Screen Tablet Computer. In Proce. CHI 2009 Conf. on Human Factors in Computing Systems, Boston, MA, pp. 1933-1942. [PDF] [video .MOV] [OfficeLabs Thought Leadership Award]

Watch the Codex video on YouTube

Book Chapter: Input Technologies and Techniques (in Human-Computer Interaction Fundamentals)

Input-Technologies-and-Techniques-HCI-Handbook-FundamentalsHinckley, K., Input Technologies and Techniques. Chapter 9 in Human-Computer Interaction Fundamentals (Human Factors and Ergonomics), ed. by Sears, A., and Jacko, J., CRC Press, Boca Raton, FL. Published March 2, 2009. Originally appeared as Chapter 9 in Human-Computer Interaction Handbook, 2nd Edition. [PDF of author’s manuscript – not final].