Tag Archives: mode switching

Project: Bimanual In-Place Commands

Here’s another interesting loose end, this one from 2012, which describes a user interface known as “In-Place Commands” that Michel Pahud, myself, and Bill Buxton developed for a range of direct-touch form factors, including everything from tablets and tabletops all the way up to electronic whiteboards a la the modern Microsoft Surface Hub devices of 2015.

Microsoft is currently running a Request for Proposals for Surface Hub research, by the way, so check it out if that sort of thing is at all up your alley. If your proposal is selected you’ll get a spiffy new Surface Hub and $25,000 to go along with it.

We’ve never written up a formal paper on our In-Place Commands work, in part because there is still much to do and we intend to pursue it further when the time is right. But in the meantime the following post and video documenting the work may be of interest to aficionados of efficient interaction on such devices. This also relates closely to the Finger Shadow and Accordion Menu explored in our Pen +Touch work, documented here and here, which collectively form a class of such techniques.

While we wouldn’t claim that any one of these represent the ultimate approach to command and control for direct input, in sum they illustrate many of the underlying issues, the rich set of capabilities we strive to support, and possible directions for future embellishments as well.

Thumbnail for In-Place CommandsKnies, R. In-Place: Interacting with Large Displays. Reporting on research by Pahud, M., Hinckley, K., and Buxton, B. TechNet Inside Microsoft Research Blog Post, Oct 4th, 2012. [Author’s cached copy of post as PDF] [Video MP4] [Watch on YouTube]

In-Place Commands Screen Shot

The user can call up commands in-place, directly where he is working, by touching both fingers down and fanning out the available tool palettes. Many of the functions thus revealed act as click-through tools, where the user may simultaneously select and apply the selected tool — as the user is about to do for the line-drawing tool in the image above.

Watch Bimanual In-Place Commands video on YouTube

Paper: ExperiScope: An Analysis Tool for Interaction Data

ExperiScopeGuimbretiére, F., Dixon, M., and Hinckley, K., ExperiScope: An Analysis Tool for Interaction Data. In Proc. CHI 2007 Conference on Human Factors in Computing Systems, San Jose, CA, April 28 – May 03, 2007, pp. 1333-1342. [PDF] [video .MOV] Note: See also the experiment reported in the Springboard paper, which this study further analyzes.

Paper: InkSeine: In Situ search for active note taking

InkSeine CHI 2007Hinckley, K., Zhao, S., Sarin, R., Baudisch, P., Cutrell,
E., Shilman, M., and Tan, D. 2007. InkSeine: In Situ search for active note taking. In Proc. CHI 2007 Conf. on Human Factors in Computing Systems, San Jose, CA, pp. 251-260.  [PDF] [video .MOV] [OfficeLabs Thought Leadership Award] [Install]

Watch the InkSeine CHI 2007 Research Paper Video on YouTube

See also:

Watch the InkSeine demo reel from the public download on YouTube.

Or Install InkSeine and try it out for yourself.

Paper: Phrasing Techniques for Multi-Stroke Selection Gestures

Phrasing TechniquesHinckley, K., Guimbretiere, F., Agrawala, M., Apitz, G., and Chen, N. Phrasing Techniques for Multi-Stroke Selection Gestures. In Proceedings of Graphics interface 2006, Quebec, Canada, June 07 – 09, 2006). Published by Canadian Information Processing Society, Toronto, Ont., Canada, pp. 147-154. [PDF] [video .WMV]

Watch Phrasing Techniques on YouTube

Paper: The Springboard: Multiple Modes in One Spring-Loaded Control

Springboard Mode Switching Techniques & AnalysisHinckley, K., Guimbretiere, F., Baudisch, P., Sarin, R., Agrawala, M., and Cutrell, E., The Springboard: Multiple Modes in One Spring-Loaded Control. In Proc. CHI 2006 Conf. on Human Factors in Computing Systems, Montréal, Québec, Canada, April 22 – 27, 2006, pp. 181-190. [PDF] [video .MOV] Note: See also the ExperiScope paper for some further analysis and discussion of this experiment’s results.

[Watch Springboard Multiple Mode Control on YouTube]

Paper: Design and Analysis of Delimiters for Selection-Action Pen Gesture Phrases in Scriboli

Pen Gestures in ScriboliHinckley, K., Baudisch, P., Ramos, G., and Guimbretiere, F., Design and Analysis of Delimiters for Selection-Action Pen Gesture Phrases in Scriboli. In Proc. CHI 2005 Conf. on Human Factors in Computing Systems, Portland, OR, April 02 – 07, 2005, pp. 451-460. [PDF] [video .WMV]

[Watch Scriboli Pen Gestures on YouTube]

Paper: Snap-And-Go: Helping Users Align Objects without the Modality of Traditional Snapping

Baudisch, P., Cutrell, E., Hinckley, K., and Eversole, A. Snap-And-Go: Helping Users Align Objects without the Modality of Traditional Snapping. In Proc. CHI 2005 Conf. on Human Factors in Computing Systems, Portland, OR, April 02 – 07, 2005, pp. 301-310. CHI 2005 Best Paper Nomination Award. [PDF] [demo on Patrick Baudisch’s Flash web page]

Paper: Experimental Analysis of Mode Switching Techniques in Pen-Based User Interfaces

Mode Switching ExperimentLi, Y., Hinckley, K., Guan, Z., and Landay, J. A., Experimental Analysis of Mode Switching Techniques in Pen-Based User Interfaces. In Proc. CHI 2005 Conf. on Human Factors in Computing Systems, Portland, OR, April 02 – 07, 2005, pp. 461-470. [PDF] [animated .GIF showing experimental task]