Paper: Implicit Bookmarking: Improving Support for Revisitation in Within-Document Reading Tasks

The March 2013 issue of the International Journal of Human-Computer Studies features a clever new technique for automatically (implicitly) bookmarking recently-visited locations in documents, which (as our paper reveals) eliminates 66% of all long-distance scrolling actions for users in active reading scenarios.

The technique, devised by Chun Yu (Tsinghua University Department of Computer Science and Technology, Beijing, China) in collaboration with  Ravin Balakrishnan, myself, Tomer Moscovish, and Yuanchun Shi, requires only minimal modification of existing scrolling behavior in document readers — in fact, our prototype works by implementing a simple layer on top of the standard Adobe PDF Reader.

The technique would be particularly valuable for students or information workers whose activities necessitate deep engagements with texts such as technical documentation, non-fiction books on e-readers, or– of course, my favorite pastime– scientific papers.

Implicit Bookmarking prototype and studyYu, C., Balakrishnan, R., Hinckley, K., Moscovich, T.,  Shi, Y., Implicit bookmarking: Improving support for revisitation in within-document reading tasks. International Journal of Human-Computer Studies, Vol. 71, Issue 3, March 2013, pp. 303-320. [Definitive Version] [Author's draft PDF -- may contain discrepancies]

Paper: Gradual Engagement between Digital Devices as a Function of Proximity: From Awareness to Progressive Reveal to Information Transfer

I collaborated on a nifty project with the fine folks from Saul Greenberg’s group at the University of Calgary exploring the emerging possibilities for devices to sense and respond to their digital ecology. When devices have fine-grained sensing of their spatial relationships to one another, as well as to the people in that space, it brings about new ways for users to interact with the resulting system of cooperating devices and displays.

This fine-grained sensing approach makes for an interesting contrast to what Nic Marquardt and I explored in GroupTogether, which intentionally took a more conservative approach towards the sensing infrastructure — with the idea in mind that sometimes, one can still do a lot with very little (sensing).

Taken together, the two papers nicely bracket some possibilities for the future of cross-device interactions and intelligent environments.

This work really underscores that we are still largely in the dark ages with regard to such possibilities for digital ecologies. As new sensors and sensing systems make this kind of rich awareness of the surround of devices and users possible, our devices, operating systems, and user experiences will grow to encompass the expanded horizons of these new possibilities as well.

The full citation and the link to our scientific paper are as follows:

Gradual Engagement with devices via proximity sensingMarquardt, N., Ballendat, T., Boring, S., Greenberg, S. and Hinckley, K., Gradual Engagement between Digital Devices as a Function of Proximity: From Awareness to Progressive Reveal to Information Transfer. In Proceedings of ACM Interactive Tabletops & Surfaces (ITS 2012). Boston, MA, USA, November 11-14. 10pp. [PDF] [video - MP4].

Watch the Gradual Engagement via Proximity video on YouTube

GroupTogether — Exploring the Future of a Society of Devices

My latest paper discussing the GroupTogether system just appeared at the 2012 ACM Symposium on User Interface Software & Technology in Cambridge, MA.

GroupTogether video available on YouTube

I’m excited about this work — it really looks hard at what some of the next steps in sensing systems might be, particularly when one starts considering how users can most effectively interact with one another in the context of the rapidly proliferating Society of Devices we are currently witnessing.

I think our paper on the GroupTogether system, in particular, does a really nice job of exploring this with strong theoretical foundations drawn from the sociological literature.

F-formations are small groups of people engaged in a joint activity.

F-formations are the various type of small groups that people form when engaged in a joint activity.

GroupTogether starts by considering the natural small-group behaviors adopted by people who come together to accomplish some joint activity.  These small groups can take a variety of distinctive forms, and are known collectively in the sociological literature as f-formations. Think of those distinctive circles of people that form spontaneously at parties: typically they are limited to a maximum of about 5 people, the orientation of the partipants clearly defines an area inside the group that is distinct from the rest of the environment outside the group, and there are fairly well established social protocols for people entering and leaving the group.

A small group of two users as sensed by GroupTogether's overhead Kinect depth-cameras

A small group of two users as sensed via GroupTogether’s overhead Kinect depth-cameras.

GroupTogether also senses the subtle orientation cues of how users handle and posture their tablet computers. These cues are known as micro-mobility, a communicative strategy that people often employ with physical paper documents, such as when a sales representative orients a document towards to to direct your attention and indicate that it is your turn to sign, for example.

Our system, then, is the first to put small-group f-formations, sensed via overhead Kinect depth-camera tracking, in play simultaneously with the micro-mobility of slate computers, sensed via embedded accelerometers and gyros.

The GroupTogether prototype sensing environment and set-up

GroupTogether uses f-formations to give meaning to the micro-mobility of slate computers. It understands which users have come together in a small group, and which users have not. So you can just tilt your tablet towards a couple of friends standing near you to share content, whereas another person who may be nearby but facing the other way — and thus clearly outside of the social circle of the small group — would not be privy to the transaction. Thus, the techniques lower the barriers to sharing information in small-group settings.

Check out the video to see what these techniques look like in action, as well as to see how the system also considers groupings of people close to situated displays such as electronic whiteboards.

The full text of our scientific paper on GroupTogether and the citation is also available.

My co-author Nic Marquardt was the first author and delivered the talk. Saul Greenberg of the University of Calgary also contributed many great insights to the paper.

Image credits: Nic Marquardt

Paper: Cross-Device Interaction via Micro-mobility and F-formations (“GroupTogether”)

GroupTogetherMarquardt, N., Hinckley, K., and Greenberg, S., Cross-Device Interaction via Micro-mobility and F-formations.  In ACM UIST 2012 Symposium on User Interface Software and Technology (UIST ’12). ACM, New York, NY, USA,  Cambridge, MA, Oct. 7-10, 2012, pp. (TBA). [PDF] [video - WMV]. Known as the GroupTogether system.

See also my post with some further perspective on the GroupTogether project.

Watch the GroupTogether video on YouTube

Paper: Informal Information Gathering Techniques for Active Reading

This is my latest project, which I will present tomorrow (May 9th) at the CHI 2012 Conference on Human Factors in Computing Systems.

I’ll have a longer post up about this project after I return from the conference, but for now enjoy the video. I also link to the PDF of our short paper below which has a nice discussion of the motivation and design rationale for this work.

Above all else, I hope this work makes clear that there is still tons of room for innovation in how we interact with the e-readers and tablet computers of the future– as well as in terms of how we consume and manipulate content to produce new creative works.

Informal Information Gathering Techniques for Active ReadingHinckley, K., Bi, X., Pahud, M., Buxton, B., Informal Information Gathering Techniques for Active Reading. 4pp Note. In Proc. CHI 2012  Conf. on Human Factors in Computing Systems, Austin, TX, May 5-10, 2012. [PDF]

[Watch Informal Information Gathering Techniques for Active Reading on YouTube]

On Success in Creative Endeavors: How to Write Literary Masterpieces of Sheer Gilded Genius by Dumpster Diving

My alter-ego has a guest blog up on the Penumbra magazine site today that talks about how to succeed at creative endeavors. You know, writing and artwork, music and sculpture. All the good stuff.

And yes, scientific research is very much a creative endeavor as well.

In this short essay I also recount a story that Gary Starkweather, the inventor of the laser printer, used to tell about how he evaluated start-up companies, and I promise you will laugh when you hear how he did it.

I used to frequent the hardware lab where Gary worked before he retired so I consider myself very lucky to have spent a lot of time talking shop with him and enjoying his jokes. And this one is one of my favorite of his stories that he would tell about working in Silicon Valley. Great fun.

How to Write Literary Masterpieces of Sheer Gilded Genius by Dumpster Diving

PS: Gary was also prominently featured in Malcolm Gladwell’s recent essay in the New Yorker about The Creation Myth. Seek it out – it’s a great read on the truth about innovation!

Happy New Year, Fiction Tab, Story Published

Hi Folks and happy new year to you all. The blog has steamrolled past a thousand subscribers now (wow that came fast!) so welcome to all who have joined recently. And definitely feel free to leave a comment, question, or provocation– for me at least, a great post usually requires a great question to spur the dark recesses of my brain, so have at it if there’s a thought burning a hole in your mind.

I have a new post in the works about the remarkable proliferation of devices in my briefcase which will be up soon… quite a fascinating little study in the digital ecology of e-readers and other gadgets that is turning out to be.

In the meantime, I’ve added a Fiction tab to the blog header– I’m a research scientist by day, a writer of offbeat speculative tales and other literary abominations by night. Occasionally I even write stories that other people actually like to read. I’ll post any new publications in my treasury of wayward fictions up there as they come through the pipeline.

I recently made a sale to Penumbra, a new professional-paying speculative fiction magazine from Musa Publishing, and the issue with my short story just went live a few days ago. It speculates on an unlikely bit of direct-input display technology that every guy secretly wishes he had. Check out my funny little tale and all the other great stories in the issue if you want an entertaining read.

“Piss Match,” written as Alistair Ainscott, in Penumbra, Vol. 1 No. 4 (Musa Publishing), January 2012. Professional market (5¢/word). [Kindle]

A high-tech urinal, a pointless workplace confrontation– what could  possibly go wrong?

You can help support a great new speculative fiction magazine by subscribing to Penumbra. Many established authors– as well as newcomers like me– appear in the pages of the magazine.

Thanks and I sincerely hope the new year finds you well.