Paper: Writing Handwritten Messages on a Small Touchscreen

Here’s the final of our three papers at the MobileHCI 2013 conference. This was a particularly fun project, spearheaded by my colleague Wolf Kienzle, looking at a clever way to do handwriting input on a touchscreen using just your finger.

In general I’m a fan of using an actual stylus for handwriting, but in the context of mobile there are many “micro” note-taking tasks, akin to scrawling a note to yourself on a post-it, that wouldn’t justify unsheathing a pen even if your device had one.

The very cool thing about this approach is that it allows you to enter overlapping multi-stroke characters using the whole screen, and without resorting to something like Palm’s old Graffiti writing or full-on handwriting recognition.


The interface also incorporates some nice fluid gestures for entering spaces between words, backspacing to delete previous strokes, or transitioning to a freeform drawing mode for inserting little sketches or smiley-faces into your instant messages, as seen above.

This paper also had the distinction of receiving an Honorable Mention Award for best paper at MobileHCI 2013. We’re glad the review committee liked our paper and saw its contributions as noteworthy, as it were (pun definitely intended).

Writing-Small-Touchscreen-thumbKienzle, W., Hinckley, K., Writing Handwritten Messages on a Small Touchscreen. In ACM 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, (MobileHCI 2013), Munich, Germany, Aug. 27-30, 2013, pp. 179-182. Honorable Mention Award (Awarded to top 5% of all papers). [PDF] [video MP4] [Watch on YouTube – coming soon.]

18 responses to “Paper: Writing Handwritten Messages on a Small Touchscreen

  1. So will it work on mobiles soon? Maybe I do not get the exact idea, but if I understand correctly it would be awesome feature!!!!

    • This type of interface where you can pan and zoom around by moving your mobile in free space is definitely possible right now, by using the motion sensors (accelerometers and gyroscopes) that are integrated with most mobile devices right now.

      For an example of how this feels, look at the many image stitching apps that are out there, which allow you to capture video and then move around in the reconstructed scene as if it were a real 3D space.

      In this paper we didn’t actually use these motion sensors, for various technical and experimental control reasons, but these types of advancements in mobile sensors is what inspired us to revisit this “through-the-lens” type of metaphor for moving through information on a small-screen device. Our study shows that this approach has some promise — if you use it for the right things — but also has its challenges. It turns out to be more difficult to design an intuitive mapping to move around in an information space than we expected, but by gaining a deeper understanding of the human factors involved, we hope also to help encourage more sophisticated designs moving forward. Unfortunately it is all too easy to come up with a design that is almost right, but exactly wrong, without such insights.

  2. It’s sort of like the Nintendo DS or 3DS.

  3. Great opprtunity if it would be really usable

  4. webcommunicatoritalia

    L’ha ribloggato su Web Communicator Italiae ha commentato:
    Ecco un articolo interessante per quanto riguarda il futuro della scrittura!

  5. A nice way to stand out from conventional messengers like whats app.

  6. Very interesting idea to use instead of touch screen keyboards. Now I have a question does your app perform autocorrection?

    • In the version we reported here, we did not, since part of the point was that we never actually converted the handwriting to typeset text, but rather preserved the characteristics of the natural handwriting while still allowing the user to write arbitrarily overlapping strokes in a small writing space.

      However, it’s straightforward to take the next step as well, in applications where it makes sense to convert to text, and we have done that in combination with autocorrection as well. The technique also works well when used that way, but we haven’t published any studies of that yet. Hopefully soon.

  7. you have a typo in the subtitle of your blog. It’s probably because of a character limit right?

  8. handwriting interpreters or gesture based interfaces like this have gotten much better since the palm pilots 🙂 Thanks for sharing.

  9. Nice! Seems like a good alternative for people who miss receiving handwritten letters!

  10. Handwriting on your mobile, with just your finger! This looks very cool. The ability to personalize your message in the form of your own handwriting, I would say this is a break away from the ordinary and regular text messaging we are so accustomed to. Drawing little sketches or smiley faces to show moods with your finger, you can personalize however you like, and not having to choose a pic from your phone app that may not necessarily show the mood you are trying to express. Personalized interfaces on your mobile such as this may achieve what an ordinary punching keypad have not.

    • Thanks, Alize. That’s exactly the sort of more personal and enjoyable expression that we had in mind when we embarked on this project. There seems to be this myth with computers that everything has to be typed in. But there are many aspects of human knowledge and expression that are simply not fit for a keyboard.

  11. Pingback: Project: The Analog Keyboard: Text Input for Small Devices | The Past and Present Future

  12. Hey not bad! Those text messages look totally from the heart. Thumb typing just isn’t the same as hand writing.

  13. Pingback: Paper: LightRing: Always-Available 2D Input on Any Surface | The Past and Present Future

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s