Communicating Gestures

When I started working on gesture-driven systems, I made do with some terrible symbols and descriptions. Eventually, I evolved a few existing ones, made hands that made me happier, and codified some symbology (the circles for tap vs. hold, arrows) to be a system. And I have been happy with it, using it on dozens of projects. Until yesterday, when France Rupert made me aware of this new, quite well-conceived system that P.J. Onori developed. I'll let him explain in detail but basically, he approached it from an iconic communications point of view. To this end, it reminds me more of why I wanted gesture icons. For the way that I have always used keypresses or direction keys (along with focus areas) to communicate the reason for a state or screen change. If there's anything missing from analysis of gestures, or development of icons to communicate them, it's a visible awareness that this is an evolution of interaction, and it's always happened. When I first drew my gestures, I first went to very old drawings I had used to describe how people open boxes, turn pages, cut things, or pick up phones. Which I used long before there were touchscreen handsets. I am also excited by this sort of work because it means there's a lot of people using diagrams to design and communicate mobile interaction. I feel there's far too much talk about prototyping, even though there's lots of (justified) need to make specifications still. I feel the focus on prototyping sets aside centuries of designing by drawing as I sometimes call it. P.J.'s discussion, and some responses, also made me aware there are several other gesture libraries I didn't know about, so I finally added a whole section to the Drawing Tools & Templates section of the wiki for the Designing Mobile Interfaces book. If you don't want to go there, here they all are, gathered up:

• Gesture Icon System by P.J. Onori (someRandomDude), currently a prototype, but eventually vector art in all the likely platforms.
• Gesture Icons by Ryan Lee, paid download, for multiple platforms. Includes some kinesthetic gestures. (PDF, Illustrator, EPS)
• Touch Gesture Reference Guide by Luke Wroblewski. Set of stencils and other supporting documentation for understanding gesture. (PDF, EPS, OmniGraffle, Visio)
• Open Source Gesture Library by GestureWorks. Usable icons plus posters and other documentation. (PNG, PDF, EPS plus Gesture Font Family in TTF/OTF)
• GestureIcon Touch Pack by Ron George. Very abstracted icon set. (EPS, PNG, Illustrator)
• Touchscreen Stencils distributed by Kicker Studio, drawn by Rachel Glaves for Dan Saffer's book ''Designing Gestural Interfaces''. (OmniGraffle, Illustrator, Photoshop, Visio, Fireworks, Axure).
• Touch Notation by Matt Legend Gemmell, another very abstracted system. (Photoshop, Illustrator, OmniGraffle)
• Mobile Design Templates by Steven Hoober, a few pages of this large library include gesture icons, on-screen and kinesthetic. Used in the book ''Designing Mobile Interfaces'' (PDF, InDesign is in section above).

If this isn't enough, I also have never been totally happy with voice or haptic input and output iconography. This is the sort of thing I have done, but it's not perfect either. Now try showing orientation changes, or proximity to an RFID reader (or another device), or the type of haptic response. We can specify all this, but there's a lot to be said for innately understandable graphic libraries. As much as it disrupts my life, I look forward to more, and better solutions.