gesture keyboard

Every couple of years I get to thinking about the ultimate keyboard (the computer/typing kind). I think it all started back when FingerWorks introduced the TouchStream LP. The TouchStream was one of those strange, ergonomically-split keyboards. Ergonomics aside, the reason I got very interested in it was that the surface of the keyboard also acts as a touch pad (for mouse movement) and a gesture pad.


Gestures were a bit cutting-edge back then before touch-interfaces took off. In fact Apple bought Fingerworks in order to use that technology in the iPhone and their Magic Trackpad.


I’m a big fan of the Trackpoint mouse in the IBM keyboards because you don’t have to move your hand away from your keyboard every time you need to move your mouse (especially valuable with design tools that require constant moving and clicking of objects — equally interspersed with typing).

The big problem I read about with the TouchStream LP was that it was difficult for touch-typists (those who don’t look at the keyboard when they type) to use since there is little feedback about where your fingers are on the keyboard.

Then recently I downloaded an app called Fleksy (free on the iTunes store). This app was originally developed for the visually impaired. With Fleksy you don’t have to hit each key exactly. Fleksy cares more about the relative position of where your fingers touch. From that it figures out what words have roughly the same relative input pattern. It picks the most likely word and fills it in. In the case it didn’t choose the correct word, you can swipe down to select the runner-up word.

So far in practice, I can type with my eyes closed on my phone’s screen and Fleksy gets the word correct an astonishing amount of the time (90%?) and the other times the alternate word is simply one or two swipes away. (BTW, Fleksy is really cool if you look at the screen keyboard and type really fast, not worrying about hitting the keys accurately.)

Then it dawned on me, combining the Fleksy with the TouchStream LP keyboard solves the accuracy issue of touch-typing on a keyboard that doesn’t have ideal position feedback.

Taking the thought a bit further, the ergonomically split keyboard of the TouchStream LP is not completely needed. It seems that a continuous, flat surface would actually be more useful for the gestures and navigating.

With Windows 8 and other touch-based operating systems/applications becoming more prevalent, a keyboard capable of gestures and navigating/selecting objects on the display can eliminate the need for an expensive touch-capable screen — not to mention eliminate the awkwardness of constantly reaching back and forth from the keyboard to the screen.

In favor of the expensive touch-sensitive display, you have the ability to touch an icon directly. By contrast, using a gesture keyboard you must drag your finger until the cursor on the screen reaches the icon  — much the same way mouse-based users work with Windows 8. Of course it would be possible to mirror the display to the “keyboard’s” surface to allow for direct interaction with icons.


The “virtual” laser keyboard is another fine example of a product that is not very desirable to touch-typists. However with a desktop sensor that detects your finger positions (viz. the Leap Motion Controller), and technology like Fleksy, this portable keyboard becomes much more attractive. In fact, as evidenced by being able to use Fleksy without looking at my phone (Fleksy has an option to hide the onscreen keyboard as well), the projected, virtual keyboard is not really needed.

Fleksy needs to add touch pad features to their app, build a widget for Windows/OSX, then we could use our iPads as a keyboard/touch pad for our desktops/laptops. It would also be neat to see a Magic Trackpad from Apple at least the size of a keyboard with the above functionality added.

Posted in Computer Peripherals, Gesture Keyboards, Technology | Tagged , , , , , , , | 1 Comment