Gesture App Flutter Flickers With Potential
Flutter lets you control the play and pause functions of apps like iTunes using hand gestures, which are recognized via your webcam. It's fairly good at identifying the correct hand motion. Playing and pausing your iTunes music is OK, but what's most interesting about Flutter is its potential and how it could be used to control a wider variety of applications.
08/27/12 5:00 AM PT
Flutter, an app from the company of the same name, is available for free at the Mac App Store.
I like to listen to music occasionally while I work, but I also end up pausing songs often -- there are phone calls to take, to make, and sometimes writing particular paragraphs requires all the brain power I have available. I usually just hit the Play/Pause button at the top of my Mac keyboard.
Enter Flutter, a Mac OS X app that uses my built-in iSight camera on my MacBook Pro to recognize my palm in the air and pause and play iTunes -- with just a gesture. And what is the special move? Hold your hand up so the camera can see it, fingers pointing skyward with your thumb out to the side -- basically, the classic 'talk-to-the-hand' gesture.
Only Pause and Play
Flutter is a handy tool right now, but not crazy amazing. It does what it says it does well -- plays and pauses iTunes. In addition, it can play and pause QuickTime, VLC and Spotify. It works best from one to six feet away from your webcam. Often, it recognizes your gesture immediately, but sometimes you have to move your hand a bit, maybe close your hand and open it or give it a little wave to get Flutter to react.
It will pause and play your iTunes songs even when iTunes is playing in the background, so if you have a gob of applications open on your Mac with a bunch of browser windows everywhere, you don't have to hunt for iTunes to pause or start the music. Of course, if you're using an Apple keyboard, you've got a built-in Pause/Play button anyway. Still, Flutter is at least somewhat cool, if not occasionally more efficient.
It runs on your menu bar at the top of your screen, it's pretty small, and you can set it to launch automatically. When it does -- or you turn the camera on through the app -- the little green light on your built-in iSight webcam shines to let you know it's on, which is a bit eery at first. Is Flutter watching me as I work? Is my camera recording my actions? Or just watching for the hand?
Flutter, for its part, explicitly says that it's not collecting images or storing its user images in any way -- check out their privacy promise for more detail. There is a slight exception, though, and that's when you manage to confuse flutter. At one point while I was trying (and failing) to get it to work with the latest version of VLC on my Mac, I triggered a pop-up window that showed four photos of me with my hand up in front of my camera while I had a perplexed look on my face. The message? "When Flutter makes a mistake, we temporarily store frames associate with gestures on your computer. You can help improve Flutter by sending us these frames."
You can choose to send or not send the frames -- or click a check box to automatically send all frames when Flutter detects your gestures.
Obviously, I was curious as to Flutter's accuracy. Would it respond to a fist? No. A roll of toilet paper? No. A paperback book? No. A sideways hand? No. The back of a hand? Yes. A hand with the fingers pointing down? No. Two hands right next to each other? Sometimes. A hand with the thumb pulled into the palm? Sometimes, but mostly only to pause. To play? Not so much. How about a gray glove? Yes.
In short, it pretty much only recognizes the gesture it's programmed to respond to, and while it doesn't always react in an instant, in my testing it never reacted to anything other than a hand shape in the upright position.
To figure out which app Flutter is controlling, well, Flutter tells you with a little pop-down window whenever you close or open one of the supported apps. If you're running iTunes, for example, then open QuickTime, Flutter tells you that it's now controlling QuickTime. If you close QuickTime, Flutter shifts to control iTunes.
If you open iTunes to the iTunes Store, Flutter tells you to "please select a track in your iTunes playlist." Otherwise, it doesn't have any idea what you might want to play.
But what if you have an Apple TV that's connecting with your Mac via WiFi to stream a video in your living room? Nothing. Flutter seems to utterly ignore the Apple TV use of iTunes via your Mac and simply control your playlists that play on your Mac.
What's Coming Next
The most interesting thing about Flutter is what it might be able to control in the future. Flutter says that coming soon, it will support YouTube, Netflix, Pandora, and Hulu. As for additional gestures, the Flutter team seems to be working on gestures for skipping and liking songs, mute, and volume control.
All of these gestures and new apps would be welcome additions, of course.
Last of all, it's worth mentioning that there's a burgeoning set of visual recognition cues going on these days, most obviously from Microsoft with its Xbox 360 Kinect gaming system and the utterly mesmerizing work from Leap Motion, which uses a special adapter to visually identify minute finger gestures to move a mouse point just by waving your finger in the air.
Then there's the next-generation Apple TV that hasn't been released, the navigation systems for which Steve Jobs famously said he had cracked ... Flutter makes me wonder if maybe that navigation isn't all about the Siri voice assistant or an old-school remote and on-screen menu system. What if it will use gestures, too?
Still, what is here, right now, for your Mac? Flutter. It's totally worth checking out, even if you don't end up using it every day. Sparks the imagination, you know.