Kinect Maps UX Lessons – Getting Users Up to Speed

With our Kinect Maps project, we had the rare opportunity to  study user behavior in an exhibition setting. The application was seen by thousands, while hundreds of people actually tried it themselves. We had a very wide range of users – from 4-5 years old youngsters to ladies over 60. From computer literates to those who have never seen a computer beyond the occasional console games. Hardcore gamers to “I am serious businessman and never play games” types. Those who are familiar with Kinect and the notion of “you are the controller” to those who have never even knew such technology existed.

So, what are the lessons we learned from a User Experience standpoint? What worked what didn’t work? I’ll share what we learned in this blog post and the next few.

Learning to use Kinect Maps

No matter how Natural a UI is, there is always a learning curve. Just like there is a learning curve with anything – riding a bicycle, using a new phone, or even reading a new book (for example, technical books usually have their own set of typography and iconography to separate tips, notes, warnings, etc).

In an exhibition scenario, there is extra emphasis on fast learning. You can probably get away with a 5-10 minute timeframe tutorial for a “real” application or a game (just look at Kinect games – they usually have a few minutes of introduction to show you the control scheme) but an exhibition is a totally different context. Visitors usually only spend a few minutes at one place – if you can’t get them hooked in seconds, they will move on to another booth in a heartbeat.

By the way, you should see my previous post before going forward – I will talk about the gestures and interaction mechanisms I described there. The control scheme of the application is a bit on the complex side, so to get the maximum out of reading this post, you should get to know them. Or at least watch this video:

So what did we do to get users involved as fast as humanly possible? Three things:

On-screen instructions

As we anticipated, on-screen instructions were the least effective. Instructions were hidden in a “border” so that they don’t interfere with the actual experience – therefore most users never even realized it was there. Nevertheless, having the instructions was very important – they helped the hostesses learn, and when a user was specifically engaged or wanted to discover the application by his or herself, these little icons helped a lot.

(note the funny pose – Kinect couldn’t see my leg when this screenshot was taken).

Watching the previous user

Fortunately, there often were queues of people waiting for their turn. As people waited, they saw how the previous user was interacting with the application, and learned most of the gestures by the time they got in front of the Kinect. We even had a 11-12 year old kid jump in front of the application and pretty much knowing everything about calibration and fly mode for the very first time.

Since waiting and watching others “play” requires a lot less concentration than actually doing it yourself, some of the people in the queue discovered the on-screen instructions and studied them before entering the play area.

Watching the previous user is one of the areas where the gestures being “Natural” pays off. The airplane-like gestures, stepping back and forth, the raising and lowering of hands is natural – at least they become natural as soon as you see them in action, and see the effect these gestures have on the 3D map. These gestures are not natural by themselves, but it seems the brain makes the connection and accepts them as the way of manipulating the virtual world immediately.

Note: this is pretty similar to the UI paradigms of the iPhone. Is touching the screen with your finger, and swiping up and down a list natural? No. It is not, because you probably wouldn’t have guessed that this is what you have to do in the era before touchscreens. Actually, you were very careful not to touch a screen, since it would leave smudges and grease all over. But Apple was a genius in this case. They flooded the TV stations with advertisements like these:

And these advertisements showed you how to use the phone! Showed you that it was OK to touch the screen with your finger, and taught how the touchscreen interface was used. And by the time you got to the shop to pick up one, you immediately knew how to use it, you immediately knew how to scroll, zoom, rotate the phone for videos, and so on. It felt natural – but only because you have seen the gestures being used before.

Employing a hostess

You can create great instructions or rely on learning by example – but the most useful thing you can do to get users involved as quickly as possible is to have an actual human being who can react to the user’s actions, explains what needs to be done and how they can get the most out of the application. I guess you could do an on-screen or even a virtual assistant in a game, but in the demanding context of an exhibition with tons of interesting stuff to see all around, going with the hostess option is the best.

Since the drivers we used were a little bit shaky, this is especially true. For example, they require a calibration for every new user. You have to stand in a specific pose before the drivers recognize you and are able to send 3D skeleton data to the application. While we had a “stand in this pose” instruction on the screen, it would not show up if other calibrated users were still in the picture.

We created the application in a way that the last user to perform the calibration gesture takes control. This allowed us to switch users even when the previous user was still tracked by Kinect. On the other hand, this meant that the hostesses had to show the calibration pose to every new user.

The hostesses were trained with fairly deep knowledge of the application. For example, they knew that the further your right leg is in front of the left one, the faster the zooming or flying happens. They knew tricks – like when in map mode, you can just put your hands out, keep the target area in the center of the screen, and control the zooming speed with your right leg. This way you can zoom in to any area in one swift motion. And they also knew to restart the application if it totally refused to recognize the user (this happened every few hours, but the drivers are called “unstable” for a reason...)

They could also help answer questions – why doesn’t Hungary have 3D buildings, who created the app, what to do to switch from map mode to fly mode, how to fly up, forward, etc. And of course, they attracted a lot of young male visitors and could effectively persuade them to actually try the application out.

Coming up

Coming up in my next blog posts on the subject of the Kinect Maps UX:

• The evolution of the Kinect Maps gesture language
• What worked and what didn’t
• What would we have added if we had more time
• The importance of user feedback

One more thing…

Would you like to try the application on your own computer? If yes, please send me an email to velvart.andras at response dot hu!