Usable Augmented Reality

If there’s one thing that can add a “wow” factor to your app, it’s probably Augmented Reality. You may not have heard of that phrase before, so, for the uninitiated, I’ll briefly explain:

Augmented Reality (or AR for short) is a technique where data is superimposed on top of the image shown by your phone’s (or tablet’s) camera, and where that data moves along with the camera’s view as you move the device around. Some apps make the superimposed data appear to be fixed to a specific location in the camera’s view -so  if you move the camera to the left, the data moves to the right –  appearing to stay in the same location – thus augmenting that view of reality. Some apps also augment the view with information or symbols that tell you something about particular features around you  such as the direction to the nearest bank.

The key to making AR work is the ability to know what direction and orientation the device is, where it is precisely in the world, and how these things are changing in real-time. It’s only possible to know these things when your mobile device includes an accelerometer, a compasses, a gyroscope and a GPS chip – something that the latest iPhone and Android devices now include as standard. Of course, the device also needs to have a camera!

Data

Once you have all the necessary hardware at your disposal, you then need to have the data that you want to show, together with its location. The latter is becoming easier and easier to obtain since most things in the world have a locational element to them and determining that can be as easy as clicking on a Google map to obtain a latitude/longitude.

As a developer wanting to offer AR in your app, you then need to write some code to do some number crunching that can take readings from all the hardware components, work out the direction that the camera is looking at, work out whether the data you want to show should appear in the view and then display it. All this in itself is hard, but I would go on to say that there’s one more thing that’s probably more challenging than that – usability.

Usability

To improve your app’s usability, you must consider the following:

• How do you show the data effectively on the screen?
• What if the user is pointing their device where you have no data?
• What if there’s too many features to show at once?

All of these questions need to be addressed if your app is to be usable.

AR in the Royal Ascot app

We addressed some of these issues when we put an AR feature into the Royal Ascot app.

In this case, the data we needed to show represented the locations of restaurants and bars for people to visit whilst at the racecourse. As the user walked around, they could see where things were in relation to their current position.

The fact that the user needed to be at the racecourse really meant that there was no need to show the features if the user wasn’t actually there – so the first thing we did was to only show the venues within 500m of the user’s position.

However, since the venues were in close proximity to each other it became apparent that in a lot of situations there were still too many features to display in the view at the same time – so much so that you couldn’t really see everything you needed to.

So, as well as plotting the venues in distance order – nearest on top, farthest underneath, we added the ability to allow the user to adjust the distance within which the venues were shown – down to 50m. This meant that restaurants and bars further away could be “filtered” out of the view, thus de-cluttering the display.

Showing the Venue Names

Each venue’s name also needed to be clearly shown. So we used the same “bubble pins” as used on the map in the app. This also helped to maintain a standardised style throughout the app.

However, on our first trial of the system all the pins were being positioned along the horizon in the camera view.

This was originally intended and in development it seemed to be a logical decision, but in action, in the field so to speak, we found that it was actually quite difficult to see what each pin was meant to be “pinned” to – the pin was obscuring the venue it was meant to be referring to!

So, we quickly decided that the pins needed to appear towards the top of the screen – above the horizon and above the venue in question.

Lessons Learned

This made for a much more user-friendly experience. But if there’s one lesson to be learned from this experience it’s the fact that, as with all development, you can never do enough testing. With AR especially, until you get out and actually try the app in anger, in real-life situations, you can never really understand how it will work.