Usable Augmented Reality

If there’s one thing that can add a “wow” factor to your app, it’s probably Augmented Reality. You may not have heard of that phrase before, so, for the uninitiated, I’ll briefly explain:

Augmented Reality (or AR for short) is a technique where data is superimposed on top of the image shown by your phone’s (or tablet’s) camera, and where that data moves along with the camera’s view as you move the device around. Some apps make the superimposed data appear to be fixed to a specific location in the camera’s view -so  if you move the camera to the left, the data moves to the right -  appearing to stay in the same location – thus augmenting that view of reality. Some apps also augment the view with information or symbols that tell you something about particular features around you  such as the direction to the nearest bank.

The key to making AR work is the ability to know what direction and orientation the device is, where it is precisely in the world, and how these things are changing in real-time. It’s only possible to know these things when your mobile device includes an accelerometer, a compasses, a gyroscope and a GPS chip – something that the latest iPhone and Android devices now include as standard. Of course, the device also needs to have a camera!

Data

Once you have all the necessary hardware at your disposal, you then need to have the data that you want to show, together with its location. The latter is becoming easier and easier to obtain since most things in the world have a locational element to them and determining that can be as easy as clicking on a Google map to obtain a latitude/longitude.

As a developer wanting to offer AR in your app, you then need to write some code to do some number crunching that can take readings from all the hardware components, work out the direction that the camera is looking at, work out whether the data you want to show should appear in the view and then display it. All this in itself is hard, but I would go on to say that there’s one more thing that’s probably more challenging than that – usability.

Usability

To improve your app’s usability, you must consider the following:

• How do you show the data effectively on the screen?
• What if the user is pointing their device where you have no data?
• What if there’s too many features to show at once?

All of these questions need to be addressed if your app is to be usable.

AR in the Royal Ascot app

We addressed some of these issues when we put an AR feature into the Royal Ascot app.

In this case, the data we needed to show represented the locations of restaurants and bars for people to visit whilst at the racecourse. As the user walked around, they could see where things were in relation to their current position.

The fact that the user needed to be at the racecourse really meant that there was no need to show the features if the user wasn’t actually there – so the first thing we did was to only show the venues within 500m of the user’s position.

However, since the venues were in close proximity to each other it became apparent that in a lot of situations there were still too many features to display in the view at the same time – so much so that you couldn’t really see everything you needed to.

So, as well as plotting the venues in distance order – nearest on top, farthest underneath, we added the ability to allow the user to adjust the distance within which the venues were shown – down to 50m. This meant that restaurants and bars further away could be “filtered” out of the view, thus de-cluttering the display.

Showing the Venue Names

Each venue’s name also needed to be clearly shown. So we used the same “bubble pins” as used on the map in the app. This also helped to maintain a standardised style throughout the app.

However, on our first trial of the system all the pins were being positioned along the horizon in the camera view.

This was originally intended and in development it seemed to be a logical decision, but in action, in the field so to speak, we found that it was actually quite difficult to see what each pin was meant to be “pinned” to – the pin was obscuring the venue it was meant to be referring to!

So, we quickly decided that the pins needed to appear towards the top of the screen – above the horizon and above the venue in question.

Lessons Learned

This made for a much more user-friendly experience. But if there’s one lesson to be learned from this experience it’s the fact that, as with all development, you can never do enough testing. With AR especially, until you get out and actually try the app in anger, in real-life situations, you can never really understand how it will work.

Royal Ascot iPhone app Proves To Be A Winner

Royal Ascot 2011

Royal Ascot is one of the most prestigious events on the racing calendar that attracts hundreds of thousands of people each year. Although the racing is the focal point, many attendees go for the sense of occasion, the chance to dress up, eat delicious food, drink champagne and possibly to catch a glimpse of the Queen.

So when we were approached by Etherlive to help them develop an iPhone app for Royal Ascot, the brief was very clear from the beginning: it needed to be useful not only to the ardent race goer, but also to the people who go there for the occasion.

We therefore put our heads together and came up with a list of features that would appeal to most people:

• List of races, runners and riders.
• Odds, race results and starting prices.
• Map of the course, showing the main venues and restaurants.
• A Live Reporter feed, giving an insider’s view of the latest news and gossip.
• Going update, News, travel and weather information.
• A list of restaurant menus.
• A song sheet for the traditional singing around the bandstand after each days racing
• A photo gallery, allowing users to upload photographs of their day at the course

Unique Features

We also wanted to take advantage of the iPhone’s special capabilities in some way or other – to do something different that would perhaps put a smile on people’s faces. After much head-scratching, we came up with the idea of allowing the user to randomly select a horse in a race by shaking the phone. This would hopefully appeal to someone who didn’t know that much about racing, but who wanted to have a way to help them choose a horse at random to bet on. The “Shake To Choose” option was born! [Think of it as the modern alternative to using a pin to pick from a list in the newspaper.]

Fit for Purpose

From the beginning, we also had to take into account where the app was most likely to be used. If we had a lot of users at the course using it, which was really one of  the main aims, we had to consider how they would be able to receive the information updates. With upwards of 50,000 people at the course on any given day, the mobile phone networks would already be under considerable strain, so we had to make sure that we only sent the minimum amount of data to the app each time an update was requested.

The app was therefore designed to handle change-only data. This was reasonably easy to do on a per-device basis. All we had to do was give each update a unique id and get the app to remember the last change ID received. The server could then be told the last change ID by the app when a request was received and then send down only the information that had changed since then. The app was also designed to store the data it had received so that if it wasn’t possible to connect to the server at a later time, the app would still work – albeit with slightly older information.

Other Race Meetings

The Royal Ascot app was first made available for the 2010 meeting and the updated app is available on the AppStore here. But Ascot wished to use the majority of features in a second app that could be used for all the other race meetings they held throughout the year. But more about that on a later post…

Update – 20th June 2011

Well, Royal Ascot has finished for another year. But I thought I’d share a few stats we gleaned during the week:

• 10,382 downloads
• 932,229 data requests made (by the app to the server)
• 1,432,668 Push Notifications sent out (each message gets sent to all devices registered at that time)
• Highest place on AppStore was: 2nd in All Sports apps; 75th overall.