A tour guide dressed like Michelangelo greets you at the entrance of the Vatican Museum. He’s funny, charming and knows the answer to pretty much any question you have. But he’s also not real, he’s a hologram. Let’s teleport to another scene where you’re in the market for a new home. You’re greeted by a realtor who asks you what you’re looking for in an ideal home and then walks you through the house giving you a complete overview of the features and history of the house. She’s helpful, knowledgeable and as you might have guessed, also a hologram.
This is the dream that got us most excited about augmented reality when we began our journey, at Placenote, with a guided tour of an office. Over the last year we’ve seen the AR community begin to connect the dots to this idea in a remarkable way. I wanted to shed some light on this trend, highlight interesting products built around it, and list the tools available to you, as a developer, if you’re interested in building something like this today.
1. Augmented reality as a UI for navigation
As developers, we tend to think of our apps and websites as independent entities from the real world. In augmented reality, you can now create digital content all around your users and essentially “immerse” them in your app. It’s an understatement to say this requires a complete rethinking of our design conventions from the 2D world.
Augmented reality (AR) allows you to highlight a point of interest directly in the real world to give users a much quicker way to understand contextual information. The downside of AR for location-based apps is that you run the risk of overwhelming your users with “too many” visual cues. Users have to work 10 times harder to search your maps for information that could pop up all around them.
2. The BIG difference between navigation and exploration
The standard model of navigation apps has always been a very utilitarian experience with one goal — getting people from point A to point B. With AR, finding your way doesn’t need to be a mechanical task anymore — It can be fun, educational and exploratory. Your digital tour guide could craft an itinerary for you, based on what it knows about you and then create a rich guided experience.
3. Augmented reality as a digital assistant.
Augmented reality (AR) could let us build bots that can communicate across a much larger range of channels — voice, body language, facial expressions and physical movement. I’m talking about importing the best parts of human face-to-face conversations into digital interactions with computers. Even voice-enabled bots like Alexa don’t really hear us. Rather they read a transcription of our words to understand us.
There is a strong need for guided location-based experiences that isn’t being met by navigation apps. AR can bridge the gap between the utility of navigation and the fun of exploration. In the next sections, I’ll show you how to build an AR digital assistant and what the current progress in this space has been so far.
Building an AR tour guide app boils down to three main components:
- Character design
- Story design
- Location awareness
Character Design (Animation and AI)
Building animated characters is a well established field in the film and gaming industries. Companies like 8i and Quantum Capture create volumetric scans of real humans with custom camera setups to record and replay animations in VR and AR apps. Other companies like Mind Show and Limitless are making it easy to animate 3D cartoon characters, rather than real humans for immersive environments.
Digital assistants should be able to interact with users and respond to questions. Amazon has built a product called Amazon Sumerian that uses the same AI engine that powers Alexa to create animated digital assistants for VR and AR apps. Perhaps, alternatively, we might use something like Facebook’s Social VR to create live telepresence sessions with animated characters embodying real tour guides remotely guiding people in AR.
Story Design
“Story design” is the part of the stack that defines the “story” or sequence of actions in an AR experience. In guided tours, this means the sequence of tour stops, story path alternatives and scripted content that will be played along the tour. There are a few startups building products that make it easy to design simple guided tour experiences on mobile and they each have built their own unique story design tools.
Location awareness
The final piece of the guided tour puzzle is “location-awareness” – rendering visual cues over the physical world in 3D requires a very accurate and responsive location-awareness system. This has traditionally been done in the 2D world with GPS, which works well enough for 2D location-based apps but doesn’t cut it in AR. Further, GPS doesn’t work indoors.
