The publicity around Google Glass has brought new attention to the idea of augmented reality (AR), which connotes the idea of overlaying information onto the physical world. These certainly are not the first glasses for doing so, and other products are technically superior. But Google’s size and interest in the field through other initiatives such as project Tango and Ingress are helping to bring new attention to the field.
Glass-based displays promise to make it easier to do many things like navigation, fixing things, and recalling details easier. However, this kind of approach is still in its early stages and faces considerable technical, social, and ergonomic challenges before it is ready for mass-market adoption.
Other approaches for implementing augmented reality on smart phones and tablets are here today, and providing value in the real world. Thousands of companies are using AR platforms for improving products, games, medicine, education, and print enhancement. A number of AR development platforms and ecosystems have emerged to help streamline the creation of these types of applications.
Semico Research predicts that revenues related to augmented reality could grow to $600 billion by 2016. The company notes that over 864 million high-end cell phones will be AR enabled by the end of 2014, and that AR capabilities will be baked into 103 million automobiles by 2020.
Michell Prunty, Consumer Analyst at Semico noted,
We’re at a time when technology is accelerating so quickly it’s hard to keep up. QR codes, NFC codes, AR tags, every day there is something new. But AR isn’t just a new fad that will only impact marketing firms. It’s a new way to see and interact with technology that everyone should be aware of. There is going to be an increased need for new software platforms, video and audio processors, NAND, and mobile DRAM. If you’re developing for the consumer or automotive industries, you must be involved with this market early on.
So what is augmented reality
The use of the word AR is still a little fuzzy at the moment, which is great for marketing but a little challenging for understanding. At a high level, it might be considered to be tools for overlaying information onto the physical world, which can include QR codes, NFC codes, beacons, and image recognition techniques. But executives interviewed at many AR tool vendors believe that the term should be reserved for more sophisticated experiences.
Bess Besecker, CEO of Marxent Labs said,
The reason we are all in this space is we are trying to create the Star Trek Holodeck. I don’t think we are far away from where designers and contractors are developing applications for visualizing something in physical space as common practice.
The most common use cases in the near future will lie in building better smart phone applications. Jack Dashwood, marketing and sales executive at Metaio explained,
AR is really still in its infancy. While dedicated researchers have been working in the field for over a decade, the technology is still evolving extremely rapidly. One of the major milestones was the introduction of smart devices like the iPhone, which freed AR from bulky laptop/webcam setups and probably for the first time, put AR apps in the hands of the average user. The techniques involved are really rooted in the larger field of Computer Vision—which has broad applications in everything from robotics to data collection, to car navigation.
There are really two key elements involved in creating an AR application said Martin Herdina, CEO at Wikitude: recognition and augmentation. Recognition can be as simple as reading QR codes or more complex processes involving image recognition and associating location with GPS, accelerometers, and gyroscope sensors built into smart phones. There are also tools for helping to recognize things, objects, and people. He argues that technically these problems have been solved. “But it is difficult to recognize the same building under different weather conditions when it is rainy or foggy. This can also be done on a case by case basis.”
Overcoming the technical limitations
The head-mounted displays and glasses are also seeing significant improvements. Dashwood said,
Head-worn displays like Glass are finally getting to a size that means they can be worn on a day to day basis. In addition, the display being right over the user’s field of view is one step closer to a truly immersive AR world. On the other hand, these devices are super-compact, which means the processors are often less powerful and battery life is more limited.
A number of vendors are rolling out see-through glasses that promise to make it easier to overlay virtual objects onto the physical world. These include products like Epson Moverio, Meta Space Glasses, Optinvent ORA, and Lumus Optical DK-series. Other products are focused on head-mounted displays for rendering information outside of the field of view including Google Glass, Vuzix M100 Smart Glasses, Recon Instruments Jet, and Oakley Airwave ski goggles.
Wikitude’s Herdina explained,
Google glass is a head mounted display, but is not suited for AR as the screen is above your eye rather than in front. The only way to achieve AR is by turning on the camera and augmenting that. This causes huge troubles with heating and power consumption. Moverio provides a see through display in front of your eye so you can augment reality in the core sense. Depending on the use case, you have to choose what device you are looking for. For warehouse workers needing to see a list of items to pick, Google Glass or Vuzix is fine. But if you are a mechanic and need a real time overlay of something you are working on, then you need a see-through display like Moverio.
Get ready for the holodeck
A number of AR tool vendors have emerged to make it easier to develop good AR-enabled applications including Marxent Labs, Metaio, Wikitude, and Layar. Most AR applications today require a dedicated AR-enabled browser app. This may change with the adoption of the WebRTC standard, which provides enhanced support for phone sensors and 3D rendering within the browser. In addition, the ARML2 standard promises to provide better cross-platform support for AR-enabled applications.
These kinds of improvements are likely to lead to better implementation of AR on all sorts of devices. Marxent Labs’ Besecker explained,
The idea of visualization in context has been around for a while. Mobile devices can make it more ubiquitous with the use of image tracking. Theoretically you can go into any room and track things.
He expects a major leap in the types of AR applications developers can implement with improvements in extended tracking, which makes it possible to overlay virtual imagery that maintains fidelity as someone pans the camera across the scene. For example, Azek, an exterior building products manufacturer, has created an app for overlaying a mockup of a new deck onto a real house.
Besecker said that Azek has been trying to make it easier for contractors to demonstrate their products using PowerPoint, videos, or Photoshop mockups. “This is another step in the continuum of visualizing in a powerful way. I think this is a problem that people already have, and they are realizing that AR gets them a step closer to what they are trying to do.”
Herdina said the majority of Wikitude developers are using the technology on smart phones and tablets. A few years ago, it was mostly used as a marketing gimmick. He is now starting to see enterprise software vendors like SAP and Hexaware use AR to streamline work processes. Harvard University has been investigating AR to improve surgery. There is also considerable interest in using AR to improve the print magazine experience. AR also shows considerable promise for creating interactive tour guides.
Working out the bugs
There are three main concerns about the widespread rollout of more immersive approaches to AR:
- perpetual distractions
The right to be ignored
The publicity around Google Glass has done a lot to catalyze fears around the growth of the surveillance society. Bars and restaurants have started banning them. To address these concerns Google has published rules for Glass etiquette. While people have become comfortably numb to the privacy implications of ubiquitous cameras on cell phones, the distinctive look of head-mounted displays has raised the bar for scrutiny and concern. As Mat Honan a Google Glass user wrote about his year of wearing Glass, “I made people very uncomfortable. That made me very uncomfortable. People get angry at Glass. They get angry at you for wearing Glass.”
Will augmentation diminish reality
Having a constant screen in view only promises to accentuate the disconnect we sometimes feel when someone’s phone rings during a pleasant conversation. As Nicole Bonilla at Cornell wrote,”But just because users don’t have to look down at a phone doesn’t mean that they aren’t getting distracted. In fact, this added layer of ubiquity could potentially make the technology more dangerous.”
This relates closely to the idea of “continuous partial attention,” (CPA) which refers to the process of dividing one’s attention among multiple sources of incoming information, focusing a limited amount on each one, so as not to not miss anything. While we exercise CPA often enough with smartphones and other technologies, our distraction could prove even more continuous with a hands-off product like Google Glass. Given our finite attention span and proven inability to effectively handle multiple information streams, increased instances of CPA could pose serious safety concerns.
Application developers will need to consider designs that minimize the potential for endless distractions. Metaio’ Dashwood explained,
The technology itself does not dictate one way or another whether to crowd up the real world with distractions. We will probably see a great deal of experimentation until developers find the sweet spot between valuable content and information overload. Some great minds will get it right, whilst others will fail – but that’s all part of the innovation process.
It is not entirely clear that wearing see-through AR glasses for long periods won’t lead to motion sickness caused by delays in tracking and rendering 3D scenery onto the physical view. This could be a real problem for workers expected to wear them for long periods. These issues were addressed in more detail in a recent TorqueMag.io story on immersive displays.
Motti Kushnir, CEO of Infinity AR believes these problems will not be as significant as with fully-immersive VR since there will be greater correlation between the physical world and the virtual one. However, the compute power of AR-glasses is not currently at the level of gaming consoles used in immersive displays.
New chip designs from Qualcomm, Intel, and Metaio promise significant improvement for AR applications. Also better algorithms from companies like InfinityAR for capturing 3D and rendering overlays in real time could also help to meet the 30 millisecond delay threshold that Kushnir believes is required for an acceptable experience.
But even with these improvements it’s not clear that the latency and ergonomics issues that have affected immersive displays will be solved for consumer grade hardware any time soon. They have been reported to be one of the bottlenecks in the rollout of the $600,000 AR display in the latest US fighter jet.
The augmented future
Going forward, Metaio’s Dashwood expects improvements in AR to lead to more realistic AR experiences in a more natural way. He explained,
We can look forward to AR applications that work in a much more passive manner. Right now, the user must almost always be alerted that there is an opportunity to experience AR before they actually fire up a device and scan. In a future ‘Always On’ world, the devices will be advanced enough to be constantly monitoring the user’s environment for points of interest, curated information, or warnings without the user needing to care about it. We will get to the point where smart devices do all the legwork and only bring something to our attention when it is relevant.
In the long run, the real success of AR will come when it becomes hidden in the background rather than a marketing gimmick. Wikitude’s Herdina explained that in 2009 the AR field was attracting considerable hype while the actual market was fairly small. Now that AR is starting to get significant traction, the most successful developers are not calling applications “AR enabled.” For example, Ford talks about the interactive car manual without highlighting the underlying technology. He said, “The whole technology has gone away and the interest has moved over to the real value being created.”
George Lawton has been infinitely fascinated yet scared about the rise of cybernetic consciousness, which he has been covering for the last twenty years for publications like IEEE Computer, Wired, and many others. He keeps wondering if there is a way all this crazy technology can bring us closer together rather than eat us. Before that, he herded cattle in Australia, sailed a Chinese junk to Antarctica, and helped build Biosphere II. You can follow him on the Web and on Twitter @glawton.