Augmented reality's role in the future of navigation, investigated by Graham Jarvis.

Research by Frost and Sullivan suggests that augmented reality heads-up displays (HUDs) will gain 10.4% of the European Union and North American market by 2025. The firm’s June 2015 report, ‘Augmented Reality in Cars In North America and the European Union’, also says that the HUD market is “expected to grow at a compound annual growth rate (CAGR) of 41.9% between 2014 and 2021” across both markets, and it predicts that the technology will launch into them in 2018.

According to the Executive Summary of the report, carmakers such as Ford, Hyundai, BMW, Audi and Mercedes-Benz, will be the most likely to launch augmented reality HUD systems in that year. The technology will only appear in selective models and it will be supported by technology enablers such as Nippon Seiki, Continental and Denso as they are working closely with the carmakers to make automotive augmented reality HUDs happen. It also claims that they are working hard on improving driver safety by integrating ADAS content and warnings into heads-up displays.

AR is not new

Krishna Jayaraman, a member of Frost of Sullivan’s Mobility Team, says his company began to research human machine interfaces in 2010. “It all started because the main idea was to balance driver distraction with what the consumer wants,” he explains. This raised questions about whether there would need to be different modes of input: e.g.  a physical button, gesture control and recognition or voice assistance. The answer to these questions showed a mixed response. How to display the information emanating from a vehicle was their second consideration and augmented reality (AR) was the third element that researcher thought about because it has the potential to offer the car drivers essential and useful information while preventing their eyes from wavering from the road ahead.

Key uses of AR

To reduce any distraction, this information is presented to the drivers on their windscreens. Jayaraman’s colleague, who was introduced to me as Ramnath E, added that there are three key areas where AR is being integrated within the automotive industry:

1.      For use in heads-up displays.

2.      The use of AR in the automotive service industry. He says BMW is using AR for the maintenance and servicing of their vehicles in partnership with Reflekt.

3.      For use is in the retail sector to enabler people wishing to purchase a vehicle to virtually test drive it before they commit to buying it.

“If a customer walks into a retail shop [such as a car showroom], and wants to look at the features, they could have an AR screen in front of them to see the features of a car with a 360-degree view,” he explains.

Benjamin Oberkersch, a leading figure of digital vehicle and future technologies communications at Mercedes-Benz, says his company’s cars have used AR implementations for years. “Two examples are Active Night View Assist and Active Parking Assist… besides AR being implemented within the cars, there are lots of AR technologies used during the production process,” he says before adding that Daimler has gained 10 years’ experience of using AR. He also believes that Mercedes-Benz group, including Daimler, has already shown some possible implementations of AR in navigation. For commercial reasons, he wouldn’t provide any further information.

Superimposing AR

Doug Simpson, founder and CEO of Navdy, nevertheless claims: “There are no existing implementations of full AR where information is superimposed and synchronised with the real world and Navdy is an example of AR 1.0 in the way that Google Glass is.” In other words, the information is superimposed over the real world without any synchronisation taking place.  He agrees though that the most natural way forward for the implementation of AR for navigation is to eliminate the cognitive transition between reading a GPS screen and the road: “The road should be superimposed over the top of the road, so navigating the route is as easy as following the car in front of you… and you can imagine highlighting the office building you are trying to find on the windscreen.”

Vitaly Ponomarev, CEO and founder of WayRay, explains that AR navigation requires many technologies. They include cameras and sensors “to allow us to position the car and to display AR content onto the road, and this content is connected to the location – very precisely,” he says. He says carmakers are using his company’s platform while developing their own interfaces on top of it. “For the cars of the future they want to have a mixture of AR navigation and other content, and they want to get rid of the traditional dashboard to display information on the windscreen,” he explains.

Wearables and HUDs

Jeremy Dalton, VR and AR Lead at PwC, agrees with the consensus that AR for navigation and for other purposes doesn’t require the use of googles or a headset. “No they don’t need to wear googles but it’s either an option or a possibility within the automotive industry as the information will travel will you when you turn your head to the left or to the right,” he comments. With regards to a headset, he claims there wouldn’t be any need for adjustment and when the information is presented on a windscreen the car knows where your line of sight is. This is to ensure that your eyes remain on the road ahead to avoid an accident.

“Generally, you need to have a reasonable amount of processing power to produce high quality rendering and the big disadvantage of using googles or a headset is that you create friction for the user… you have to load it up and headsets can be lost because they’re not attached to the vehicle,” he explains. Most of the headsets aren’t very fashionable yet too. In contrast, a HUD system is there – ready and waiting to go whenever it is needed by you and your vehicle. 

Ponomarev adds: “Headsets still have many problems in terms of their rendering capabilities. You can’t do real-time rendering with them and with the googles because they offer a limited field of view and so we are using a platform that is embedded into the car.” This vehicles ability to use AR is supported by a “very powerful computer that does the rendering,” he says.

Servicing and maintenance

With regards to vehicle servicing and maintenance, Dalton concludes: “Companies such as VW and BMW have been looking at how to make the maintenance of vehicles easier as there are different procedures for different models. AR can be used by someone holding up an iPad to the front of a vehicle to identify it, and then to give step-by-step instructions on maintenance steps, helping to get to an area with the right tools.” Beyond this, he thinks that AR is a “stepping-stone to autonomous vehicles but people lack the confidence in the idea of self-driving vehicles at the moment.” It is, therefore, hoped that AR for navigation and other purposes will demonstrate how safe they could be.

[Mob.Jarvis.07.10]

TU-Automotive ADAS & Autonomous USA 2017

02 Oct 2017 - 03 Oct 2017, NOVI, USA

The most focused forum on the here and now of self-driving technology. As these technologies storm the headlines, we focus on the current challenges and unite players from research labs, automakers, tier 1’s and the complete supply chain to plan for the imminent future.