How the latest developments of voice activated HMI is changing the user experience. Graham Jarvis reports.
Chris Schreiner, director of UK innovation practiceat Strategy Analytics says speech recognition use has increased from 2016 to 2017. However, the technology saw a slight dip between 2015 and 2016 despite in-vehicle speech recognition systems improving over the past few years. There is, for example, better communication available about the systems’ functions and about which voice commands that enable them to operate. He, therefore, thinks that the move towards voice assistants, as offered by BMW and Nuance’s human voice interface (HMI) system, represents a great step forward. Yet, the proclaimed advancements haven’t quite kept up with rising consumer expectations.
“Consumer expectations have risen faster and OEMs are still not meeting them as they have previously experienced voice assistants such as Alexa and Google Home, which provide a better experience than any in-car system”, he comments before adding: “The issue is compounded by the automotive product design cycle, which as we all know lags behind that of consumer electronics. This leaves cars at an inherent disadvantage, at least until we have over-the-air updates.”
Lars Schultheiss, head of sales and portfolio, business unit infotainment and connectivity at Continental, interjects: “With the increasing use of cloud based speech recognition solutions in the CE world, the users’ expectations in reliability and ease of use are tremendously increasing whilst embedded automotive solutions can still not keep pace with the rapid development of CE solutions. System failures let users lose faith in embedded solutions and create frustration.”
Schreiner says Strategy Analytics has found that: “The vast majority of issues we see in our user testing are design issues. While we still run into some accuracy issues where the user says “Call John” and the system tries to call Jim, it is still less frequent than some of the other problems. The biggest problems are consumers talking over the beep or not knowing what to say. Consumers press the speech record button, hesitate, use filler words like, “Er!”, and then are taken down a black hole of error messages and help prompts. The technology of understanding different regional accents has improved and iFlytek has invested heavily in addressing this issue.”
John Kosinski II, human interaction architect at Brand Motion believes there is more to achieve beyond simplistic marketing: “We have moved from the early adopters (accept issues and adapt) into the general public (must work as expected with a fixed mindset). The user needs to cognitively process more for speech. It has been marketed using simple phrases but until the voice systems learn to engage with a conversation (longer discussion) there will be issues. What can I say? How do I search? We need to better understand how to talk with others, this does not translate to talking with a computer. AI should help with this, I believe more psychology understanding is needed in this field for proper artificial intelligence (AI) operation.”
“Speech recognition programmes are running on machine learning and deep learning algorithms that require a learning period,” says Sahand Malek, a consultant at Ptolemus. So, for every new language, new dialect or accent the algorithm needs to go through a learning process, which users could find slow and frustrating. “So, by having these algorithms running on multiple devices (search engine on different apps, house appliances, etc.) the algorithm will become better at recognising the command”, he predicts.
When we talk about HMI, the discussion shouldn’t just be about voice. Honda, for example, patented a hybrid voice, gesture recognition and eye-tracking platform about three years ago in 2014 with Edge3 Technologies. The aim of HMI systems is to enable drivers to handle an increasing number of sensory inputs, and they are expected to take over more control functions as the industry steps closer to fully autonomous vehicles. The principal aim of the patent was to “allow drivers to more accurately and naturally to control their automobile’s infotainment and climate systems”. In the future such control functions will involve data analytics, sensors and a whole lot more to allow drivers to become just passengers with the ability to instruct a vehicle to carry out an action.
In the meantime there are some pitfalls to overcome, or at least to avoid. According to Adam Emfield, in his January 2017 Autoblog article ‘The 5 Common Automotive HMI Usability Pitfalls’, these include:
1. Poor input (audio and touch).
2. Giving the users too much.
3. Mismatched prompts and dialogue.
4. Trying too hard to avoid errors.
5. Assuming a shorter interaction is a better and safer interaction.
With data protection, user experience, usability and brand promotion in mind, infotainment designers have a hard task to complete. Continental’s Lars Schultheiss, therefore, comments that there are several different ways to have speech recognition in the car:
“Infotainment designers either need to provide a homogeneous user interface that combines different assistants in the background or different assistants are used in parallel, mapped to different services or focus areas. Depending on the approach, users can enjoy a customised, branded user experience in the car or rather use the speech interfaces they already know from using their CE devices. “
He thinks that the different areas that HMI will focus on include: vehicle functions, entertainment (media, radio), connectivity (WhatsApp, SMS, messenger, calls), online services, connected home, online apps and services. They will be handled by cloud based speech recognition solutions.
Schreiner describes the questions of how the integration of digital assistants tests infotainment designers, and whether consumers will ultimately have a plethora of different assistants to choose from as being a hot topic. He explains why this is the case in his opinion: “We are now talking about an embedded speech system and on top of that Alexa, Siri, OK Google, etc. These different systems have to be designed into a holistic system; they all do many of the same things and tasks. So, you need to communicate how to get into the different speech recognition systems and it’s not a trivial problem to communicate what they can do for you.”
Schultheiss offers his view on how to address this complexity: “Dependent on the approach, providing either a single customised user interface or use different ones in parallel, HMI designers have to either go for a homogeneous user interface combining different assistants in the background or for a reasonable integration of existing assistants using their respective user interfaces.”
He also points out that there are some “cultural differences concerning syntax, dialogue flow, order or sequence of dialogues, user preferences, functions etc.” to consider. They play an important role in the choice of the digital assistant and user interface.
Daimler says there are different standards and guidelines across the globe regarding driver distraction, and so it claims that it puts safety down as one of its core values. To meet them it has developed its own standards, but they don’t necessarily fit in with global standards because the company believes its own standards and guidelines are more reliable and more complex. The firm goes so far as to boast that “Mercedes-Benz has higher standards than any country in the world”.
Malek concludes: “Buying a new car is about selling an experience, we passed the era of engine size and horsepower, the new generation wants to integrate their phone to their home to their car and all of them have to about their lifestyle and individual preferences.” So, despite the pitfalls and the need for improvement, the industry is confident that HMI will find a place in all kinds of vehicles and it has a promising future so long as it as a technology matches and considers consumer expectations.
06 Jun 2018 - 07 Jun 2018, Novi, USA
With 150 speakers, a 200,000 sq ft exhibition and 3000 attendees, TU-Automotive Detroit is the world's biggest conference & expo for connected & autonomous cars.