The technology needed to electrify and automate cars is coming of age in tandem with the technology that powers augmented reality. Automakers keen on capturing the attention of their customers, and even attracting alternate forms of revenue, are considering the potential use cases of AR, now and in the future. The integrations that are coming to market today are less gimmicky than you’d expect, given the metaverse-fueled hype around AR, and are actually poised to be useful to drivers.
Augmented reality safety features that can help drivers with navigation and detecting possible threats on the road are the first applications that the auto industry is seeing, and those deliver value now, providing an on-ramp to a future-proof business that anticipates a driverless future.
A handful of software companies are racing to offer products for the growing sector and integrate their tech with OEMs.
On Wednesday, Basemark, a Finnish company that specializes in automotive software, announced that its AR over-video application would now be available in some of the latest BMW iX models. The next day, Harman, a subsidiary of Samsung that specializes in connected car technology and other IoT solutions, said it would be acquiring Apostera, a company whose AR software would help Harman expand its automotive offerings to provide an AR platform-as-a-product.
“For us this was a really important acquisition, just because we feel there’s a tremendous change going through the industry, and it’s not only electrification,” Armin Prommersberger, senior vice president of product management in Harman’s automotive department, told TechCrunch. “Electrification is just the starting point. What’s really happening from our perspective is consumerization. The car is becoming much more than a transportation device from A to B.”
Basemark’s integration with BMW and Harman’s acquisition of Apostera not only signal some of the players in the industry that are making their mark, but also what use cases we can expect to see from augmented reality in vehicles.
Applying AR to video feeds for safer navigation
Drivers with certain BMW i4 vehicles will experience augmented reality directly through their infotainment screen while they are using BMW’s navigation system, according to Tero Sarkkinen, CEO and founder of Basemark. The vehicle’s front-view camera will automatically send live footage of the street in front of the car to the touchscreen, where interactive arrows appear over the real-world environment to show the driver when and where they need to turn or if they should switch lanes. The screen will be split to also show the map alongside the video.
There are of course other applications for this kind of tech, like heads-up displays (HUD), which appear over windshields so drivers don’t have to take their eyes off the road, Sarkkinen says.
Apostera already has a HUD product in production with Audi’s Q4 e-tron, which is reactive enough to accurately stick to a driver’s real environment. The AR windshield on the Audis displays important information like the driving speed, traffic signs, the status of the driver assist system and navigation symbols as static displays. Drivers will also be able to perceive floating symbols to be about 30 feet away, and those will alert drivers to things like lane departure warnings or highlight an active car driving in front when in adaptive cruise control mode.
Basemark is working toward using sensors to give drivers more information about their surroundings. At CES, the company displayed its AR with object detection prototype, which takes in raw camera and radar data and performs sensor fusion to help drivers when there’s low visibility to increase safety.
Basemark is currently conducting pilots with other OEMs, so we might be seeing more of them in the future.
Augmenting advanced driver assistance systems
Many of the impressive ADAS features on new vehicles are taking over an increasing number of driving tasks, but that doesn’t always put drivers at ease, says Andrey Golubinskiy, former CEO of Apostera and now senior director of Harman’s ADAS strategic business unit.
“If you’re turning on some functionality, you as a driver don’t have an understanding of why the car behaves in a certain way,” Golubinskiy told TechCrunch. “So we solve it in the way that we constantly visualize what the car sees and provide object recognition for the driver. We also visualize what the car thinks, so you know what the car is going to do next.”
Harman’s software offering, with Apostera’s IP integrated, will compute this information and visualize it for the driver and passenger so they know why the car might suddenly change lanes or try to avoid an obstacle. This helps to increase the trust, and therefore the usage of the already available ADAS systems in vehicles, says Golubinskiy.
But it doesn’t stop at visual cues. The new ADAS unit at Harman is also working on adding audio warnings to accompany visual object recognition.
“That’s pretty helpful because everything that goes through the ears ends up in your brain much faster and is processed much faster than any visual,” said Prommersberger. “So this kind of immersive combination of augmented reality, mixed reality, visuals and audio is the next evolutionary step. And now with our acquisition of Apostera, we have all the components in-house.”
While Harman says its AR platform is already in the market with some customers, it’s able to continuously onboard new features with over-the-air updates. The company says its product is hardware, operating system and sensor-agnostic, so it’s designed to reuse data that’s coming in from any vehicle network.
The passenger’s AR experience
Down the line, in a future where self-driving is the norm, AR will allow passengers to engage with the surrounding of the vehicle, says Golubinskiy.
“For example, you might drive through the Alps and see some beautiful churches or lakes, and you can engage with a touchscreen on the windows and get a different level of interaction,” said Golubinskiy. “The information is actually projected, and with a touchscreen you can touch, for example, the mountain and get information about the height or other information.”
With in-car tech comes subscription offerings
Cars with augmented reality options are still few and far between, but the trend we’re seeing with other in-car tech at the moment is to monetize smart vehicle offerings through subscriptions. At CES, Google and Amazon introduced new in-car features that enhance a car’s infotainment systems, even allowing passengers to stream YouTube videos or Amazon Prime, things that are likely to become available through a monthly purchase.
Last year, General Motors said it expected its in-car subscription services for things like OnStar, emergency services, ADAS systems and navigation to generate nearly $2 billion in revenue, with a potential to make as much as $25 billion for the company by the end of the decade.
In a world where pandemic-related supply chain issues are causing delays in new vehicle production and automakers are spending more on electrifying, it’s easy to see how AR applications might be used to give automakers a new avenue to increased profits.