Disclosure: WardsAuto accepted travel and lodging, paid by Toyota, in order to attend this invitation-only event. Toyota had no direct influence over this story.
Dive Brief:
- At a recent event serving as a first open house for Toyota Motor Corporation’s Woven City mobility test ground, in Japan near Mount Fuji, the automaker revealed its AI Vision Engine, a large-scale AI foundational vision language model that brings together visual, behavioral and environmental data.
- Toyota claims the AI Vision Engine is capable of identifying patterns, detecting potential risks and enabling coordinated action across connected systems to improve safety. It’s set to be a core building block for the automaker’s next-generation Anzen driver-assist systems and Arene software platform for millions of software-defined vehicles produced in a wide range of configurations.
- Toyota sees commercialization of the AI Vision Engine beyond cars. It employs AI to understand and analyze video data and is being trained from scratch, and Toyota says it might also be applied to retail environments, airports or offices.
Dive Insight:
At the Woven City event, attended by WardsAuto, Toyota further touted the capabilities and versatility of its AI Vision Engine, which was built entirely in-house by Woven by Toyota, the automaker’s software subsidiary.
The automaker noted in a release that it’s one of the world’s leading VLMs, as ranked by MVBench Leaderboard. And while Toyota’s software unit had cars and mobility in mind when it developed the AI Vision Engine, it sees applications well beyond that.
The AI Vision Engine is capable of understanding the behavior of people, objects and mobility through camera footage, Woven says. But it does not employ facial recognition. What distinguishes it from other VLMs, officials explained at Woven City, is that it can quickly summarize what is happening from camera video, report on what has happened, or help anticipate what will happen.
“For us, autonomous driving is actually a learning problem,” said Dushyant Wadivkar, Toyota’s global head of AD/ADAS at Woven by Toyota. “It’s less about the algorithm that is specifically programmed for specific scenarios, rather a system that is able to learn and continuously handle the nuanced aspects of driving.”
To solve the learning problem Woven by Toyota is building two core products: a machine learning stack that is designed to learn from drivers, vehicle data and road conditions; as well as an entire ecosystem that powers the stack. Together, they comprise what Wadivkar calls an “active learning loop” that will allow Toyota to extract data from millions of vehicles to train machine learning models, verify and validate data, and deploy it back to customers.
SDVs that receive regular over-the-air updates define a direct relationship with the end consumer. So with the brand’s image at stake, it’s an ecosystem that Toyota wants to oversee every aspect of, including an execution platform that has to comply with regulatory authorities.
In working toward full SDVs, Toyota plans to set up the platform step-by-step over a period of years. As Woven by Toyota’s CTO John Absmeier explained, the automaker started intentionally with multimedia and ADAS, which he said are the two most complex pieces.
“But it’s not just about learning for ADAS purposes,” said Absmeier. “We’re talking about the active learning loop for every purpose in the car, so learning about how users use in certain regions, in certain environments, and under certain regulatory and certification differences.”
Of more than 100 million Toyotas on the road around the world, there are dozens of markets for which Toyota distinguishes product lines and model variations differently to deliver vehicles with the right features and pricing. “This creates a tremendous challenge for us to try to provide software to those millions of units that we have to improve over time,” summed Jean-François Campeau, head of Arene and vice president of Woven by Toyota.
To navigate past the build variations and potential incompatibility woes, Arene enters what Absmeier calls a “post-domain era,” with software tools that don’t care about the hardware and can basically test and develop software independently of hardware.
The machine-learning stack is designed to be “generic,” emphasized Wadivkar, who has previously focused on autonomous driving products at Bosch, Cruise and Torc Robotics, so it could be deployed on any car. “We’re not designing a specific application for a specific product, but we’re designing an application system that can work on the multitude of cars that Toyota and Lexus have,” he said.
As part of a vehicle safety system, Wadivkar pointed out that you don’t need a separate application for a five-sensor environment and another for a six-sensor layout, or to redesign the system if using a different lidar supplier.
Absmeier said Toyota plans to deploy Arene globally and across its product lineup “over the coming years,” with the system going into everything from “very, very low-end cars to very, very high-end cars, and in every region of the world.”
Arene debuted in the 2026 Toyota RAV4 for the vehicle’s infotainment and safety systems. The Lexus ES will arrive later this year with similar deployments of Arene, and Toyota has said its next-generation EVs will be built on it.
While the active learning loop is going to be fully automated, Toyota at present always has a developer supervising the whole process, according to Wadivkar. Toyota says that although it’s actively testing its Arene-based systems in California, Michigan and Japan, at Woven City it can develop new ideas, and it has willing participants in ”weavers” from the city — people who are there, perhaps with their families, to build things with participating companies.
Toyota is also tapping its team of professional master drivers normally tasked with product development for help in identifying what “good driving” actually is, which Wadivkar recognizes is challenging and situational. “What is good driving is a very difficult question; but once we get this input from these experienced and highly elite drivers, we are able to curate and use our data very effectively,” he said.
