Waymo, the driverless ride-hailing arm of Google parent company Alphabet, has now launched a new AI research model for its self-driving operations.

In a pair of press releases on its approach to AI and its new end-to-end multimodal model for autonomous driving, dubbed EMMA, Waymo has shared details about its plans for the AI research model going forward. The company says it is still using the EMMA model in research stages, rather than in operational vehicles, and the approach comes as an alternative that looks a lot like Tesla’s Full Self-Driving (FSD) and other end-to-end model approaches.

“EMMA is research that demonstrates the power and relevance of multimodal models for autonomous driving,” said Drago Anguelov, VP and Head of Research at Waymo. “We are excited to continue exploring how multimodal methods and components can contribute towards building an even more generalizable and adaptable driving stack.”

Waymo says the EMMA model uses real-world knowledge based on its Gemini language model, while the end-to-end approach is expected to eventually let autonomous vehicles operate directly from sensor data and real-time driving scenarios. The company has also highlighted its use of Large Language Models (LLMs) and Vision-Language Models (VLMs), calling its architecture the Waymo Foundation Model.

Hear the company’s executive detail the Waymo research and AI program more below.

EMMA research and criticisms

In the announcement press release about EMMA, Waymo lays out the following as key aspects of the research program:

1. End-to-End Learning: EMMA processes raw camera inputs and textual data to generate various driving outputs including planner trajectories, perception objects, and road graph elements.
2. Unified Language Space: EMMA maximizes Gemini’s world knowledge by representing non-sensor inputs and outputs as natural language text.
3. Chain-of-Thought Reasoning: EMMA uses chain-of-thought reasoning to enhance its decision-making process, improving end-to-end planning performance by 6.7% and providing interpretable rationale for its driving decisions.

“The problem we’re trying to solve is how to build autonomous agents that navigate in the real world,” says Srikanth Thirumalai, Waymo VP of Engineering. “This goes far beyond what many AI companies out there are trying to do.”

Still, some have cast doubt on the large-scale end-to-end model, saying that it may be too risky to utilize generative AI models without including significant safeguards.

“It’s bandwagoning around something that sounds impressive but is not a solution,” said Sterling Anderson, Aurora Innovation’s Chief Product Officer, in a statement to Automotive News.

Mobileye CTO Shai Shalev-Shwartz called end-to-end approaches “a huge risk,” especially regarding the verification of decision-making process for vehicles operating on the model. It’s also worth noting that Waymo is currently only researching the approach, and it doesn’t currently have any plans to make it commercially available.

The news comes after Waymo recently closed on a $5.6 billion funding round, effectively bringing the company’s valuation up past $45 billion. The company is also working on its next generation of self-driving vehicles based on the Hyundai Ioniq 5, built at a new factory in Georgia.

Waymo hires former Tesla Executive 

What are your thoughts? Let me know at zach@teslarati.com, find me on X at @zacharyvisconti, or send us tips at tips@teslarati.com.

• https://shop.teslarati.com/collections/tesla-cybertruck-accessories
• https://shop.teslarati.com/collections/tesla-model-y-accessories
• https://shop.teslarati.com/collections/tesla-model-3-accessories