News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Investor's Corner
Tesla Optimus Gen 3 shows off a cleaner, factory-ready design in new app discovery
Renders hidden inside Tesla’s Android app show Optimus Gen 3’s design before any official reveal.
![Tesla Optimus Gen 3 [Credit: Tesla]](https://www.teslarati.com/wp-content/uploads/2026/03/tesla-optimus-gen3-diner.jpg)
Tesla may have accidentally given a first look at its newest Optimus Gen 3 humanoid robot when photos were found in the Tesla smartphone app.
Design files tucked inside a recent Android version of the Tesla app appear to show the third generation humanoid robot next to the older Gen 2.5 prototype. The assets were extracted from the app package by Tesla community member @wholemars, who posted the renders on X late Tuesday night before deleting them. The Tesla Newswire reshared the side by side comparison on Wednesday, and the images have circulated widely since.
Tesla Optimus Gen 2.5 vs Gen 3 comparison via @WholeMars on X
The files are labeled “gen3” and were built as models for Tesla app’s own interface, as validated directly by Grok to be official.
The comparison shows a robot that looks built for a factory line rather than a lab bench. Gen 2.5’s exposed mechanical linkages and gold plating on the knees and shins are gone. In their place, Gen 3 uses matte black fairings on the lower legs, paired with a more contoured champagne gold body. Flexible covers now seal the joint where the torso meets the upper thighs, keeping bearings and moving parts sealed from debris and unnecessary contact.
The body panels fit more tightly, and the hands, which Tesla has said carry 22 degrees of freedom, look far more refined than those on earlier units.
This most recent leak fills a gap Tesla has left open for most of the year. Elon Musk said on March 31 that Optimus 3 was walking around but needed “some finishing touches” before it could be shown, a delay Teslarati covered when Tesla missed its first quarter reveal target. Musk later said Tesla would hold the design back until closer to production, partly to keep competitors from copying it. No reveal date has been announced.
The app itself has been preparing for Optimus for months. In July, code in the Tesla app pointed to a dedicated robot phone key, a consent screen for collecting video and spatial data while Optimus works inside a home, and an alert system for low battery and mechanical faults. Finished 3D models of Gen 3 suggest the interface owners will eventually use is moving past placeholder code toward something Tesla intends to ship.
Manufacturing is moving in parallel. Tesla tore out the original Model S and Model X lines at Fremont this summer to make room for Optimus production, with a planned capacity of one million robots a year. At Giga Texas, the steel frame of a dedicated Optimus factory is nearing completion ahead of a targeted 2027 start, with Musk pointing to an eventual output of 10 million units annually.
News
Tesla Semi lands the biggest electric truck deal in U.S. history
Tesla leads a record 2,500 truck order, but not every truck will be a Semi.
Tesla has landed the largest electric truck order in U.S. history. ZET SCALE, a new alliance of shippers and carriers, named Tesla its primary manufacturer on Tuesday for an initial order of 2,500 electric Class 8 trucks. The deal alone would nearly double the number of electric heavy trucks operating in the country.
According to the press release from Catalyst Mobility, the nonprofit formerly known as CALSTART, Kenworth, RIDE and Volvo were also selected as secondary manufacturers that carriers can pick if their operations call for it. No split between the four brands has been published, so the exact number of Semis in the order is not yet known.
Tesla was selected as the primary OEM for the largest electric Class 8 order in the US by ZET SCALE, a new shippers’ alliance
With 2,500 Semis on order, this will double the entire US electric Class 8 fleet
We’re serious about scale, and ZET SCALE is too!… pic.twitter.com/IiTAdzgken
— Tesla Semi (@tesla_semi) September 22, 2026
Tesla won the top slot through a competitive request for proposals. The alliance, which Catalyst Mobility runs with the Smart Freight Centre, scored bidders on price, range, charging capability and production capacity. Pooling freight demand from founding shippers, including Microsoft and PepsiCo, let every truck maker bid lower than it would for a single fleet. “The Tesla Semi is designed for lower cost per mile operations than diesel,” said Dan Priestley, director of the Tesla Semi program, as noted in the press release.
The financing is built to pull in carriers who have avoided electric trucks. ZET Financial is issuing the purchase order for all 2,500 units and will place them with fleets through a fair market value lease. The trucks will be deployed over the next few years across 10 freight hubs in Los Angeles, Stockton, Bakersfield, Seattle and Tacoma, Houston, Dallas, San Antonio, Chicago, Atlanta, and the Newark and New York area. ZET SCALE says the first order is only the opening round, with a longer term goal of 10,000 trucks or more.
Even if Tesla ends up with only a majority share, it would still be the biggest Semi deal to date. Einride’s 500 unit order in August was the previous record, and WattEV’s 370 truck order in May was the largest California deal at the time. Einride’s CEO has since said he expects all 500 trucks delivered by the end of 2027.
The announcement lands two days before Tesla formally inaugurates its Semi factory in Nevada on September 24. The 1.7 million square foot plant sits next to Gigafactory Nevada’s 4680 cell lines and is designed for 50,000 trucks a year.
Tesla has expanded Grok from an in-car chatbot into a hands-free work assistant. On September 22, Tesla officially launched Grok Bot capability, confirming that drivers can now manage email, calendars, files, chats, and tasks by voice and then hand more ambitious errands to the AI-fueled productivity cheat code.
Grok itself is built by xAI. The new car features split into two layers: Connectors link Grok to outside accounts. Grok Bot, currently limited to SuperGrok Heavy subscribers, can complete multi-step tasks such as placing a usual coffee order, booking a reservation, or scheduling an appointment. It truly puts the driver in a nearly complete hands-free driving and productivity setting, with ironically the only task truly requiring your hands being to touch the “Start Self-Driving” button.
We were granted access to Grok Bot’s Tesla integration a few weeks back, and we’ve been able to do a handful of things with it. On a handful of occasions, we’ve used it to order food and have it ready for pickup slightly later into the evening; we’ve managed to pick up groceries after a day of errands with Grok Bot, and outside of the car, it’s helped with budgeting and even my fantasy football draft.
We’ve been using @Grok Bot in Tesla for a few weeks after gaining Early Access, which we thank the awesome engineers for
Grok Bot makes things much easier across your entire life. From the Tesla, I’ve used it to place pickup orders for my Fiancée and I, we’ve ordered groceries… https://t.co/ZJSLieG1s5
— TESLARATI (@Teslarati) September 22, 2026
Tesla shows another way to utilize it: in their demo, a driver says “Hey Grok,” asks the assistant to check an inbox, and hears that a message concerns a weekend reservation. Grok then scans the calendar, reports no conflicts, and confirms the Tahoe trip is clear. It can also add check-in details to a road-trip itinerary. The point is not novelty chat. It is keeping eyes on the road, or on Full Self-Driving, while the car handles the paperwork of a trip:
.@Grok in your Tesla can now do meaningful work for you
With Connectors, you can manage your inbox, clean up your calendar, or talk through existing files/chat/tasks – all hands-free pic.twitter.com/W1LuybQh0P
— Tesla (@Tesla) September 22, 2026
This Grok rollout is not a gadget add-on as much as it is Tesla’s thesis in software form: the car should stop being a machine you operate and start being a room you occupy.
Connectors and Grok Bot treat the cabin as an office that happens to move, and that has truly been Tesla’s intention for years now. The car has slowly become an extension of a home more than a vehicle. Inbox, calendar, groceries, takeout, and reservations become voice work, not dashboard chores that you need to do before you get in your car.
Responsibility shifts from the driver to the stack, and as many Tesla owners rely on FSD for travel, Grok Bot now handles the monotony of dinner reservations or appointments.
![Tesla Optimus Gen 3 [Credit: Tesla]](https://www.teslarati.com/wp-content/uploads/2026/03/tesla-optimus-gen3-diner-80x80.jpg)
Tesla Optimus Gen 3 shows off a cleaner, factory-ready design in new app discovery
Tesla Semi lands the biggest electric truck deal in U.S. history
