News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
-
Tesla has started to deploy the 2026 Summer Update to owners across its fleet, and among the biggest changes are improvements to Navigation, a new startup animation for the Model 3 and Model Y, Caraoke scoring, and new capabilities for Grok.
As the update has started making its way to some cars, we can now see a few of the features operating in real-time. We will show you what some of the new features look like in this article, along with some additional details on what changed.
Not all of the new features in the 2026 Summer Update have quite made an appearance, but some of them have, so we’ll show those here:
New Animation Screen for 3/Y
Tesla is rolling out a new startup animation for Tesla Model 3 and Model Y owners. This is present in Launch Edition and Performance Model 3 and Model Y, but other trim levels do not have anything like this.
The new animation screen on Model 3/Y with the 2026 Summer Update
Apparently color will also be customizable: pic.twitter.com/wSdwggtG3S
— TESLARATI (@Teslarati) July 27, 2026
-
Owners can adjust the color associated with the startup animation to suit their preferences. It is a surprise that more automakers do not focus on this animation for their vehicles; it can be a great first impression piece and make the car immediately feel more luxurious.
Tesla has included this on more premium trims, but it is nice to see it on the Model 3 and Model Y.
Grok Improvements
Grok can now adjust more things in the car outside of the Navigation system. Now, drivers can adjust anything from climate to driving settings by simply speaking to the AI assistant in the car:
@Tesla‘s AI assistant (Grok) with its new vehicle commands in action.
Set the climate, your destination and vehicle settings all in one go! pic.twitter.com/0bBTqaAk6G
— Not a Tesla App (@NotATeslaApp) July 28, 2026
You don’t even have to push a button, either. Instead, you can just say “Hey, Grok,” if you have it enabled. That feature rolled out with the 2026 Spring Update just a few months back.
-
This is a great feature, especially pertinent for the Robotaxi platform, as there will be no buttons inside the Cybercab when it eventually starts giving rides to the public. It also broadens Grok’s capabilities, which were relatively limited in terms of vehicle setting adjustments beforehand.
Tesla briefly offered one Robotaxi part in its Parts Catalog for your personal car, only to remove it just a short time after it was first noticed.
Tesla’s Robotaxi camera washer apparatus was briefly available for purchase on the company’s Online Parts Catalog. The camera washer was first noticed on Model Y Robotaxi vehicles about six months ago in Austin.
🚨 Tesla looks to have installed Camera Washers on the side repeater cameras on Robotaxis in Austin
pic.twitter.com/xemRtDtlRR— TESLARATI (@Teslarati) January 23, 2026
First noticed by Not a Tesla App, the Camera Washer entries appeared for the new “Juniper” Model Y under a category called “Halo,” which has also now disappeared. Interestingly, Halo probably is related to Tesla’s internal “Project Halo,” which was a project that aimed to retrofit customer-owned Model Ys into functional Robotaxis.
This hardware addition would likely be required for the vehicle to operate as a Robotaxi, as the Camera Washer seems to be a non-negotiable part of the vision-based system Tesla utilizes for self-driving efforts.
However, this part has since been removed and is no longer visible on the EPC.
-
Now the true question lingers: Why would Tesla add this Camera Washer to the Model Y parts catalog? Is it planning to make it available for owners to utilize on their own cars for personal use, or will it become a prerequisite for Robotaxi operation in customer-owned cars?
While discussing the upgrade options for Hardware 3 vehicles during the Q1 Earnings Call, Tesla CEO Elon Musk had said that the company could establish small, satellite shops that would upgrade cameras and self-driving computers. Perhaps this same strategy could be utilized for vehicles that want to be included in Robotaxi but do not have the correct hardware.
AI4 is currently represented as capable of unsupervised self-driving, and the same was said about HW3 at one point, only for Tesla to admit last quarter that it would, unfortunately, not be possible. Perhaps AI4 vehicles might need this camera washer as a prerequisite, just as HW3 cars will need that camera and computer upgrade.
This could be the first hint of that’s where we are headed.
Tesla has started rolling out a broader change to the ride experience for its Robotaxi fleet by silencing turn signals, but the change is certainly polarizing.
Tesla has generally made it clear that its purpose-built ride-hailing platform, Robotaxi, will cater to the rider in nearly every way possible. This includes having climate preferences, music, and other personal settings loaded up in the car as the rider enters.
But Tesla is taking it a step further by muting turn signal chimes altogether, a change that appears to be a way to make the ride more peaceful:
Tesla has finally turned off the blinker/turn signal noises in their Model Y Robotaxis.
This is a small update, but will definitely be noticeable for riders. It didn’t really make much sense to leave the turn signal sound on when it’s Unsupervised. pic.twitter.com/ITwx60PAMO
— Sawyer Merritt (@SawyerMerritt) July 27, 2026
However, there are a handful of people who are not thrilled about this change. Turn signals are a conditioned part of the human mind for those who ride in a car regularly.
-
Taking a turn without one feels strange and odd, and not hearing it click while activated could set off some alarms for riders, who might use the noise as confirmation that other drivers know of their intention to turn.
I kind of like it being on, threw me off when riding with Zoox who has had it disabled
— Dan Burkland (@DBurkland) July 27, 2026
Turn signal noises are still audible in customer cars, so if you use FSD in your personal vehicle, you will still hear the turn signal.
The move is certainly one that is unique, but not one that separates it from other ride-sharing services. In a normal car, the clicking sound confirms to the driver that the blinker is active. In a fully driverless Robotaxi, that feedback serves no purpose for passengers, other than peace of mind.
Tesla unlocks in-car web conferencing to extend Google Meet, Teams and beyond
Tesla Summer Update begins rolling out: a look at the new features
