News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
A new report from Reuters claims that a transport authority in Sweden is pushing back against the approval of Tesla’s Full Self-Driving suite because it will travel over speed limits.
The report says the Swedish Transport Administration (TRV) recommends the European Union votes against FSD’s approval. TRV believes it should not be approved until Tesla disables FSD’s ability to speed.
TRV sent a letter to the European Union’s Technical Committee on Motor Vehicles (TCMV), which is set to meet on June 30 to discuss the potential approval of the Tesla FSD suite in the country. Tesla, which has received various approvals in Europe over the past two months, has not provided a comment.
Teslas operating on FSD do travel over the speed limit, depending on the Speed Profile that is chosen. Drivers have the ability to disengage FSD at any point; Tesla specifically states that those supervising the suite are responsible for its actions.
Let’s cut to the chase: humans operating any vehicle speed almost daily in the United States. Realistically, speed limits in the U.S. are more frequently treated as speed minimums. However, other countries are different, and driving behaviors are less aggressive.
TRV believes that “allowing automated systems to systematically exceed legal speed limits…risks undermining both the legal framework and the expected safety benefits of vehicle automation,” the report stated. It’s surprising that Tesla has not received this claim from other countries previously.
This could be a good argument to bring Max Speed back, the setting that previously allowed the driver to choose the absolute fastest the car would travel.
This would still put the responsibility of supervision in the hands of the driver. It would allow the driver to choose whether the car would travel over the speed limit or not, acknowledging that they set the speed, and if they get pulled over, there would be no ability to argue it.
However, it does not seem as if this is something Tesla will do, especially considering many U.S. drivers have requested the feature in an effort to eliminate speeding or at least tone it down. The company has not shown any interest in bringing it back.
Tesla has approvals for FSD in Europe in Estonia, Lithuania, Denmark, the Netherlands, and Belgium.
Tesla is going to let you guide Full Self-Driving with Grok in 3 months, CEO Elon Musk confirmed on X.
The response from Musk, which revealed Tesla plans to allow drivers to effectively control the car and its navigation more explicitly using Grok, puts the feature for about September.
A Tesla owner said that Full Self-Driving is great, but owners should be able to “converse with Grok like we can with an Uber driver.” She then used examples like, “Grok, turn right here,” and “Drop us off right here, we’ll walk due to traffic,” and finally,” Drop at entrance first, then park far away.”
Coincidentally, the final piece of dialogue would also mean features like Banish are potentially on the way soon.
This functionality will be there in about 3 months or so
— Elon Musk (@elonmusk) June 18, 2026
Banish is also referred to as “Reverse Summon,” and would enable the car to self-park while dropping occupants off at their destination.
This would be a great way to improve the overall experience while supervising FSD. Navigation is already a major painpoint that many owners complain about. Manual overrides when a maneuver is requested or canceled (like using the turn signal stalk to override a navigation route), do not always work.
The feature could be especially useful in street parking scenarios in a city, where spots are sometimes tough to come by. Many of us who grab dinner in a more populated area will park a street or two over from wherever we’re going, because sometimes you know that’s the best you will get. If a driver using FSD could say, “Hey Grok, turn right here on Queen St. and park in that open spot on the right,” it could save a lot of confusion FSD might have on its own.
Musk teased that a similar feature was “coming” back in February:
Tesla Full Self-Driving set to get an awesome new feature, Elon Musk says
It is certainly surprising that Tesla is doing it at this point. The company’s more recent moves have been more evident of taking control and inputs away from humans and putting them in the AI’s hands more frequently. The biggest example of this was taking away Max Speed in AI4 cars, giving us Speed Profiles, and not having any input on the fastest speed the car will travel.
Of course, giving navigation preferences to Grok is availble already in Teslas, but not at the drop of a hat. Instead, you can suggest a certain route at the beginning of your drive.
Here’s an example of that from December:
🚨🏈 I am taking my parents and Fiancee to the @Ravens game next weekend and asked @Grok to help me route my @Tesla through a specific neighborhood to reach the correct Lot we will park in.
This is a great example of the new @grok nav integration with the Tesla Holiday Update: pic.twitter.com/rPp4I7q8Yv
— TESLARATI (@Teslarati) December 13, 2025
Finally, the original post that Musk responded to mentioned a parking preference after dropping off the occupants, which describes the Banish feature that Tesla has teased for years.
We’re not sure if Musk was responding more to the ability to guide the car with Grok, or whether he also was including Banish in the three-month prediction timeframe.
A close-up image of a Cybercab engineering vehicle in Peabody, Massachusetts, reveals a compact triangular side repeater camera housing equipped with an integrated washer mechanism.
This seemingly small hardware addition could prove to be one of the most critical components for achieving reliable, unsupervised Full Self-Driving (FSD) — not just for the dedicated Robotaxi but potentially for existing AI4-equipped vehicles as well.
The washer system’s importance cannot be overstated in Tesla’s vision-only autonomy approach. Cameras are the sole sensory input for the neural networks powering FSD, constantly interpreting the environment for safe navigation. In real-world conditions, however, lenses quickly accumulate rain, snow, mud, dust, or road spray.
Many of us Tesla owners, especially those who deal with any sort of winter weather at all, know the all-too-common alert that pops up when cameras are obstructed:
Even brief obstructions can drop perception confidence, trigger safety disengagements, or force the vehicle to pull over, although these are relatively rare. Instead, most of the time, the camera will need a wipe from the owner next time they stop the car.
But unlike human drivers who can manually clear their view, a Robotaxi operating 24/7 without a steering wheel or mirrors must maintain pristine vision autonomously. The Cybercab’s side repeater washer delivers targeted cleaning bursts precisely where needed for merging, lane changes, and blind-spot monitoring — functions that demand uninterrupted visibility from the external cameras:
And this is how the side camera and washer look like on a Cybercab. This is from an Engineering vehicle in Peabody MA. pic.twitter.com/Re8VknpmLM
— Tobias Goebel (Unsupervised) (@tpgoebel) June 17, 2026
This hardware directly tackles a known pain point in current FSD deployments. Owners frequently report camera-related alerts during inclement weather, which is understandable, but needs to be solved for a true autonomous experience.
For a production Robotaxi fleet aiming for high utilization and minimal downtime, robust washer systems represent a foundational reliability upgrade; essentially, they’re a must-have. Early sightings suggest the design may extend to rear cameras as well, creating a comprehensive cleaning architecture that keeps the entire vision suite operational in harsh environments.
Without it, even the most advanced neural nets struggle when their “eyes” are compromised.
What Does This Mean for AI4 Cars?
This Cybercab detail raises timely questions for AI4 cars already on the road. While Hardware 4 delivers superior compute and camera resolution compared to earlier versions, production models typically lack dedicated side and rear washers. Tesla has included them on Model Y robotaxis that it is using in the fleet:
Tesla Robotaxi has a highly-requested hardware feature not available on typical Model Ys
As Tesla refines unsupervised FSD for broader release, the gap in environmental resilience becomes evident. Software improvements can help mitigate issues, but they cannot fully replace physical cleaning in heavy rain or muddy conditions. Analysts and owners increasingly speculate that AI4 vehicles may eventually require similar washer retrofits — or a future AI4.5 variant — to match the Cybercab’s all-weather readiness and support the same level of autonomy.
As testing progresses, the Cybercab’s washer mechanism highlights Tesla’s pragmatic focus on real-world robustness. It may well become the hardware piece that determines how quickly and reliably FSD scales from prototypes to everyday vehicles.
Tesla faces Full Self-Driving pushback in EU over ‘speeding’
Tesla teases greater Grok FSD integration and ‘Banish’ feature ‘in about 3 months’
