News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Elon Musk
SpaceXAI just launched into your kitchen with their new app
SpaceXAI just powered its first consumer app and it predicts what you want to buy.
SpaceXAI just made its first move into consumer AI, and it involves your grocery cart. On June 3, 2026, Gopuff and SpaceXAI announced the launch of Go, a Grok-powered shopping assistant built directly into the Gopuff app that predicts what you need before you even start searching for it.
Gopuff is an instant delivery platform that operates more than 400 micro-fulfillment centers across the U.S., delivering everyday essentials, snacks, drinks, and household items in as little as 15 minutes. It is not a restaurant delivery app or a marketplace. It owns its inventory, controls its warehouses, and handles its own logistics, which means it has built one of the most detailed consumer behavior datasets in retail over its 13-year history.
Go combines SpaceXAI’s advanced reasoning, voice, and image generation models with Gopuff’s dataset of hundreds of millions of orders and real-time cultural signals from X to prepare a suggested cart the moment a customer opens the app. It learns each shopper’s habits and automatically builds a personalized cart based on time of day, location, order history, and real-time indicators. Returning customers can check out with a single tap.
Rather than searching for specific items, users can describe a situation like a game-day party or the desire for a healthy breakfast and Go will assemble a cart automatically. It can also predict when shoppers are running low on items like coffee or paper towels and have them packed and delivered in under 15 minutes. Grok voice integration lets users talk to the app in plain conversational language and check out completely hands-free.
Gopuff co-founder and co-CEO Yakir Gola said: “Today, we believe the greatest friction left in commerce is not delivery or instantaneous access to the essentials customers need. It’s the moment before: the thinking, the deciding, the remembering. We’re combining Gopuff’s demand intelligence with xAI’s frontier reasoning to create an everyday shopping experience that feels like a true extension of you.”
Why SpaceX just made a $60 billion bet on AI coding ahead of historic IPO
The timing carries context beyond the product launch. SpaceXAI was formed after SpaceX completed an all-stock merger with Elon Musk’s xAI earlier this year, folding one of the most advanced AI labs in the world into the same corporate structure as the company preparing what could be the largest IPO in history. SpaceXAI is dipping into consumer-focused AI just as it prepares for its public debut, and while Musk has openly discussed building an everything app, this launch uses Grok to power another company’s product rather than launching a standalone consumer platform. Every consumer-facing deployment of Grok ahead of the IPO roadshow adds tangible evidence that SpaceXAI is not just an infrastructure play but a direct competitor in the AI application layer where OpenAI and Google are already fighting for dominance.
Tesla has introduced an enhanced visualization in its Supercharger navigation system, building directly on the Site Maps feature rolled out a few months ago.
This latest software update adds detailed 3D icons that represent specific vehicle models parked at charging stalls, offering drivers a more precise view of site occupancy and layout.
The Site Maps debuted in Tesla’s 2025 Holiday Update, providing 3D overviews of select Supercharger locations with real-time stall availability.
Tesla supplements Holiday Update by sneaking in new Full Self-Driving version
Drivers could see which spots were open, occupied, or out of service when navigating to supported stations.
Now, the system takes this capability further by rendering accurate representations of Tesla vehicles, including distinctions between models such as the Model 3, Model Y, Model S, Model X, and Cybertruck. These icons appear as lifelike 3D renderings, complete with recognizable shapes and proportions that match the actual cars charging at the site:
Supercharger update now shows type of Tesla at charger as well.
Pretty cool. pic.twitter.com/J3NRSIgM0m
— DennisCW | wen my L (@DennisCW_) June 2, 2026
This refinement improves the user experience during road trips and daily charging stops. As drivers approach a Supercharger, the navigation display now shows not just generic occupied markers but identifiable vehicle types plugged into each stall.
Blue indicators highlight active charging sessions, while other visual cues denote availability or maintenance status. The feature integrates seamlessly with the existing map interface, allowing quick assessment of the best available spot based on vehicle size and positioning.
Tesla continues to expand the availability of these detailed Site Maps across its global network. Initially piloted at a limited number of locations, the rollout has progressed steadily, with more stations gaining support in recent software versions.
Owners benefit from better planning, as the system helps identify compatible stalls and reduces uncertainty upon arrival. The update reflects Tesla’s ongoing commitment to refining its navigation and charging ecosystem through iterative software improvements.
In addition to model-specific icons, the enhanced maps maintain all prior functionalities, such as integration with nearby amenities and energy usage predictions. This ensures a comprehensive tool for efficient Supercharging.
As Tesla’s fleet grows and the network scales, such features play a key role in optimizing the overall ownership experience. Future updates may extend similar visualizations to additional sites and incorporate even more data points for drivers.
With this piggyback enhancement, Tesla demonstrates how small but thoughtful additions can elevate an already useful tool, making Supercharger visits smoother and more informed for its customers. The company is expected to broaden the feature’s reach in upcoming releases, further solidifying its leadership in EV charging infrastructure.
Tesla Full Self-Driving v14.3.3 driver monitoring was reportedly scaled back in recent releases, but a new version that was released in the early hours of June 3 aimed to do a better job of keeping those in control of their cars honest, according to release notes.
The release notes for FSD v14.3.3, via Software Version 2026.14.6.7 added:
“Improved driver monitoring system sensitivity with better eye gaze tracking, eye wear handling, and higher accuracy in variable lighting conditions.”
However, Tesla said this was already enabled in the first rollout of FSD v14.3.3 in late May. We tested it anyway, especially as the Standard Speed Profile seemed less-than-worried about what you were doing during operation.
I decided to try out the Hurry and Mad Max Speed Profiles for this test, and it gave me results that I would have expected. Tesla has evidently ramped up driver monitoring based on the Speed Profile you are using to travel.
The more aggressive the Speed Profile, the more on the hook you will be for taking your attention away from the road. Our testing showed that Mad Max was less likely to allow you to do normal things like change music or adjust navigation without getting an on-screen warning or nag from the driver monitoring system.
Hurry Mode Results
On Hurry, the driver monitoring system on FSD v14.3.3, via Software Version 2026.14.6.7, was more restrictive than Standard but less restrictive than Mad Max. I found that I could scroll through music options for a considerable amount of time, more than 30 seconds:
Roughly :31 between first touching the center screen and getting the first nag
— TESLARATI (@Teslarati) June 3, 2026
Standard gave me about 80 seconds of phone scrolling with absolutely no nags or warnings in a previous test. It is worth noting that this was a previous branch of v14.3.3, but Standard is such a goodie-two-shoes on the road that it is my impression it would not change much.
Here’s an 80-second phone nag test on Tesla FSD v14.3.3.
No alerts, no nagging, no annoyance. https://t.co/1dxvTOw5Cn pic.twitter.com/vYViFpjfoK
— TESLARATI (@Teslarati) May 29, 2026
Mad Max Results
I spent the majority of the drive on Mad Max to see how it truly reacted to the driver having their attention elsewhere. While I did do a short phone test, I am aiming to steer away from those and use the center screen. I think it is a valid criticism that the phone test is dangerous and, not to mention, illegal in Pennsylvania. Changing the navigation and music is a more reasonable, more responsible, and safer test.
With Mad Max being the fastest and most aggressive Speed Profile, I anticipated this being the quickest mode to give me an alert that I needed to look at the road. That was the case with music:
🎥 Testing Tesla FSD v14.3.3 (via 2026.14.6.7) nags on Mad Max https://t.co/qZALU2OujY pic.twitter.com/XddOJ0D47x
— TESLARATI (@Teslarati) June 3, 2026
As well as adjusting Navigation, when I received two nags:
🎥 Testing Tesla FSD v14.3.3 (via 2026.14.6.7) nag while adjusting navigation
Two nags here https://t.co/qZALU2OujY pic.twitter.com/xa3dtaDG1L
— TESLARATI (@Teslarati) June 3, 2026
These nags were more than reasonable, and I think it’s probably good that Tesla is ramping up the driver monitoring. I do believe that it should be relatively strict across all of the Speed Profiles, especially with phone use. When using the center screen, the nag intervals should be based on the speed profile you are utilizing at the time.
These driver monitoring adjustments are a great thing to have while FSD is still under its “Supervised” moniker, but I expect Tesla to continue pushing the limits on what it will allow, especially considering CEO Elon Musk has hinted that phone use is capable with the more recent versions.
You can watch the full drive on YouTube below:
SpaceXAI just launched into your kitchen with their new app
Tesla adds new Supercharger feature for a better idea of what to expect
