News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Tesla has taken another concrete step toward owning its battery supply chain, and it’s doing so with what is perhaps the most important vehicle in its short-but-storied history.
On September 23, Tesla announced that it has officially built the first Cybercab with cathode material produced in-house at the company’s first cathode plant in the U.S., and the first in the U.S. overall.
First Cybercab made using our in-house cathode material – from the first cathode plant in the Americas pic.twitter.com/X95aVXsT9H
— Robotaxi (@robotaxi) September 23, 2026
Active cathode material is the most expensive piece of a lithium-ion battery cell, and it often accounts for more than a third of cell cost. For years, the industry sourced a majority of it from Asia, but Tesla’s decision to make it in the United States bodes well for the Cybercab project. This is the latest chapter in Tesla’s vertical integration strategy, which began in public at Battery Day in 2020.
At the Battery Day Event, Elon Musk said the company would build a North American cathode plant and overhaul the process to cut costs and waste, while also making some of the most powerful and long-lasting cells in the industry.
The Austin facility took years to appear. Tesla filed permits for “Project Cathode” in 2022 on land near Giga Texas. By mid-2022, the building frame was up and Tesla later invested hundreds of millions of dollars as part of a larger expansion of the Giga Texas plant. The company stated it was operating the first large-scale cathode production facility in North America to supplement 4680 cell production.
One month later, that material reached a finished Cybercab.
Made with nickel cathode manufactured locally at Gigafactory Texas! https://t.co/DqMm5fZV3n
— Elon Musk (@elonmusk) September 24, 2026
The timing of this breakthrough is monumental for the Cybercab program. As Tesla officially launched the first Cybercab rides to the public earlier this month, production of the ride-hailing-geared vehicle is moving forward on the planned S-curve that CEO Elon Musk told everyone to expect.
Nevertheless, packs of Cybercab units have been spotted throughout the United States, in an effort to potentially activate the fleet as soon as the company gains regulatory approval in various geographic areas.
On top of that, Tesla owning the cathode step and pairing it with its own in-house lithium from the Gulf Coast refinery shortens the supply chain that once stretched thousands of miles and subjects every pack to fewer external price shocks and geopolitical risks.
Tesla is not yet independent of all of its foreign suppliers, as some precursor metals come from mines and chemical plants. But the first in-house cathode Cybercab shows the company is closing the most expensive and most concentrated gap in its battery production efforts. For a vehicle like Cybercab to operate at a high utilization within the Robotaxi network, that control over cost is so crucial.
It is arguably as important as the software that drives it.
Tesla is set to inaugurate its dedicated Semi factory in Sparks, Nevada, this evening, but drone footage has already started to show who is expected on the handover stage.
Zanegler, a drone operator who regularly documents Tesla Semi operations in Nevada, filmed a row of new-generation Semi units parked outside of the plant with customer branding applied. The liveries visible in the video include companies like PepsiCo, U.S. Foods, Einride, DHL, WattEV, and others that have committed to utilizing Tesla’s Class 8 electric truck in their fleets.
Tesla Semi Pre-Reveal Revealed!
Excited to share this segment of the drone flight one day prior to the Tesla Semi Reveal event.
They have added branding to several Tesla new generation semis.
Including:
Pepsico
US Foods
Einride
LTS
Nevoya
OK Produce
ABF Freight
DHL
WattEV… pic.twitter.com/KoFOg2FouR— Zanegler (@HinrichsZane) September 24, 2026
Several of those company names already sit within Tesla’s public ordering book, including PepsiCo, which has been the program’s primary anchor customer since the first Semi deliveries in December 2022. PepsiCo now runs dozens of Tesla Semi units in daily service, with drivers completing regional routes.
Einride placed a 500-unit order in August, which was, at the time, the largest single Semi commitment. First deployments of their Semi units are planned for this month and will eventually span across California, Texas, New Jersey, Illinois, and Georgia. Meanwhile, IMC Logistics announced a 50-truck California order this week.
Tesla Semi lands the biggest electric truck deal in U.S. history
Other names, like LTS, OK Produce, and HMD, have not been widely confirmed as firm buyers, but their presence on the lot is perhaps the clearest signal that Tesla intends to expand on that list this evening at the event.
Tesla first unveiled the Semi back in 2017 with a 2019 production target.
Limited pilot builds reached PepsiCo in late 2022, and high-volume production began at the new 1.7-million-square-foot plant that sits logically positioned next to Tesla’s Gigafactory Nevada. Tesla Semi program director Dan Priestley has said that Tesla expects to build “many thousands” of Semi units this year, but as with every Tesla release, there have been set expectations of an S-curve ramp.
Outside estimates still put 2026 deliveries in the 5,000-to-15,000 range, but it will ultimately depend on Tesla’s readiness to put those units out, as well as how smoothly production is moving internally.
Tesla’s Semi Rollout event is set to start tonight at the Sparks factory, and it will be livestreamed on X starting at 6 p.m. PDT / 9 p.m. ET.
The Tesla App is now giving owners a better indication of what to expect when visiting a Supercharger site, as it is now sharing Site Maps in the app itself, making the feature no longer exclusively available in the car.
Tesla owners can now preview a Supercharger’s layout and live stall status from their phone thanks to the App’s new software version, v4.61.0. Zooming into a supported Supercharger on the charging map now switches the familiar street view into a site-specific map that shows stall positions, occupied bays, out-of-service posts, and a live count of open Superchargers.
It will even show you which specific Tesla model is charging at each occupied stall.
You can now see Live Site Maps in the latest Tesla app (4.61.0.) pic.twitter.com/JIL8QzR8hu
— Zack (@BLKMDL3) September 18, 2026
This is the same view drivers first saw inside the car with the 2025 Holiday Update, which started as a pilot for Tesla in California and Texas. Those early maps showed layout, nearby amenities, and the status of each operational stall. The feature has spread to most Superchargers in the U.S., but now Tesla wants the feature available through the Smartphone App, which is now rolling out to owners.
The in-vehicle screen maps remain a richer 3D canvas and originally required AMD Ryzen infotainment hardware, which left many Intel-based Model 3 and Model Y owners without the view. The app version removes that hardware barrier, so any owner can now get a preview of their upcoming Supercharger stop on their phone.
Anyone with the updated version of the app can open a Site Map on their phone. The feature has provided some additional peace of mind to anxious travelers and those who might be looking for a place that is less congested for faster charging speeds. It is a meaningful addition to many owners, especially for trips to unfamiliar locations.
You can now pick a Supercharger location that might be more aligned with what you are looking for or need: you can pick a site based on what stalls are open, where ADA stalls sit, how you will pull in, and more, instead of circling a crowded lot after arriving. Owners of older cars now have access to the feature without having to purchase a new vehicle altogether.
Site Mpas will not reserve a stall, nor will it change how Fast Charging works. Tesla does something simpler and more useful by giving owners a dedicated preview of the Supercharger they’re about to charge, turning an unknown parking lot into a place that may seem more familiar thanks to the preview.
Interested in ordering a Tesla? Use my referral code for three free months of Full Self-Driving (Supervised) here.
Tesla Cybercab gets initial tie-in to localized, in-house cathode plant
Tesla leaks Semi customers ahead of handover event
