News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Tesla AI lead Ashok Elluswamy said this week that 24-hour Robotaxi service is close. Replying on X to a rider who wanted Cybercab trips all night, he wrote that the capability would arrive “next month or so” once “the next tech to merge on the v15 plan” is ready.
The comment landed on September 4, one day after Tesla opened public Cybercab rides in Austin. It is the clearest near-term timeline yet for overnight unsupervised operation. Tesla’s paid Robotaxi network currently runs from 6 a.m. to 10 p.m. seven days a week across Austin, Dallas, Houston, Miami, Orlando, and Tampa.
next month or so. the next tech to merge on the v15 plan will enable it.
— Ashok Elluswamy (@aelluswamy) September 4, 2026
That 16-hour window is shorter than the 6 a.m. to 2 a.m. schedule the company used for much of the prior year.
Elluswamy did not name the specific feature or say whether the change would apply first to purpose-built Cybercabs, the existing Model Y fleet, or both. He also offered no city-by-city rollout list. The link to Full Self-Driving v15 is nevertheless significant.
Tesla has described v15 as a step-change architecture with seven parallel improvement tracks and roughly ten times more parameters than earlier builds. Early versions of that software already operate on the Robotaxi fleet and contain about 40 percent of the planned gains.
By July 2026, the unsupervised fleet had logged more than 380,000 miles across six cities in two states with what the company called an impeccable safety record and no notable incidents caused by the vehicles themselves. Tesla has repeatedly argued that camera-based end-to-end neural networks, rather than extra sensors, are the core of the solution.
Overnight service would test that claim in lower-light conditions and would also raise vehicle utilization, a key variable for Robotaxi unit economics. The company has already begun using public Superchargers at night and is building dedicated Robotaxi charging sites.
Riders have asked why software must change if the cars already drive in the dark. The practical answer appears to be reliability and scale: Tesla has held back mass expansion until more of the v15 stack is merged, citing the need for higher confidence before putting thousands of unoccupied vehicles on streets around the clock.
If the next module arrives on the timetable Elluswamy sketched, 24-hour service could begin in October 2026 in at least some markets.
That would mark a shift from a daytime-bounded pilot to a service that can run whenever demand exists, including the late-night hours that have so far remained out of reach.
Tesla is beginning to roll out Full Self-Driving Supervised v14.3.9 with a new active safety layer that can take control even when the driver is operating the car manually.
Tesla AI said the software can activate FSD on the driver’s behalf when an imminent collision is detected and Automatic Emergency Braking may not be enough. It may also engage if the system detects heavy distraction or an accidental FSD disengagement.
FSD Supervised v14.3.9 starting to roll out shortly
This release includes a new active safety feature set: FSD Supervised can now activate on your behalf when an imminent collision is detected and Automatic Emergency Braking (AEB) may not be enough.
It may also engage if we…
— Tesla AI (@Tesla_AI) September 4, 2026
The capability is essentially Automatic Collision Evasion. However, unlike conventional AEB, which mainly applies the brakes in a straight line, this feature can use steering, braking, and acceleration together if the car calculates that stopping alone will not prevent impact and a safer path exists. The system may change lanes or move toward a shoulder when conditions allow, then continue driving after the immediate threat is handled rather than simply coming to a stop.
The intervention is meant as a last-resort safety net, not a replacement for attentive driving.
Tesla Full Self-Driving v14.3.7 early review: FSD saved me from an accident
Tesla’s own description still frames FSD as supervised assistance. Secondary reports on internal release notes say the feature can fire while the car is being driven manually if cabin-camera monitoring suggests the driver is not sufficiently attentive, such as reaching toward the back seat, or if FSD appears to have been turned off unintentionally.
After the emergency maneuver, the car is expected to alert the driver and request a return to manual control.
The safety case is straightforward. Many collisions happen in the last second because a driver is looking away, fumbles a control, or faces an obstacle that braking cannot fully solve. A system that can both recognize that AEB is insufficient and execute a coordinated evasive path can reduce those remaining high-severity events.
Re-engaging after accidental disengagement also addresses a practical failure mode: a small steering nudge that drops FSD at the worst moment. The advantage is a background safety net that uses the same vision stack already running in v14, instead of leaving the car solely to emergency braking once the driver is no longer in command.
The feature still depends on FSD being enabled and, according to reports, an active FSD purchase or subscription. It does not make the vehicle unsupervised. Drivers remain responsible, and Tesla has not published how often the system is expected to intervene or how it will handle false positives.
If the rollout is conservative and the false-alarm rate stays low, the update is a meaningful step: FSD is no longer only a feature the driver turns on. In the rare moments when disaster is already forming, it can step in.
Tesla launched the all-electric, steering wheel-less, and pedal-less Cybercab last night at a quiet and small event in downtown Austin, Texas.
The launch, which marked the beginning of unsupervised ride-hailing for Tesla’s Robotaxi platform with Cybercab, has already caught the attention of the National Highway Traffic Safety Administration (NHTSA) who has more questions.
NHTSA opened an Audit Query (AQ) into the Cybercab’s Federal Motor Vehicle Safety Standards (FMVSS) certification that Tesla gave the vehicle. Manufacturers self-certify vehicles much of the time to avoid excessive regulatory delays.
Tesla Cybercab interior, note the lack of steering wheel and pedals. (Credit: @niccruzpatane/X< /a>)
However, the agency needs more information; it said in a summary:
“On September 3, 2026, Tesla began commercial deployment with a small number of its Cybercab vehicles in Austin, Texas. Tesla notified the Agency that it certified those Cybercab vehicles as compliant with all applicable Federal Motor Vehicle Safety Standards (FMVSS). Tesla also notified the Agency that it plans to gradually expand commercial deployment of the Cybercab to include additional vehicles and locations.”
It also went on to state that the Cybercab lacks traditional automotive controls, which is a groundbreaking move. The process is entirely new to the NHTSA, which gives the agency some leverage to put Tesla’s launch under a microscope:
“The vehicles lack permanently attached, conventional manual controls, such as a brake pedal, gas pedal, steering wheel, and mirrors. NHTSA is opening this AQ to examine the process and technical data on which Tesla relied when certifying the Cybercab and related issues. Among other things, NHTSA will consider the extent to which Tesla’s certification depended on determinations that certain FMVSS are inapplicable to the Cybercab.”
Tesla has added 45 Cybercab units to its fleet of Robotaxi-enabled cars in Austin, according to public documents the company submitted to the State of Texas over the past week. Enabling this level of self-driving is something Tesla has worked toward for many years, and now that it is finally here, it seems more than reasonable that regulatory agencies will have some questions.
Many outlets might try to frame this as a negative, but it is truly an agency looking to gain more information about groundbreaking tech that Tesla has been developing for years.
In an effort to keep riders, pedestrians, and property safe, any and all data accumulated from these first days, weeks, and months of rides will likely be shared with the NHTSA to enable broader rollout strategies across the United States and more in the future.
Tesla Robotaxi will be a 24/7 service: here’s when
Tesla Full Self-Driving will now overtake manual driving to avoid disaster
