News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Just hours after Starship Flight 13 achieved a successful soft splashdown of its upper stage in the Indian Ocean on July 24, Elon Musk announced an ambitious next step for the company’s next launch of the rocket.
“Unless we discover problems after mission data review, SpaceX will attempt to catch the ship with the tower on [the] next flight,” the SpaceX CEO posted on X on Friday.
That “next flight” is expected to be Flight 14. The plan involves returning the Starship upper stage, commonly called the “ship,” to the Starbase launch tower in Texas and catching it mid-air using the same mechanical “chopsticks” arms that have already proven themselves with the Super Heavy booster.
Unless we discover problems after mission data review, SpaceX will attempt to catch the ship with the tower on next flight
— Elon Musk (@elonmusk) July 25, 2026
A successful catch would mark the first time an orbital-class upper stage has been recovered this way, advancing SpaceX’s goal of full and rapid reusability for the entire vehicle.
SpaceX has already demonstrated the tower-catch technique multiple times with Super Heavy. The first successful catch came on Flight 5 in October 2024, when Booster 12 was plucked from the sky by the Mechazilla arms. Subsequent flights, including those involving Boosters 14 and 15, repeated the feat. Several of those recovered boosters were later inspected, refurbished, and flown again, proving the system’s viability for quick turnaround.
Traditional reusable rockets, such as SpaceX’s own Falcon 9 or Blue Origin’s New Shepard, land on legs either on land or droneships. Rocket Lab has recovered its small Electron first stages by helicopter, but those are far lighter vehicles.
SpaceX Starship just nailed something it’s never done before
The China Academy of Launch Vehicle Technology (CALT), a subsidiary of the China Aerospace Science and Technology Corp. (CASC), completed a catch of its booster on July 10. They are the only entity besides SpaceX to attempt and complete the feat.
Flight 13 provided encouraging data. The ship executed a controlled reentry, flipped, and soft-landed intact in the ocean after deploying Starlink satellites, offering the first clear post-splashdown views of an undamaged heat shield. The Super Heavy booster, meanwhile, experienced a harder splashdown in the Gulf of Mexico.
Musk has previously stressed that ship catches would only follow multiple successful soft ocean landings to minimize risk of debris over land.
If Flight 14 succeeds, SpaceX would take a major stride toward routine, rapid reuse of both stages—critical for lowering launch costs and supporting ambitious plans for lunar and Mars missions. For now, teams are reviewing the Flight 13 data. Should everything check out, the next Starship flight could deliver one of the most spectacular recoveries in aerospace history.
In a move echoing its earlier commitment to open innovation, Tesla CEO Elon Musk announced recently that the company plans to make the design and software of its Model S and Model X fully open source.
This follows the same approach Tesla took with its original Roadster, releasing all available design, engineering, and diagnostic materials in November 2023 so that “whatever we have, you now have.”
Just as Tesla made the original Roadster design & software open source, we plan to do the same with Model S & X
— Elon Musk (@elonmusk) July 24, 2026
The Model S, introduced in 2012, was Tesla’s first mass-produced vehicle and a groundbreaking luxury electric sedan. It offered impressive range, rapid acceleration, and over-the-air software updates that redefined expectations for electric cars.
The Model X, launched in 2015, built on that foundation as a high-performance electric SUV notable for its distinctive falcon-wing doors, spacious interior, and advanced safety features. Both models served as flagships that helped establish Tesla as a leader in the EV industry and popularized long-range battery-electric vehicles.
Production of the Model S and Model X was wound down earlier in 2026, with manufacturing ending in the second quarter. Tesla redirected the Fremont factory space previously used for these vehicles toward higher-priority projects, including Optimus humanoid robots and the Cybercab autonomous vehicle.
By the time of Musk’s open-source announcement, custom orders had closed and only remaining inventory was available.
Open-sourcing the designs and software offers several clear advantages. Owners of these aging but still capable vehicles gain better access to technical documentation, diagnostic tools, and software resources, making independent repairs and modifications easier and more affordable.
Independent repair shops and third-party specialists can support the large existing fleet without relying solely on Tesla’s service network. Enthusiasts and engineers can study real-world implementations of Tesla’s battery, powertrain, and software systems, potentially accelerating broader industry progress in electric mobility.
The step aligns with Tesla’s 2014 patent pledge and its overall mission to advance sustainable transport by sharing hard-won knowledge rather than locking it behind proprietary walls.
By releasing these materials now that the models have left production, Tesla ensures continued support for its early adopters while freeing internal resources for future technologies. The open-source release of the original Roadster already enabled simulations, community projects, and deeper technical understanding.
Extending that practice to the Model S and Model X should deliver similar benefits on a larger scale, helping keep these influential vehicles relevant and repairable for years to come
Tesla flexed one incredible Robotaxi metric during the Q2 Earnings Call that skeptics have to hate to hear. The company’s platform has already driven more than 380,000 miles of unsupervised ride-hailing across several states with no notable incidents.
During the company’s Q2 Earnings Call on Wednesday, Vice President of AI, Ashok Elluswamy, said:
“First of all, I’d like to state that the Robotaxi program has been operating extremely well. Especially in terms of safety, the program has had an impeccable safety record. We have driven more than 380,000 miles of unsupervised Robotaxi, now across six cities in two different states. We have had zero notable incidents. Any reports have been of other actors impacting us when we were stationary. I like to emphasize how safe the operation has been so far. Zero notable incidents over 380,000 miles.”
Elluswamy’s claim over Robotaxi miles is a significant milestone for Tesla in the grand scheme, especially considering this is a sizeable number of miles without any incident.
0 notable incidents across over 380,000 miles traveled by Robotaxi
— Tesla (@Tesla) July 22, 2026
Tesla’s self-driving approach is much different than that of other companies. Tesla has maintained that vision is the only thing needed to have a solid and effective self-driving suite. Many self-driving companies utilize things like LiDAR, sensors, and other elements to improve performance, but Elluswamy sent a jab at those who believe it’s needed.
“Historically, the so-called experts have always claimed that you need LiDARs, radars, HD maps, and the entire kitchen sink to drive safely. Here we show that such is not true. You can have safe, comfortable, and affordable autonomy with just cameras. This record should be a huge validation of Tesla’s entire AI approach.”
The feat of accumulating this many miles without any driver behind the wheel is impressive. The thing is, Tesla is also doing this across several different locations, with varying traffic rules, pedestrian levels, weather patterns, and other important factors.
While Tesla is not ready to roll out an unsupervised platform completely, it is a slow but steady indication that the company is well on its way to figuring things out.
The company’s attitude toward expansion is slow, safe, and controlled, and despite this huge milestone, it will still be some time until we see Tesla truly unleash unsupervised rides more aggressively.
SpaceX wants to catch Starship for launch 14, Elon Musk says
Tesla to open source Model S and Model X designs and software
