News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Tesla has officially launched public Robotaxi rides in Austin, Texas, without a Safety Monitor in the vehicle, marking the first time the company has removed anyone from the vehicle other than the rider.
The Safety Monitor has been present in Tesla Robotaxis in Austin since its launch last June, maintaining safety for passengers and other vehicles, and was placed in the passenger’s seat.
Tesla planned to remove the Safety Monitor at the end of 2025, but it was not quite ready to do so. Now, in January, riders are officially reporting that they are able to hail a ride from a Model Y Robotaxi without anyone in the vehicle:
I am in a robotaxi without safety monitor pic.twitter.com/fzHu385oIb
— TSLA99T (@Tsla99T) January 22, 2026
Tesla started testing this internally late last year and had several employees show that they were riding in the vehicle without anyone else there to intervene in case of an emergency.
Tesla has now expanded that program to the public. It is not active in the entire fleet, but there are a “few unsupervised vehicles mixed in with the broader robotaxi fleet with safety monitors,” Ashok Elluswamy said:
Robotaxi rides without any safety monitors are now publicly available in Austin.
Starting with a few unsupervised vehicles mixed in with the broader robotaxi fleet with safety monitors, and the ratio will increase over time. https://t.co/ShMpZjefwB
— Ashok Elluswamy (@aelluswamy) January 22, 2026
Tesla Robotaxi goes driverless as Musk confirms Safety Monitor removal testing
The Robotaxi program also operates in the California Bay Area, where the fleet is much larger, but Safety Monitors are placed in the driver’s seat and utilize Full Self-Driving, so it is essentially the same as an Uber driver using a Tesla with FSD.
In Austin, the removal of Safety Monitors marks a substantial achievement for Tesla moving forward. Now that it has enough confidence to remove Safety Monitors from Robotaxis altogether, there are nearly unlimited options for the company in terms of expansion.
While it is hoping to launch the ride-hailing service in more cities across the U.S. this year, this is a much larger development than expansion, at least for now, as it is the first time it is performing driverless rides in Robotaxi anywhere in the world for the public to enjoy.
Tesla has scheduled its Earnings Call for Q4 and Full Year 2025 for next Wednesday, January 28, at 5:30 p.m. EST, and investors are already preparing to get some answers from executives regarding a wide variety of topics.
The company accepts several questions from retail investors through the platform Say, which then allows shareholders to vote on the best questions.
Tesla does not answer anything regarding future product releases, but they are willing to shed light on current timelines, progress of certain projects, and other plans.
There are five questions that range over a variety of topics, including SpaceX, Full Self-Driving, Robotaxi, and Optimus, which are currently in the lead to be asked and potentially answered by Elon Musk and other Tesla executives:
- You once said: Loyalty deserves loyalty. Will long-term Tesla shareholders still be prioritized if SpaceX does an IPO?
- Our Take – With a lot of speculation regarding an incoming SpaceX IPO, Tesla investors, especially long-term ones, should be able to benefit from an early opportunity to purchase shares. This has been discussed endlessly over the past year, and we must be getting close to it.
- When is FSD going to be 100% unsupervised?
- Our Take – Musk said today that this is essentially a solved problem, and it could be available in the U.S. by the end of this year.
- What is the current bottleneck to increase Robotaxi deployment & personal use unsupervised FSD? The safety/performance of the most recent models or people to monitor robots, robotaxis, in-car, or remotely? Or something else?
- Our Take – The bottleneck seems to be based on data, which Musk said Tesla needs 10 billion miles of data to achieve unsupervised FSD. Once that happens, regulatory issues will be what hold things up from moving forward.
- Regarding Optimus, could you share the current number of units deployed in Tesla factories and actively performing production tasks? What specific roles or operations are they handling, and how has their integration impacted factory efficiency or output?
- Our Take – Optimus is going to have a larger role in factories moving forward, and later this year, they will have larger responsibilities.
- Can you please tie purchased FSD to our owner accounts vs. locked to the car? This will help us enjoy it in any Tesla we drive/buy and reward us for hanging in so long, some of us since 2017.
- Our Take – This is a good one and should get us some additional information on the FSD transfer plans and Subscription-only model that Tesla will adopt soon.
Tesla will have its Earnings Call on Wednesday, January 28.
Elon Musk shared an incredible detail about Tesla Cybercab’s potential efficiency, as the company has hinted in the past that it could be one of the most affordable vehicles to operate from a per-mile basis.
ARK Invest released a report recently that shed some light on the potential incremental cost per mile of various Robotaxis that will be available on the market in the coming years.
The Cybercab, which is detailed for the year 2030, has an exceptionally low cost of operation, which is something Tesla revealed when it unveiled the vehicle a year and a half ago at the “We, Robot” event in Los Angeles.
Musk said on numerous occasions that Tesla plans to hit the $0.20 cents per mile mark with the Cybercab, describing a “clear path” to achieving that figure and emphasizing it is the “full considered” cost, which would include energy, maintenance, cleaning, depreciation, and insurance.
Probably true
— Elon Musk (@elonmusk) January 22, 2026
ARK’s report showed that the Cybercab would be roughly half the cost of the Waymo 6th Gen Robotaxi in 2030, as that would come in at around $0.40 per mile all in. Cybercab, at scale, would be at $0.20.
Credit: ARK Invest
This would be a dramatic decrease in the cost of operation for Tesla, and the savings would then be passed on to customers who choose to utilize the ride-sharing service for their own transportation needs.
The U.S. average cost of new vehicle ownership is about $0.77 per mile, according to AAA. Meanwhile, Uber and Lyft rideshares often cost between $1 and $4 per mile, while Waymo can cost between $0.60 and $1 or more per mile, according to some estimates.
Tesla’s engineering has been the true driver of these cost efficiencies, and its focus on creating a vehicle that is as cost-effective to operate as possible is truly going to pay off as the vehicle begins to scale. Tesla wants to get the Cybercab to about 5.5-6 miles per kWh, which has been discussed with prototypes.
Additionally, fewer parts due to the umboxed manufacturing process, a lower initial cost, and eliminating the need to pay humans for their labor would also contribute to a cheaper operational cost overall. While aspirational, all of the ingredients for this to be a real goal are there.
It may take some time as Tesla needs to hammer the manufacturing processes, and Musk has said there will be growing pains early. This week, he said regarding the early production efforts:
“…initial production is always very slow and follows an S-curve. The speed of production ramp is inversely proportionate to how many new parts and steps there are. For Cybercab and Optimus, almost everything is new, so the early production rate will be agonizingly slow, but eventually end up being insanely fast.”
BREAKING: Tesla launches public Robotaxi rides in Austin with no Safety Monitor
Tesla Earnings Call: Top 5 questions investors are asking
