News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
-
Tesla released Full Self-Driving version 14.3.6 last week, and after what was potentially one of the best FSD releases in v14.3.5, there has been a bit of a regression. While there are some bright spots, the changes made to v14.3.6 seem to have backtracked some behaviors.
Overall, it is hard to really complain about FSD in any sense; it has revolutionized how I travel literally anywhere. According to my self-driving app, the last time I went a day without using it was 59 days ago.
However, I think it’s also important to recognize when things are just plain bad with FSD. There are times it does truly mind-boggling things, and I’ll dive into those here. Additionally, I only had these issues on local roads, not on highways. Highway operation, generally, is always incredible other than the occasional complaint about speed or left lane camping.
With those things being said, my personal experience may not represent others’ experiences. A handful of people have said they have had a similar experience on v14.3.6, while others have said it is more than normal.
Turning Hesitancy, Inaccuracy
I’ve noticed more inaccuracy turning into multi-lane stretches of road than in any version I can remember. I’ve had at least three instances of FSD turning into a stretch of roadway that has two or more lanes, and not selecting a lane confidently as it has in past versions.
I think Tesla FSD v14.3.6 is a regression from previous .3 branches
This version in particular has been incredibly hesitant, jerky, and indecisive at times. There’s actually been two drives that I have decided to not used FSD for the remainder of the trip. These were local trips… pic.twitter.com/YobHDOKRS6
— TESLARATI (@Teslarati) July 26, 2026
-
Instead, the car will drive over one of the dashed road lines, and the steering wheel will jerk back and forth before picking the lane. It should be said that it has always picked the correct lane when choosing based on the navigation, but it is still very indecisive. The steering wheel jerking is reminiscent of some of the later versions of v13.
I admit I really hate to see the steering wheel jerking come back. However, I think when Tesla releases v14.3.7, it won’t be present. When there are occurrences of it in FSD versions, it is usually resolved by the following release.
FSD Disregards Manual Turn Signals
This is my biggest bone to pick with FSD other than Navigation issues, but this one seems like it would be such an easy fix.
If Tesla is going to put the word “Supervised” on the end of “Full Self-Driving,” then when I tell the car to do something, it should do it. If I input an increase in speed by pressing the accelerator, the car will immediately respond. It does not disregard my input because it feels it is traveling at the right speed.
FSD should never disobey and turn off turn signals that the driver inputs. Trying to direct the car into the correct lane, I had initiated the left turn signal not once, not twice, but three times, with the car turning it off all three times and continuing in a lane that would end in just one block. The only solution at this point would be to zipper merge.
This goes back to the fact that self-driving’s biggest bottleneck might be rider preference. A zipper merge might have been more than reasonable, might have saved me time that I spent sitting through an additional light cycle, and might be something many drivers would do. I was in no hurry, I traditionally do not try to zipper merge because it feels inconsiderate, and lastly, the car should have just followed my input.
-
This caused me to disengage and drive manually the rest of the way home. Sometimes I just do not need FSD to try to pass every car it can at intersections.
Bird Braking is a Thing of the Past
The big complaint with recent versions of Full Self-Driving has been what we’ve coined as “bird braking,” which is when the car will brake suddenly as a bird flies past.
There have been zero issues with this so far in v14.3.6, which is an excellent improvement.
FSD Might Already Be Taking Note of Driver Preferences
Another thing I have noticed over the past few days is that v14.3.6 seems to already be taking my preferences with navigation into account.
This is something that is supposed to be rolling out with the Summer Update, but I have a hunch it’s already present and might have been included in this v14.3.6 build. On Friday, FSD pulled into an entrance to a local convenience store that it had never attempted to go into before.
Typically, I manually pull into this entrance because it avoids heavy cross traffic at the main entrance. FSD has always chosen that congested main entrance.
I don’t want to get anyone excited but my Tesla running FSD v14.3.6 just:
✅ Pulled into an entrance at my local Sheetz that I routinely pull into manually but FSD never has entered
-✅ Pulled into my assigned parking space at home despite many other spots being vacant https://t.co/ZUShvknTWO pic.twitter.com/83O5a2S0eh
— TESLARATI (@Teslarati) July 24, 2026
Additionally, FSD has pulled into my assigned parking spot at my townhouse community on multiple occasions with this release. This is something that used to happen ocassionally, but not consistently.
It also navigated back to the same convenience store last night, drove through crazy cars scrambling to gas pumps, navigated out of the parking lot correctly, drove me home, and, once again, parked in my assigned spot.
As previously stated, this release just seems to have a few things that need to be brought to Tesla’s attention, and also to make others who use FSD aware of some things that I’ve experienced. I look forward to the next release that will remedy these issues, just as Tesla has always done in the past.
-
Elon Musk’s massive Terafab project, which will be the first true conglomeration between each of his major entities, is set to get its final location soon, the CEO said on Tesla’s recent earnings call.
“The Terafab, we expect to announce a location soon, and provide more details about our plans in that regard. We’ll leave that to the product, the launch announcement rather than try to squeeze it into an earnings call,” Musk said last Wednesday.
Tesla Terafab set for launch: Inside the $20B AI chip factory that will reshape the auto industry
Terafab was announced by Musk back in March and was essentially a massive, vertically integrated semiconductor manufacturing project that would provide all the chips the three companies needed for their AI initiatives without needing third-party companies.
The plant will produce over 1 terawatt of AI compute each year, and will help back up projects like Optimus, Full Self-Driving, and other AI-based projects that Musk’s companies are working on.
In April, less than a month after the project was launched, Intel announced it would join the project, contributing manufacturing expertise and consulting to Terafab as a whole. Intel is one of three chip manufacturers that produce sub-5 nanometer chips at scale. TSMC and Samsung are the other two.
However, there was no true indication of where Terafab would end up, but most believe it will likely be somewhere in Texas. Business Insider has reported that SpaceX plans to build out Terafab in Grimes County, Texas, but this is unconfirmed.
-
Musk confirmed recently that it would not be on Giga Texas property, as it is simply too large.
The sheer scale of TERAFAB is going to be insane.
Elon said it wouldn’t be suitable for anywhere on Giga Texas property because it’s too big:
“We couldn’t possibly fit the Terafab on the GigaTexas campus. It will be far bigger than everything else combined there.
Several… pic.twitter.com/79GbhNNuf4
— TESLARATI (@Teslarati) March 23, 2026
Terafab holds much of Musk’s grand ambitions for the future within its construct. It holds so much responsibility for the future and the biggest projects that Musk’s companies can imagine.
-
“I think this is a very big announcement and it deserves to have its own day in the spotlight and not be squeezed into an earnings call,” he said. “I do think Terafab is going to be an amazing initiative and a necessary one, and one without which we will be constrained in our ability to scale Optimus production, because we simply won’t have enough AI chips.”
He continued by stating that Terafab is necessary for scaling Optimus, which Musk said could be the biggest product of any kind of all time. “It’s crucial to solve that, and we’ll have to solve memory, logic, and packaging in order to scale Optimus.”
SpaceX performed a secret test on a specific portion of Starship with its recent 13th test flight last week, CEO Elon Musk revealed.
Starship’s 13th test flight took place last Friday, and in many aspects, it was one of the most overwhelmingly successful launches in the project’s history.
All of the mission objectives were met without incident, both the Super Heavy Booster and Ship managed to perform safe splashdowns in the Gulf of America and the Indian Ocean, respectively, and the deployment of Starlink satellites came and went without any complications.
— Elon Musk (@elonmusk) July 25, 2026
However, there was more on the agenda for SpaceX with Flight 13. Musk revealed an internal test of the ship’s heat shield tiles, as the space exploration company wanted to push them to the limits after previous issues.
Many noticed that Starship’s initial launch seemed to be more accelerated than normal, and that was not a mistake. Musk revealed that SpaceX decided to give Flight 13 an intentionally aggressive acceleration rate in an effort to test how well the tiles would remain attached to the ship:
-
This flight intentionally had much higher acceleration to test how well the heat shield tiles would remain attached at high dynamic pressure.
Test was successful.
— Elon Musk (@elonmusk) July 25, 2026
SpaceX had issues with some of the heat shield tiles remaining attached early on in the Starship program. The first six test flights presented some kind of anomaly with them, so the company’s big focus with them was to figure out a way to keep them intact through the duration of the flight.
Things truly improved as Flight 10 showed that ceramic tiles generally stayed attached to the ship far better due to refined attachment, as SpaceX utilized pins instead of adhesives. Flights 10 through 13 truly showed some clear progress with the heat shield tiles, and this latest test seems to be where some real progress was noticed, especially by Musk.
The 13th Starship launch last Friday was the second with Starship V3, SpaceX’s latest and greatest iteration of the spacecraft. Goals and ambitions are getting even grander as the project continues to progress. Musk has already hinted that SpaceX will likely try to catch Starship with Flight 14.
-
Tesla claims supplier is holding Cybertruck production hostage
Tesla Full Self-Driving v14.3.6 review: a rare regression, but some bright spots
