News
Tesla Smart Summon patent highlights progress in 3D labeling for full self-driving features
A recently published Tesla patent application details the machine learning methods behind Smart Summon, specifically highlighting the progress being made with 3D labeling in training data.
The application, titled “Autonomous and User Controlled Vehicle Summon to a Target,” utilizes machine learning methods explicitly detailed in two other recent Tesla patent publications in its functionality. This series of three inventions altogether describes an automated way of generating training data which is then used by a machine learning model to accomplish an expansive list of self-driving capabilities in Summon.
“Traditionally, much of the effort to curate a training data set is done manually by reviewing potential training data and properly labeling the features associated with the data,” Tesla’s first application in the series states. “The effort required to create a training set with accurate labels can be significant and is often tedious… Therefore, there exists a need to improve the process for generating training data with accurate labeled features.”
-
- A method flow chart from Tesla’s autonomous 3D labeling patent. | Image: Tesla/USPTO
-
- A method flow chart from Tesla’s Smart Summon patent application. | Image: Tesla/USPTO
The application goes on to describe how labeled training data is made autonomously in their invention using sensors and the collection of what’s called a “time series,” i.e., a series of images captured over a period of time.
“Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created,” it explains. “In some embodiments, a three-dimensional representation of a feature, such as a lane line, is created from the group of time series elements that corresponds to the ground truth… As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels…a single image of the group and the actual path taken can be used as training data to predict the path of the vehicle.”
Tesla CEO Elon Musk has previously mentioned that better labeling is one of the keys to speeding up the rollout of self-driving functionality and features like Reverse Summon. “We need to finish work on Autopilot core foundation code & 3D labeling, then functionality will happen quickly. Not long now,” Musk wrote on Twitter in March this year. With better labeling (more accurate training data) comes safer and more capable software due to improved predictions from the modeling.
When it comes to Tesla’s Smart Summon, prediction modeling is essential considering there isn’t a driver in the vehicle during its operation. The patent publication covering Summon embodies the first application’s time series functionality and a second application’s implementation of the time series’ training data in its methods, demonstrating one of the numerous potential uses for the machine learning invention. Hints about future developments using Smart Summon are also detailed in the application. Examples include:
- Syncing the Smart Summon with a calendar so the vehicle “automatically navigates to arrive at the location at the ending time, such as the end of a dinner party, a wedding, a restaurant reservation, etc.”
- Implementing a multi-part destination into the Summon instructions such as waypoints at an airport to pick up multiple passengers.
- Monitoring the heartbeat of a Summon user to ensure they are maintaining a connection with the vehicle while operating the feature.
- Customizing the vehicle’s arrival settings such as interior lighting, exterior lighting, hazard lights, welcome music, and climate control preferences.
One of the more unique bits about the Smart Summon patent application is the appearance of Elon Musk as an inventor. While the CEO is known to be intimately involved in nearly all aspects of vehicle design, software features, and business operations, his name is unexpectedly absent from most of the company’s inventions. However, this is apparently on purpose. “I generally try my best not to be on patents,” he revealed on Twitter in reply to a post about the Smart Summon application. Notably, inventorship is a legal definition based on the conception of an invention, i.e., not the person/people who suggested or directed its creation, but the person/people who devised the means to accomplish it.
Prior to the most recent patent publication, Musk contributed inventorship to the door and body styling of the Model X. He also contributed the same to both the design and function of Tesla’s vehicle charge inlets.
Tesla has made two big interior changes to several Model Y vehicles in its lineup, and the changes come just as the new model year begins production.
Last year, Tesla launched the Model Y Standard, which separated the previous models into the “Premium” category. The Standard vehicles lack several features, including more premium interior materials, acoustic-lined glass, and storage.
@teslarati There are some BIG differences between the Tesla Model Y Standard and Tesla Model Y Premium #tesla #teslamodely ♬ Sia – Xeptemper
The Model Y “Premium” trims are now getting several new upgrades, which come after the company launched a seven-seat configuration of the vehicle last night in the North American market for an upcharge of $2,500.
The new Model Y seven-seat configuration did not come with just an additional row of seating; it also came with a slew of other goodies that now come standard and were previously only available on the Model Y Performance, which was launched late last year.
All Black Headliner
The new Tesla Model Y Premium trims will now come standard with a black headliner, something that many owners have been requesting for some time.
The previous grey headliner and trim within the vehicle is now gone; it will be all black on all of the Premium trims from here on out, a welcome change:
Credit: Tesla
Larger and Higher Resolution Center Touchscreen
The center touchscreen in the new Model Y Premium configuration is now larger and has a higher resolution than the previous version.
In last year’s Model Y configurations (apart from the Performance), the center touchscreen was 15.4″. Now, Tesla has decided to go with the 16″ version across all Premium trims, which is a nice step up. It was nice to see this in the Performance, but it is really great to see Tesla include this in the Model Y’s more Premium trim levels.
Tesla Model Y Seven Seater
Tesla launched the latest iteration of the seven-seater for the Model Y on Monday night. Traditionally, the Model Y seats five passengers in total, but there were calls for a more spacious version several years ago.
Tesla released it, but it was extremely tight in the back, basically reserving those back seats for only small people or children.
Credit: Tesla
The new configuration looks to be slightly more spacious in the third row, but not as much space as most would require or want. Instead,
Elon Musk
Lufthansa Group to equip Starlink on its 850-aircraft fleet
Under the collaboration, Lufthansa Group will install Starlink technology on both its existing fleet and all newly delivered aircraft, as noted by the group in a press release.
Lufthansa Group has announced a partnership with Starlink that will bring high-speed internet connectivity to every aircraft across all its carriers.
This means that aircraft across the group’s brands, from Lufthansa, SWISS, and Austrian Airlines to Brussels Airlines, would be able to enjoy high-speed internet access using the industry-leading satellite internet solution.
Starlink in-flight internet
Under the collaboration, Lufthansa Group will install Starlink technology on both its existing fleet and all newly delivered aircraft, as noted by the group in a press release.
Starlink’s low-Earth orbit satellites are expected to provide significantly higher bandwidth and lower latency than traditional in-flight Wi-Fi, which should enable streaming, online work, and other data-intensive applications for passengers during flights.
Starlink-powered internet is expected to be available on the first commercial flights as early as the second half of 2026. The rollout will continue through the decade, with the entire Lufthansa Group fleet scheduled to be fully equipped with Starlink by 2029. Once complete, no other European airline group will operate more Starlink-connected aircraft.
Free high-speed access
As part of the initiative, Lufthansa Group will offer the new high-speed internet free of charge to all status customers and Travel ID users, regardless of cabin class. Chief Commercial Officer Dieter Vranckx shared his expectations for the program.
“In our anniversary year, in which we are celebrating Lufthansa’s 100th birthday, we have decided to introduce a new high-speed internet solution from Starlink for all our airlines. The Lufthansa Group is taking the next step and setting an essential milestone for the premium travel experience of our customers.
“Connectivity on board plays an important role today, and with Starlink, we are not only investing in the best product on the market, but also in the satisfaction of our passengers,” Vranckx said.
Elon Musk
Tesla locks in Elon Musk’s top problem solver as it enters its most ambitious era
The generous equity award was disclosed by the electric vehicle maker in a recent regulatory filing.
Tesla has granted Senior Vice President of Automotive Tom Zhu more than 520,000 stock options, tying a significant portion of his compensation to the company’s long-term performance.
The generous equity award was disclosed by the electric vehicle maker in a recent regulatory filing.
Tesla secures top talent
According to a Form 4 filing with the U.S. Securities and Exchange Commission, Tom Zhu received 520,021 stock options with an exercise price of $435.80 per share. Since the award will not fully vest until March 5, 2031, Zhu must remain at Tesla for more than five years to realize the award’s full benefit.
Considering that Tesla shares are currently trading at around the $445 to $450 per share level, Zhu will really only see gains in his equity award if Tesla’s stock price sees a notable rise over the years, as noted in a Sina Finance report.
Still, even at today’s prices, Zhu’s stock award is already worth over $230 million. If Tesla reaches the market cap targets set forth in Elon Musk’s 2025 CEO Performance Award, Zhu would become a billionaire from this equity award alone.
Tesla’s problem solver
Zhu joined Tesla in April 2014 and initially led the company’s Supercharger rollout in China. Later that year, he assumed the leadership of Tesla’s China business, where he played a central role in Tesla’s localization efforts, including expanding retail and service networks, and later, overseeing the development of Gigafactory Shanghai.
Zhu’s efforts helped transform China into one of Tesla’s most important markets and production hubs. In 2023, Tesla promoted Zhu to Senior Vice President of Automotive, placing him among the company’s core global executives and expanding his influence beyond China. He has since garnered a reputation as the company’s problem solver, being tapped by Elon Musk to help ramp Giga Texas’s vehicle production.
With this in mind, Tesla’s recent filing seems to suggest that the company is locking in its top talent as it enters its newest, most ambitious era to date. As could be seen in the targets of Elon Musk’s 2025 pay package, Tesla is now aiming to be the world’s largest company by market cap, and it is aiming to achieve production levels that are unheard of. Zhu’s talents would definitely be of use in this stage of the company’s growth.
Tesla makes two big interior changes to several Model Y vehicles
Lufthansa Group to equip Starlink on its 850-aircraft fleet
