News
Tesla is patenting a clever way to train Autopilot with augmented camera images
Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.
As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.
Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.
With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”
Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.
Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:
“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”
Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.
-
“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”
Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.
SpaceX just picked up another $1.6 billion from the Pentagon, with the U.S. Space Force awarding two task orders worth $1.6 billion to fly 18 Falcon 9 missions from Vandenberg Space Force Base in California through the end of 2027. The launches will carry satellites for the Space Based Sensing and Targeting portfolio, a set of programs meant to help the military detect and track airborne threats and relay that information across forces in near real time.
The award falls under National Security Space Launch Phase 3 Lane 1, the Space Force’s faster, commercial style procurement track for missions that do not require the military’s most demanding certification process. It is also the largest single order publicly disclosed under that program so far, and the first task order issued since the Space Force nearly tripled Lane 1’s contract ceiling from $5.6 billion to $17 billion on July 17.
SpaceX to become America’s Military data backbone for missiles, drones, and warfighters
Eric Zarybnisky, the Space Force’s acting portfolio acquisition executive for space access, said the entire process, from identifying the requirement to signing the contract, took about two months, including a month set aside for companies to prepare proposals.
SpaceX is not just launching these satellites. It already holds the contracts to build two of the programs within the same portfolio, $4.16 billion for the Space Based Airborne Moving Target Indicator system and $2.29 billion for the Space Data Network Backbone, which Teslarati covered in May. That means SpaceX is now responsible for both building key pieces of the military’s next generation sensing network and getting them into orbit.
With this latest award, SpaceX’s Pentagon contract total for 2026 alone tops $8 billion, adding to a defense portfolio that already includes the Golden Dome missile defense software group SpaceX joined in April and a string of GPS launches it inherited after ULA’s Vulcan rocket ran into a booster anomaly, which we detailed in March.
Lane 1’s vendor pool technically includes seven companies: SpaceX, ULA, Blue Origin, Rocket Lab, Stoke Space, Impulse Space, and Relativity Space. In practice, SpaceX remains the only provider with the combination of launch cadence, flight proven Falcon 9 hardware, and West Coast infrastructure to support a campaign requiring roughly one Vandenberg launch a month for the next year and a half.
-
Some lawmakers have flagged the growing concentration of national security launches with one company as a risk worth watching. For now, the Space Force keeps backing SpaceX, with it being the company that shows up ready to launch.
Elon Musk shed two new bits of detail on Starship following its 13th test launch, which was an overwhelming success.
SpaceX launched Starship for the 13th time last Friday after two delays: one on Monday when several Raptor engines did not ignite, and another on Thursday due to unfavorable weather conditions in Starbase, Texas.
The launch was overwhelmingly successful. SpaceX was able to complete a necessary test of the heat shield tiles by increasing the acceleration of Starship from launch throughout the flight; 20 Starlink v3 satellites were released with no issue; the Super Heavy Booster landed safely in the Gulf of America, Ship successfully reignited engines while in space; and it also splashed down without incident in the Indian Ocean.
SpaceX Starship just nailed something it’s never done before
Nevertheless, more details are coming out about Starship, and Musk is doing the talking.
Starship Will Be Retrieved in the Ocean
Musk revealed on Tuesday night that Starship would be recovered by a ship in the Indian Ocean. SpaceX routinely tries to recover Starship after splashdown in an effort to find out more about the flight by examining the spacecraft afterward.
We’re sending a ship out to recover Starship https://t.co/fUqUZTITO9
— Elon Musk (@elonmusk) July 28, 2026
-
This helps engineers find out more about why things might have happened, allows them to examine any potential damage or anomalies that might have occurred, and increases the chances of an even more successful flight next time thanks to the additional information recovered.
Ship Could Have Been Caught by Tower Arms, Musk claims
Musk has already indicated that SpaceX will plan to attempt a catch of Starship with the 14th test flight. While this would be a major accomplishment, it would be an expected next step, considering the fact that the Super Heavy Booster has already been caught by the chopsticks on numerous occasions.
The ship landing was precise, meaning that it would have been caught by the tower arms https://t.co/6nbNrRfX9P
— Elon Musk (@elonmusk) July 29, 2026
A ship catch would be a great indication of where SpaceX stands in terms of reusability and launch cadence. A successful catch with relatively no incidents would be a good sign that SpaceX is nearing a more frequent launch of Starship, but also that the reusability of the massive rocket would be something many would expect in the near future.
It is a necessity to make life multiplanetary.
-
Elon Musk has updated his timeline for when humans will walk on Mars and for when ships will simply get there.
The objective of getting to Mars has been one of Musk’s biggest goals since becoming a serial entrepreneur and realizing that time on Earth is limited. Musk has said several times he hopes to die on Mars, and not by impact.
Musk now believes that people will be on Mars in “roughly 5 to 7 years.” He said that a Mars lander will get there “a few years sooner.”
People on Mars in roughly 5 to 7 years.
Mars lander a few years sooner.
— Elon Musk (@elonmusk) July 29, 2026
The response from Musk comes after NASA Administrator Jared Isaacman said that SpaceX’s biggest priority is the Moon and not Mars. Because of this, Isaacman conceded that he believes nuclear power and propulsion investments will provide “potentially the pathway with the fewest miracles required to put four people on Mars in the next 10 to 15 years.”
-
Of course, this is what NASA can do through taxpayer funding and nuclear investments, he added.
Musk’s grand ambitions are much more optimistic than most, and it is certainly a double-edged sword. This is not the first time timelines for Mars have been somewhat lofty, especially to those normal thinkers like you and me, not super geniuses like Musk.
In fact, the SpaceX and Tesla frontman has said on at least a dozen occasions that we could be on Mars in the coming years. Musk said 2020 would be the big year as early as 2009. In 2020, he was “highly confident” of a landing in 2026, and had even said 2024 in a best-case scenario.
The point is, the range has varied, and it’s anyone’s guess when we’ll get there. This latest adjustment to the timeline is typical of Musk, and while the Moon has seemingly taken priority over Mars, it is still worth mentioning that the ultimate goal is to make life multiplanetary, and it starts potentially with the Red Planet.
SpaceX scores another massive Pentagon deal to support military satellites
Elon Musk sheds two new bits of detail on Starship after 13th test launch
