News
Tesla is patenting a clever way to train Autopilot with augmented camera images
Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.
As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.
Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.
With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”
Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.
Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:
“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”
Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.
“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”
Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.
NASA has filed a procurement notice announcing its intent to add six post-certification missions to SpaceX’s existing Commercial Crew Transportation Capability contract. The agency said it would order up to three of those missions immediately upon adding them to the contract, with the remaining three available as needed through the end of the International Space Station’s planned operations in 2030.
The reason for the expansion is straightforward. NASA cited recently shortened ISS mission durations, technical issues and schedule delays encountered by Boeing, the allocation of missions between Boeing and SpaceX, and the ongoing technical challenges of maintaining a reliable crew transportation capability as the driving factors behind the decision. Boeing’s CST-100 Starliner has still not been certified for crewed flights, and a cargo-only Starliner mission was not included on NASA’s most recent mission manifest. With Boeing effectively sidelined for the foreseeable future, SpaceX is the only American company capable of rotating crews to the station.
The history behind this contract tells the fuller story of how SpaceX got here. NASA originally awarded SpaceX its Commercial Crew contract in 2014 for $2.6 billion. In 2022 NASA modified the contract to add five missions covering Crew-10 through Crew-14, worth $1.436 billion, bringing the total contract value at that point to $4.9 billion. The recent May 18 filing by NASA extends that runway further, with Crew-12 currently docked at the station and Crew-13 assigned and targeting a mid-September 2026 launch.
According to a report by SpaceNews, NASA stated in its filing: “It is necessary to award additional PCMs to SpaceX given the recently shortened ISS mission durations, technical issues and schedule delays encountered by Boeing, the allocation of missions between Boeing and SpaceX, NASA’s projections for when an alternative crew transportation system may become available, and the ongoing technical challenges of maintaining a reliable capability for crewed flights to ISS.”
No dollar value for the new six missions has been publicly confirmed yet, but based on the 2022 precedent of roughly $287 million per mission, the new block could represent close to $1.7 billion in additional contract value. With SpaceX simultaneously preparing Starship as NASA’s Artemis lunar lander, filing its S-1 for a June IPO, and now absorbing more ISS crew rotation work, the company’s role as the primary contractor for American human spaceflight is no longer a matter of circumstance. It is NASA policy.
In a notable intersection of Big Tech powerhouses, Meta, led by Mark Zuckerberg, has partnered with Canadian energy infrastructure giant Enbridge on a significant renewable energy initiative that will rely on battery technology from Elon Musk’s Tesla.
The project, which was announced this week, marks another step in Meta’s aggressive push to power its expanding data center operations with clean energy, dispelling many of the complaints people have about them.
This new development is located near Cheyenne, Wyoming, and will feature a 365-megawatt (MW) solar farm paired with a 200 MW/1,600 megawatt-hour (MWh) battery energy storage system, also known as BESS. Tesla is providing the batteries for the project, valued at roughly $200 million.
The story was originally reported by Utility Dive.
This Wyoming project represents the first phase of Enbridge and Meta’s joint “Cowboy Project.” Once operational, it will deliver power to Meta’s regional data centers through Cheyenne Light, Fuel, and Power under Wyoming’s Large Power Contract Service tariff.
This tariff, originally developed in collaboration with Microsoft and Black Hills Energy, is designed specifically for large loads like data centers. It ensures that the renewable supply serves hyperscale customers without impacting retail electricity rates for other users.
The battery system will operate under a long-term tolling agreement, providing dispatchable capacity that enhances grid reliability. During periods of high demand, the utility can access the backup generation, addressing one of the key challenges of integrating large-scale renewables with the explosive growth of data center electricity demand driven by artificial intelligence.
This latest collaboration builds on prior joint efforts between Enbridge and Meta in Texas, including the 600 MW Clear Fork Solar, 152 MW Easter Wind, and 300 MW Cone Wind projects. Together with the Wyoming initiative, the companies have now partnered on roughly 1.6 gigawatts (GW) of combined solar, wind, and storage capacity.
The deal highlights the intensifying demand for reliable, low-carbon power from technology giants. Meta has committed to supporting its data center growth with renewable energy, joining peers like Microsoft and Google in seeking large-scale solutions. Enbridge’s Allen Capps described the project as “one of the larger utility-scale battery installations supporting U.S. data center operations and growth.”
The involvement of Tesla’s battery technology adds an intriguing layer, linking two of the world’s most prominent tech leaders—Zuckerberg and Musk—in the clean energy transition.
As data centers continue to drive unprecedented electricity load growth across the United States, projects like this one illustrate how hyperscalers are turning to strategic partnerships with traditional energy players and innovative storage solutions to meet both sustainability goals and reliability needs.
SpaceX has decided to stand down from what was supposed to be the first test launch of Starship’s v3 rocket tonight after a minor issue with a hydraulic pin delayed the flight once more.
The company scrubbed its first test flight of the upgraded Starship v3 on May 21 in the final minutes of the countdown. SpaceX CEO Elon Musk quickly took to social media platform X, explaining that a hydraulic pin on the launch tower’s “chopsticks” arm failed to retract properly.
Musk added that the company would fix the issue this evening. SpaceX will attempt another launch tomorrow night at 5:30 p.m. CT, 6:30 p.m. ET, and 3:30 p.m. PT.
The hydraulic pin holding the tower arm in place did not retract.
If that can be fixed tonight, there will be another launch attempt tomorrow at 5:30 CT. https://t.co/DJAdvDYQpH
— Elon Musk (@elonmusk) May 21, 2026
The countdown for Starship Flight 12 — featuring the taller and more capable V3 stack with Booster 19 and Ship 39 — had been progressing smoothly until the late-stage issue surfaced. The Mechazilla tower arm, designed to secure the vehicle on the pad and eventually catch returning boosters, could not complete its retraction sequence.
SpaceX teams immediately began troubleshooting the hydraulic system for an overnight repair.
Starship V3 introduces several significant upgrades over earlier versions. These include greater propellant capacity, more powerful Raptor 3 engines, larger grid fins, enhanced heat shielding, and an improved fuel transfer system.
We covered the changes that were announced just days ago by SpaceX:
SpaceX unveils sweeping Starship V3 upgrades ahead of May 19 launch
The changes are intended to increase payload performance, support higher flight rates, and advance the vehicle toward operational missions, including Starlink deployments, NASA Artemis lunar landings, and future crewed Mars flights. The debut flight from Starbase’s new Launch Pad 2 marked an important milestone in scaling up the fully reusable Starship system.
This stand-down highlights the intricate challenges of preparing the world’s most powerful rocket for flight. Despite extensive pre-launch checks, a single component in the ground support equipment can force a scrub.
The incident aligns with Starship’s proven iterative development approach. Previous test flights have encountered both successes and setbacks, each providing critical data that refines hardware and procedures. Some outlets may call some of these flights “failures,” when in reality, they are all opportunities for SpaceX to learn for the next attempt.
With V3, SpaceX aims to reduce ground-system dependencies and increase launch cadence to meet ambitious long-term goals.
NASA just gave SpaceX more crew missions because Boeing can’t certify
Elon Musk called it Epic: The full story of SpaceX’s Starship Flight 12
