News
Tesla is patenting a clever way to train Autopilot with augmented camera images
Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.
As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.
Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.
With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”
Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.
Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:
“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”
Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.
“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”
Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.
The Tesla Semi is entering a new Pilot Program with Paper Transport, LLC (PTI), a Wisconsin-based transportation provider. The company will test the Semi’s Long Range configuration through “dedicated operations within the Chicago market.”
Chicago presents an interesting challenge for the Semi, as it will be a colder-weather climate that will test the Semi’s ability to operate in lower temperatures and in potentially large accumulations of snow. This is something Tesla has been testing with the Semi in Alaska and even in Northern California during the colder months, but Chicago will present a truly tough midwestern winter.
Tesla Semi spotted on journey home after winter performance testing
PTI says it is using the Semi to evaluate its strategy of reducing transportation emissions while maintaining performance, reliability, and cost efficiency. These are major arguments for the Semi being introduced into new fleets.
CEO of PTI Tyler Ellison said:
“PTI has been a leader in sustainable transportation solutions for over 15 years. We take a consultative approach to helping customers identify and implement the right transportation solution for their network. Our partnership with Tesla expands our portfolio alongside renewable natural gas and intermodal, giving customers more ways to reduce Scope 3 emissions without compromising service or economics.”
PTI is far from the first company to adopt the Semi within a fleet, as Tesla entered strategic agreements with PepsiCo. and its subsidiary Frito-Lay for a Pilot Program that extended throughout the California region.
Tesla has let companies like those utilize the Semi to determine whether it would be suitable for their operations. Additionally, Tesla gets valuable information regarding the Semi’s performance, knowing what to improve and what is ideal for companies that will utilize the all-electric truck for regional and nationwide logistics.
PTI plans to utilize the Long Range configuration, which is priced at $290,000 and features a range of approximately 500 miles, a three-motor powertrain, up to 800 kW of drive power, and consumption of just 1.7 kWh per mile.
Tesla Semi pricing revealed after company uncovers trim levels
VP of Maintenance at PTI, Bryan Ellen, added:
“We are excited to partner with Tesla, leveraging their ever-evolving technology. We are bullish in our estimation of the parallels available between our dedicated model and the efficiency of their fully electric Class 8 tractor. We anticipate a growing synergy between our businesses as we work to facilitate this sustainable solution for our customers.”
PTI has logged more than 87 million miles using sources like compressed and renewable gas, but now is looking to take it a step further with fully electric operations.
Tesla revealed on Monday that it is building a new autonomous vehicle at Gigafactory Texas, its plant just outside of the City of Austin. This particular vehicle will be geared toward those who are in need of a wheelchair-accessible car that would require no human driver for operation.
According to a new report from Wired, Tesla’s Senior Policy Advisor, India Herdman, told members of the Washington D.C. City Council on Monday:
“We are in development for a purpose-built, wheelchair-accessible autonomous vehicle. We know that paratransit can be very difficult, and people who are confined to wheelchairs permanently should still be able to move around freely, so that is an active product being built by Tesla in Texas.”
This builds upon what CEO Elon Musk said last year on X, which confirmed the company was working on accessible rides within its Robotaxi platform, which currently is confined to the Model Y.
Absolutely
— Elon Musk (@elonmusk) September 19, 2025
Tesla is also developing the Cybercab, which started employee rides last week. However, this vehicle is not necessarily geared toward wheelchair accessibility.
That leaves a major gap in the autonomous ride-sharing program that Tesla is attempting to build; the company has been pretty clear that it does not want to complicate its manufacturing lines by bringing in a wide array of body styles.
However, it seems necessary to have something larger that could help transport people to appointments when they cannot drive. For wheelchair accessibility, the Robovan, which was unveiled at the “We, Robot” event in October 2024, seems to be the most ideal solution:
Herdman did not indicate whether she was referring to the Robovan or if Tesla is building yet another body style that is geared toward full autonomy but also caters to the handicapped.
Tesla might need to develop something specifically for the handicapped in order to align with the Americans with Disabilities Act, which prevents discrimination against people with disabilities in transportation services. Uber was hit with a lawsuit late last year for “refusing to reasonably modify its policies, practices, or procedures where necessary to avoid discriminating against riders with disabilities.”
Tesla would obviously like to avoid this.
It will be interesting to see what Tesla will do with this project, and whether it will introduce something new to the market or just continue with the Robovan.
Tesla weirdly confirmed that its steering wheel-less and pedal-less Cybercab vehicle is now in the process of giving employees rides, a huge milestone for the vehicle program.
But the entire thing was super strange. On Friday, Tesla released a video stating that there was “Cool news from Giga Texas” and that employees were now taking rides in Cybercabs that have no manual controls. The units seen on public roads are engineering vehicles that have manual controls inside, a necessity as Tesla moved through the testing phase.
However, Tesla removed the video and reposted it shortly after with a more vague title. It seems like the employee rides are still going, but the video was adjusted slightly. The initial upload showed employees doing things like watching movies and adjusting the climate, but these snippets were removed in the second upload.
Cool news from Giga Texas pic.twitter.com/gvbG456Tzw
— Tesla Robotaxi (@robotaxi) July 11, 2026
Both images below were uploaded with the first video, but were removed after Tesla re-uploaded the announcement. These are not available in the second upload
Credit: Tesla
Credit: Tesla
Nevertheless, the announcement from Tesla is that the Cybercab is operating with employees inside who can control the vehicle’s audio, video, climate, and destination settings through their smartphone app.
Tesla has already been testing Cybercab engineering units, but last month, it was able to self-certify for SAE Level 4, which would enable unsupervised self-driving in Texas. The company is moving toward that, and the plans have always been to launch Cybercab rides this year.
The Cybercab is potentially looked at as the next generation of Tesla’s mobility leg. For the past 15 years, the company has been known as somewhat of an automaker, among many other things. However, these passenger vehicles that Tesla has manufactured are now moving into a new realm, as they will eventually drive themselves with no supervision thanks to the Full Self-Driving suite.
The Cybercab is just the next step of that: a true vehicle developed for the sole purpose of ride-hailing. It has no human controls, it has only two seats, and it will get passengers from Point A to Point B with no awkward driver, no need for manual inputs, and with no stress.
Tesla is moving forward with other developments related to the Cybercab project as well. However, the big announcement will come when Tesla finally announces that it is launching Cybercab rides to the general public, something that it plans to launch either late this year or early 2027.
Tesla Semi enters new Pilot Program with interesting challenge
Tesla is building a wheelchair-accessible Robotaxi
