News
Tesla is patenting a clever way to train Autopilot with augmented camera images
Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.
As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.
Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.
With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”
Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.
Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:
“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”
Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.
“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”
Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.
News
Tesla FSD fleet is nearing 7 billion total miles, including 2.5 billion city miles
As can be seen on Tesla’s official FSD webpage, vehicles equipped with the system have now navigated over 6.99 billion miles.
Tesla’s Full Self-Driving (Supervised) fleet is closing in on almost 7 billion total miles driven, as per data posted by the company on its official FSD webpage.
These figures hint at the massive scale of data fueling Tesla’s rapid FSD improvements, which have been quite notable as of late.
FSD mileage milestones
As can be seen on Tesla’s official FSD webpage, vehicles equipped with the system have now navigated over 6.99 billion miles. Tesla owner and avid FSD tester Whole Mars Catalog also shared a screenshot indicating that from the nearly 7 billion miles traveled by the FSD fleet, more than 2.5 billion miles were driven inside cities.
City miles are particularly valuable for complex urban scenarios like unprotected turns, pedestrian interactions, and traffic lights. This is also the difference-maker for FSD, as only complex solutions, such as Waymo’s self-driving taxis, operate similarly on inner-city streets. And even then, incidents such as the San Francisco blackouts have proven challenging for sensor-rich vehicles like Waymos.
Tesla’s data edge
Tesla has a number of advantages in the autonomous vehicle sector, one of which is the size of its fleet and the number of vehicles training FSD on real-world roads. Tesla’s nearly 7 billion FSD miles then allow the company to roll out updates that make its vehicles behave like they are being driven by experienced drivers, even if they are operating on their own.
So notable are Tesla’s improvements to FSD that NVIDIA Director of Robotics Jim Fan, after experiencing FSD v14, noted that the system is the first AI that passes what he described as a “Physical Turing Test.”
“Despite knowing exactly how robot learning works, I still find it magical watching the steering wheel turn by itself. First it feels surreal, next it becomes routine. Then, like the smartphone, taking it away actively hurts. This is how humanity gets rewired and glued to god-like technologies,” Fan wrote in a post on X.
News
Tesla starts showing how FSD will change lives in Europe
Local officials tested the system on narrow country roads and were impressed by FSD’s smooth, human-like driving, with some calling the service a game-changer for everyday life in areas that are far from urban centers.
Tesla has launched Europe’s first public shuttle service using Full Self-Driving (Supervised) in the rural Eifelkreis Bitburg-Prüm region of Germany, demonstrating how the technology can restore independence and mobility for people who struggle with limited transport options.
Local officials tested the system on narrow country roads and were impressed by FSD’s smooth, human-like driving, with some calling the service a game-changer for everyday life in areas that are far from urban centers.
Officials see real impact on rural residents
Arzfeld Mayor Johannes Kuhl and District Administrator Andreas Kruppert personally tested the Tesla shuttle service. This allowed them to see just how well FSD navigated winding lanes and rural roads confidently. Kruppert said, “Autonomous driving sounds like science fiction to many, but we simply see here that it works totally well in rural regions too.” Kuhl, for his part, also noted that FSD “feels like a very experienced driver.”
The pilot complements the area’s “Citizen Bus” program, which provides on-demand rides for elderly residents who can no longer drive themselves. Tesla Europe shared a video of a demonstration of the service, highlighting how FSD gives people their freedom back, even in places where public transport is not as prevalent.
What the Ministry for Economic Affairs and Transport says
Rhineland-Palatinate’s Minister Daniela Schmitt supported the project, praising the collaboration that made this “first of its kind in Europe” possible. As per the ministry, the rural rollout for the service shows FSD’s potential beyond major cities, and it delivers tangible benefits like grocery runs, doctor visits, and social connections for isolated residents.
“Reliable and flexible mobility is especially vital in rural areas. With the launch of a shuttle service using self-driving vehicles (FSD supervised) by Tesla in the Eifelkreis Bitburg-Prüm, an innovative pilot project is now getting underway that complements local community bus services. It is the first project of its kind in Europe.
“The result is a real gain for rural mobility: greater accessibility, more flexibility and tangible benefits for everyday life. A strong signal for innovation, cooperation and future-oriented mobility beyond urban centers,” the ministry wrote in a LinkedIn post.
News
Tesla China quietly posts Robotaxi-related job listing
Tesla China is currently seeking a Low Voltage Electrical Engineer to work on circuit board design for the company’s autonomous vehicles.
Tesla has posted a new job listing in Shanghai explicitly tied to its Robotaxi program, fueling speculation that the company is preparing to launch its dedicated autonomous ride-hailing service in China.
As noted in the listing, Tesla China is currently seeking a Low Voltage Electrical Engineer to work on circuit board design for the company’s autonomous vehicles.
Robotaxi-specific role
The listing, which was shared on social media platform X by industry watcher @tslaming, suggested that Tesla China is looking to fill the role urgently. The job listing itself specifically mentions that the person hired for the role will be working on the Low Voltage Hardware team, which would design the circuit boards that would serve as the nervous system of the Robotaxi.
Key tasks for the role, as indicated in the job listing, include collaboration with PCB layout, firmware, mechanical, program management, and validation teams, among other responsibilities. The role is based in Shanghai.
China Robotaxi launch
China represents a massive potential market for robotaxis, with its dense urban centers and supportive policies in select cities. Tesla has limited permission to roll out FSD in the country, though despite this, its vehicles have been hailed as among the best in the market when it comes to autonomous features. So far, at least, it appears that China supports Tesla’s FSD and Robotaxi rollout.
This was hinted at in November, when Tesla brought the Cybercab to the 8th China International Import Expo (CIIE) in Shanghai, marking the first time that the autonomous two-seater was brought to the Asia-Pacific region. The vehicle, despite not having a release date in China, received a significant amount of interest among the event’s attendees.
Tesla FSD fleet is nearing 7 billion total miles, including 2.5 billion city miles
Tesla starts showing how FSD will change lives in Europe
