Connect with us
Tesla Autopilot construction zone lane Tesla Autopilot construction zone lane

News

Tesla is patenting a clever way to train Autopilot with augmented camera images

Tesla Autopilot construction zone lane (Credit: YouTube/Cf Tesla)

Published

on

Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.

As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.

Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.

With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”

A block diagram of an environment for computer model training. (Credit: Patentscope.wipo.int)

Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.

Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:

Advertisement

“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”

Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.

“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”

Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.

Advertisement

Simon is an experienced automotive reporter with a passion for electric cars and clean energy. Fascinated by the world envisioned by Elon Musk, he hopes to make it to Mars (at least as a tourist) someday. For stories or tips--or even to just say a simple hello--send a message to his email, simon@teslarati.com or his handle on X, @ResidentSponge.

Advertisement
Comments

News

Tesla Semi gets new adoptee in latest sighting

Tesla is continuing to increase the number of companies that have access to the Semi.

Published

on

Credit: Tesla

The Tesla Semi looks to have its newest adoptee as US Foods, the second-largest foodservice distributor in the United States, was seen with its badging on the all-electric Class 8 truck.

US Foods trails only Sysco Corporation in terms of foodservice distributors by volume in the United States. The company joined several other massive food industry entities, such as PepsiCo, and its subsidiary, Frito-Lay, have both utilized the Tesla Semi in their fleets.

Tesla Semi futuristic sci-fi acceleration sound will never get old

The Semi in question was spotted by X user Zanegler, a prominent tracker of the Tesla Semi and its factory, which sits near the company’s Gigafactory Nevada plant in Reno:

US Foods also has a distribution center in Reno, which could have something to do with its decision to start using the Semi in regional logistics.

With an influx of EV semitrucks hitting the market from many different manufacturers, it is evident that companies are taking the idea of making their fleets more environmentally friendly very seriously.

Advertisement

Tesla is still very close to unloading the Semi in a more volumized fashion, as the company’s Vice President of Vehicle Engineering, Lars Moravy, said in January:

“We just closed out the Semi factory roofing walls last week in Reno…but we’re prepping for mechanical installation of all the equipment in the coming months… The first builds of the high-volume Semi design come late this year in 2025 and begin ramping early in 2026.”

First Tesla Semi high-volume production builds expected this 2025

With the latest addition of US Foods, Tesla increases its list of companies that are planning or are already using the Semi to help with local logistics and transportation. Among them are:

  • PepsiCo
  • Walmart
  • Sysco
  • Costco
  • Martin Brower
  • Saia Inc.
  • UPS
  • Anheuser-Busch
  • DHL

Many other companies have plans to use the Semi in their fleets. Currently, Tesla appears to be hand-picking those who have access to the vehicle as the pilot program continues.

Advertisement
Continue Reading

Lifestyle

Tesla Cybertruck takes a bump from epic failing Dodge Charger

The Cybertruck seemed unharmed by the charging Charger.

Published

on

Credit: Hammer_of_something/Reddit

There comes a time in a driver’s life when one is faced with one’s limitations. For the driver of a Dodge Charger, this time came when he lost control and crashed into a Tesla Cybertruck–an absolute epic fail. 

A video of the rather unfortunate incident was shared on the r/TeslaLounge subreddit.

Charging Charger Fails

As could be seen in the video, which was posted on the subreddit by Model Y owner u/Hammer_of_something, a group of teens in a Dodge Charger decided to do some burnouts at a Tesla Supercharger. Unfortunately, the driver of the Charger failed in his burnout or donut attempt, resulting in the mopar sedan going over a curb and bumping a charging Cybertruck.

Ironically, the Dodge Charger seemed to have been parked at a Supercharger stall before its driver decided to perform the failed stunt. This suggests that the vehicle was likely ICE-ing a charging stall before it had its epic fail moment. Amusingly enough, the subreddit member noted that the Cybertruck did not seem like it took any damage at all despite its bump. The Charger, however, seemed like it ran into some trouble after crashing into the truck.

Alleged Aftermath

As per the the r/TeslaLounge subreddit member, the Cybertruck owner came rushing out to his vehicle after the Dodge Charger crashed into it. The Model Y owner then sent over the full video of the incident, which clearly showed the Charger attempting a burnout, failing, and bumping into the Cybertruck. The Cybertruck owner likely appreciated the video, in part because it showed the driver of the Dodge Charger absolutely freaking out after the incident.

Advertisement

The Cybertruck is not an impregnable vehicle, but it can take bumps pretty well thanks to its thick stainless steel body. Based on this video, it appears that the Cybertruck can even take bumps from a charging Charger, all while chilling and charging at a Supercharger. As for the teens in the Dodge, they likely had to provide a long explanation to authorities after the incident, since the cops were called to the location.

Continue Reading

News

Tesla confronts Service complaints with new AI-backed strategy

Tesla will use a new AI Agent to help expedite Service claims and improve communication with customers.

Published

on

tesla service
Credit: Tesla

Tesla is implementing a new AI-based strategy to improve service for owners. Service has routinely been among the biggest complaints of owners, although it has improved drastically over the past few years.

Some of the complaints that Tesla has received regarding its Service platform have evidently been used to develop new strategies to not only streamline the entire experience but also to make things easier for the company, which deals with many claims each week.

As a result of complaints in the past, Tesla has used various strategies to make things better for customers. However, the latest improvement comes as a result of AI, something Tesla leans on in many facets of its business.

Tesla adds ingenious solution to app to streamline Service appointments

Tesla’s AI and IT Infrastructure, Cybersecurity, and Vehicle Service head Raj Jegannathan said on X this week that the company is rolling out a new AI Agent specifically designed to handle service comms with customers.

He said the new Service AI Agent will detect delays in communications between the company and the customers, monitor the sentiment of these conversations, and auto-escalate certain claims directly to leaders.

It will also allow customers to type the word ‘Escalate’ in the message center portion of the phone app after two weeks of delays. This will help customers reach higher-ups more easily and likely will eliminate the complaints that many have had over the past few years.

The company is rolling out the AI Agent in ten pilot locations to start. Its first day being active was May 8.

Jegannathan said:

“Tesla Service’s new AI Agent detects comms delays, monitors sentiment, & auto-escalates to leaders. Starts tomorrow at 10 pilot locations. In 2 weeks, type “Escalate” in ‘message center’ to reach managers. Guardrails in place to prevent abuse. We’ll keep improving!”

Service has had a lot of interesting strategies used to improve things, but it has definitely been a weak point of the Tesla ownership experience. In a perfect world, vehicles wouldn’t need repair for anything, but that is not realistic.

Instead, Tesla has worked to expedite the entire Service experience through various strategies, including F1-style service, and a goal to fix two-thirds of repair claims within the same day.

Parts availability sometimes takes this goal out of reasonableness, but these constant attempts at improving the repair experience show Tesla is doing what it can to make things better for owners.

Continue Reading

Trending