Connect with us
Tesla Autopilot construction zone lane Tesla Autopilot construction zone lane

News

Tesla is patenting a clever way to train Autopilot with augmented camera images

Tesla Autopilot construction zone lane (Credit: YouTube/Cf Tesla)

Published

on

Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.

As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.

Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.

With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”

A block diagram of an environment for computer model training. (Credit: Patentscope.wipo.int)

Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.

Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:

“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”

Advertisement
-

Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.

“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”

Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.

Simon is an experienced automotive reporter with a passion for electric cars and clean energy. Fascinated by the world envisioned by Elon Musk, he hopes to make it to Mars (at least as a tourist) someday. For stories or tips--or even to just say a simple hello--send a message to his email, simon@teslarati.com or his handle on X, @ResidentSponge.

Advertisement
Comments

News

Why SpaceX is finishing another space-internet system that isn’t Starlink

SpaceX launched three final O3b mPower satellites Sunday, finishing a lesser known SES satellite network.

Published

on

By

SpaceX had an 87 minute window opening at 2:49 p.m. Eastern on Sunday to fly a Falcon 9 out of Cape Canaveral carrying the final three satellites for SES’s O3b mPower constellation, a project that has taken more than a decade to finish since Boeing and SES first signed SpaceX on for the work.

Unlike the thousands of Starlink satellites SpaceX has stacked into orbit over the years, O3b mPower flies in a different neighborhood entirely. The three new satellites, tagged F11, F12 and F13, are headed for medium Earth orbit at roughly 5,000 miles up, more than ten times higher than Starlink’s shell around 340 miles but still a small fraction of the 22,000 miles where old school geostationary satellites sit. That middle position is the whole point, because a satellite that far out needs far fewer siblings to blanket the globe than a low orbit constellation does. Essentially, SES only needed 13 satellites total to build a network offering quick, steady service that used to require thousands of spacecraft.

With most people having heard plenty about Starlink and almost nothing about O3b mPower, SES and SpaceX already blend the two networks for some customers. Both SpaceX and SES sell satellite broadband, but they’re aimed at different buyers. Starlink is built for volume, direct to consumers, RVs, homes, small businesses, plus a growing aviation and maritime business. O3b mPower skips consumers entirely and sells enterprise grade connectivity to airlines, cruise lines, offshore energy operators, telecoms needing backhaul, and governments, priced and provisioned more like a dedicated circuit.

A 2023 partnership lets cruise ships combine Starlink’s speed with O3b mPower’s steady capacity depending on what a ship needs at a given moment. Sunday’s completed 13 satellite constellation effectively finishes the medium orbit half of that pairing, years after.

Sunday’s mission was already a something on SpaceX’s manifest well before O3b mPower entered the picture. This flight marked its 29th trip to orbit, a history that includes two crewed Axiom missions, the European Space Agency’s Euclid telescope and 22 separate Starlink batches. SpaceX has landed boosters on the droneship A Shortfall of Gravitas so often that Sunday’s touchdown attempt, if it went as planned, was set to be the 661st successful Falcon booster landing to date.

For a company that pushed the Starlink constellation past 11,000 satellites back in August, almost entirely through bulk launches from California, Sunday’s flight was a reminder that SpaceX’s schedule still has room for someone else’s satellites too. SES gets a finished network built for a narrower set of customers, and Falcon 9 gets one more line on an already long resume.

Advertisement
-
Continue Reading

Elon Musk

Tesla gives the Roadster an official “Go for launch” demonstration date

Tesla teased an October 1 Roadster reveal, reviving years of delayed SpaceX thruster hover promises.

Published

on

By

Concept rendering of a Tesla Roadster with SpaceX Package via Grok
Concept rendering of a Tesla Roadster with SpaceX Package via Grok

Tesla teased an October 1 event date for its next generation Roadster, posting an image on X Saturday that shows the car lit up like it is sitting on a launch pad, with the date “10.01” stamped across the bottom and the caption “Go for launch.” A countdown clock on Tesla’s Roadster order page now points to the same date, which falls on a Thursday. The company has not said where the event will happen or whether it will be streamed at the moment. Stay with us @Teslarati for live updates.


Tesla has since sent formal invitations to reservation holders confirming the event will take place in Waco, Texas, about 90 minutes north of its Austin headquarters, based on a digital ticket shared on X by Sawyer Merritt. Tesla did not name the exact venue, though Waco sits close to SpaceX’s McGregor, Texas, rocket test site, previously reported as the planned location for a Roadster thruster demonstration. The invite sets the reveal for 8:30 p.m. Eastern on October 1, requires RSVPs by midnight on September 16, and limits entry to guests 21 and older. Invitations are non-transferable.

The tease follows nine years of a project defined by unimaginable specs along with slipped dates. Musk first showed the second generation Roadster in November 2017 as a surprise reveal at the end of the Tesla Semi event, promising a 0 to 60 mph time under two seconds, a top speed above 250 mph, 620 miles of range from a 200 kWh battery, and production starting in 2020. At last November’s shareholder meeting, Musk set an April 1 demo date and joked the choice gave him “deniability” if it slipped again, which it did, moving first to late April, then to “a month or so,” then to August.

Tesla Roadster SpaceX Package’s 1.1-second 0-60 mph launch visualized in concept video

Whatever Tesla shows on October 1 is expected to center on the SpaceX developed thruster package Musk has described since 2018. Internally code named A71, a nod to the Lockheed SR-71 Blackbird, the system reportedly uses cold gas thrusters fed by a composite overwrapped pressure vessel, the same tank design SpaceX uses on Falcon 9. Musk has said a thruster equipped Roadster could hit 60 mph in about 1.1 seconds under roughly 2.75 g of launch force, well past the 1.9 second figure quoted for the standard car. That version reportedly will not be street legal and has reportedly been discussed as a limited run sold through a track only program.

Advertisement
-

The standard Roadster is still expected to carry the original $200,000 base price and $250,000 Founders Series tier, both set when Tesla opened $50,000 and $250,000 reservations in 2017. Tesla VP of Vehicle Engineering Lars Moravy has confirmed production will happen at Gigafactory Texas, with Musk targeting 2027 or 2028, 12 to 18 months after whatever the company demonstrates next month.

Continue Reading

News

Tesla plans big safety improvements for Full Self-Driving v15

Published

on

Credit: Tesla

Tesla is planning to roll out some pretty significant safety and accident avoidance features with Full Self-Driving version 15, which will be the next major FSD deployment from the company.

Tesla AI lead Ashok Elluswamy used a near-miss this week to preview what the company says is the next leap in Full Self-Driving.

In response to a driver whose car had swerved away from another vehicle pulling out of a parking lot, Elluswamy wrote that he was glad the owner was safe and that “even earlier prediction of hazards, even faster reaction time and overall significantly better safety and collision avoidance” would arrive with FSD v15.

The comment landed as Tesla continues to treat software as the primary safety upgrade path. v15 is described internally as a larger architectural step, with a much bigger neural network and tighter coupling between prediction and control.

Advertisement
-

The company has already begun using early v15 software in some robotaxi operations while rolling out safety features such as Automatic Collision Evasion into current customer cars, allowing the driving stack to intervene even when the driver is in manual control.

Tesla is rolling out a new FSD version with a massive safety addition

Tesla’s published telemetry is the backbone of its safety argument. In recent North American Vehicle Safety Report data, vehicles with FSD (Supervised) engaged traveled roughly 5.1 million to 5.7 million miles between major collisions, defined as airbag-deployment events.

Tesla’s estimate of the U.S. average over the same period is about 699,000 miles per comparable crash. That is the comparison Tesla often frames as roughly seven times fewer major collisions.

A tighter comparison uses the same Tesla fleet. Cars driven manually with active safety features such as automatic emergency braking still recorded a major collision about every 2.1 million miles. Against that baseline, FSD’s advantage shrinks to roughly 2.4 to 2.7 times fewer severe crashes, which independent researchers argue is the more apples-to-apples figure.

European data released in 2026 pointed in the same direction: Tesla reported FSD as 3.5 times safer than manual driving in the Netherlands and 4.1 times fewer collisions than manually driven Teslas with active safety across more than 100 million kilometers in five approved countries.

Advertisement
-

Those numbers do not settle every debate. NHTSA’s Standing General Order still shows Tesla accounting for the large majority of U.S. Level 2 driver-assist crash reports, in part because the fleet logs far more assisted miles than rivals. Critics also note that Tesla’s “U.S. average” mixes crash definitions and driving mix.

Even so, Tesla’s own same-car comparisons, plus lower rates of automatic emergency braking and harsh maneuvers when FSD is engaged, are the evidence Elluswamy is pointing to when he says v15 will push prediction and collision avoidance further. The claim is not that software already eliminates risk. It is that each major version is meant to widen the gap between the system and an unaided human driver.

Continue Reading