Connect with us
Tesla Autopilot construction zone lane Tesla Autopilot construction zone lane

News

Tesla is patenting a clever way to train Autopilot with augmented camera images

Tesla Autopilot construction zone lane (Credit: YouTube/Cf Tesla)

Published

on

Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.

As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.

Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.

With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”

A block diagram of an environment for computer model training. (Credit: Patentscope.wipo.int)

Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.

Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:

“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”

Advertisement
-

Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.

“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”

Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.

Simon is an experienced automotive reporter with a passion for electric cars and clean energy. Fascinated by the world envisioned by Elon Musk, he hopes to make it to Mars (at least as a tourist) someday. For stories or tips--or even to just say a simple hello--send a message to his email, simon@teslarati.com or his handle on X, @ResidentSponge.

Advertisement
Comments

News

NHTSA just escalated its Tesla Cybercab investigation in a big way

NHTSA escalated its Cybercab audit into a sworn Special Order with a September 30 deadline.

Published

on

By

Federal regulators have moved from asking Tesla questions about its Cybercab to demanding sworn answers. The National Highway Traffic Safety Administration issued a Special Order that requires a Tesla officer to sign an affidavit attesting to the completeness of the company’s responses, with a deadline of September 30.

The order builds on Audit Query AQ26002, which NHTSA opened on September 3, the same day Tesla began commercial Cybercab service in Austin. Teslarati covered that initial inquiry when it surfaced, noting the agency wanted to understand how Tesla certified a vehicle with no permanently attached steering wheel, pedals, or mirrors as compliant with Federal Motor Vehicle Safety Standards. A Special Order is a different tool and converts a fact finding review into a legally enforceable demand, the same mechanism NHTSA used against Tesla in 2023 during its Autopilot investigation.

Several of the 21 requests target a specific gap in Cybercab’s design. One asks whether Tesla used temporarily attached human controls at any point to help certify the vehicle, and if so, which standards depended on that equipment being present. Another quotes an existing rule directly: “The service brakes shall be activated by means of a foot control.” Cybercab has no foot pedal. NHTSA wants a detailed explanation of how the vehicle satisfies that requirement, and how it complies without the kind of exemption granted to Zoox in July under Part 555, the regulatory pathway built for steering wheel free vehicles.

The order does not claim Cybercab is unsafe or that Tesla broke a rule. It requires Tesla to explain, under oath, the reasoning behind decisions the company already made when it self-certified the vehicle. That distinction matters, but so does the exposure. Motor1’s reporting, summarized here, put potential civil penalty exposure as high as $139 million if NHTSA later finds the certification was flawed, on top of whatever criminal risk comes with a false sworn statement.

Tesla has not said publicly how it plans to respond. Cybercab is still carrying passengers in Austin through the Robotaxi app while the September 30 deadline approaches, and the company has continued expanding the vehicle’s footprint even as the regulatory question remains open. The Special Order does not pause any of that and just sets a date by which Tesla has to put its certification logic on the record, with a company officer’s name attached to it.

Continue Reading

Investor's Corner

Tesla uber bull Ron Baron says ‘the time to buy the stock is now’

Published

on

Credit: Tesla

In a new interview on Wednesday, Tesla uber bull Ron Baron said that anyone looking to buy the company’s stock should do so as soon as they can.

Baron, founder and CEO of Baron Capital and one of Tesla’s most persistent institutional bulls, used a CNBC Squawk Box appearance on Wednesday to deliver a familiar message with fresh urgency: In his opinion, Tesla stock is a buy:

“The time to buy the stock is now. FSD is catching on, and it’s going to be bigger and bigger. 55% of new buyers are buying it (Teslas) with FSD. It’s going to be everywhere. It’s safer.”

The Baron Capital frontman’s case is built around Full Self-Driving. Tesla reported 1.48 million active FSD subscriptions in the second quarter, up 56 percent year over year, and company officials have said roughly 55 percent of new North American deliveries left with a subscription enabled.

Baron framed that attach rate as proof the product is moving from enthusiast extra to default expectation, and as a reason software, not just vehicle volume, should drive the next phase of value.

His conviction on Tesla shares is not theoretical, as Baron Capital made its first Tesla investment in 2014, after years of meetings that began around the 2010 IPO roadshow. The firm later built a large SpaceX position starting in 2017.

Advertisement
-

Baron said those Musk-led bets have generated about $30 billion of the $71 billion in profits Baron Capital has produced for clients. He put the firm’s current exposure at roughly $25 billion in SpaceX and $5 billion in Tesla. Personally, he described SpaceX as his largest holding, at about $5 billion, with about $1.5 billion in Tesla and additional Tesla exposure through the firm’s funds.

That concentration is also a statement of loyalty. Asked about talk of a SpaceX-Tesla combination, Baron said he had already walked Elon Musk through arguments for and against a deal, then declined to repeat them on air. His public position was simpler: “Whatever you decide is better is what I’m going to support,” he said to Musk.

Baron also said that he picked up the farewell edition of the Model S after Tesla decided to sunset the vehicle earlier this year, calling it his favorite car he’s ever driven.

Advertisement
-
Continue Reading

Elon Musk

SpaceX’s next Starship launch is about to attempt its biggest leap yet

SpaceX targets September 22 for Starship Flight 14, its first attempt to reach real orbit.

Published

on

By

SpaceX has set September 22 as the target date for Starship’s 14th test flight, and this one carries a different goal than any of the 13 that came before it. Every previous Starship mission has intentionally flown a suborbital arc, reentering the atmosphere within the same hour it launched. Flight 14 is designed to send the craft into a genuine orbit around Earth for the first time.

The launch window opens at 7:15 a.m. Central time at Starbase in South Texas and runs for 75 minutes, pending regulatory approval, according to SpaceX’s mission description published Tuesday. If the flight goes as planned, Starship will circle the planet roughly six times at an altitude near 275 kilometers over about ten hours before a deorbit burn sends it toward a splashdown in the Pacific Ocean west of Chile, a departure from the Indian Ocean recoveries used on the last several flights.

The mission also marks the first attempt to put a working batch of Starlink V3 satellites into actual service. Flight 13 carried 20 of the new satellites in July, but because that mission never left a suborbital trajectory, the payload reentered along with the ship instead of separating into orbit.

SpaceX tells the FCC that Starship Flight 14 is going to orbit

Each V3 satellite is rated for roughly one terabit per second of downlink capacity, so a successful deployment on Flight 14 would be SpaceX’s largest single jump in network bandwidth since Starlink began flying on Falcon 9.

Flight 13 still did the heavier lifting on the technical side. That July mission flew a deliberately more stressful reentry profile to test Starship’s heat shield, and the ship survived its softest splashdown yet, intact enough for drone inspections shortly after landing. Elon Musk said the flight delivered “all the heat shield data we needed and then some,” a result Teslarati covered in detail when he later said SpaceX had solved the vehicle’s biggest reusability challenge. Flight 14 is where SpaceX starts spending that confidence on an actual orbital insertion rather than another controlled fall back to Earth.

Advertisement
-

One thing Flight 14 will not attempt is a tower catch of the ship. Musk floated the idea right after Flight 13, but walked the timeline back in August, saying a catch attempt was more likely “in a few months.” The Super Heavy booster will still aim for its own recovery, targeting an offshore landing point in the Gulf of America, the same approach used on recent flights.

September 22 is SpaceX’s own target, not a locked date. Starship’s schedule has slipped before over hardware readiness and FAA sign off, and the company has said as much in its own mission notes. But the plan itself represents the clearest marker yet that Starship is moving from a suborbital test program into something meant to carry paying payloads and, eventually, people.

Continue Reading