News
Tesla is patenting a clever way to train Autopilot with augmented camera images
Tesla is currently tackling what could only be described as its biggest challenge to date. In his Master Plan, Part Deux, CEO Elon Musk envisioned a fleet of zero-emissions vehicles that are capable of driving on their own. Tesla has made steps towards this goal with improvements and refinements to its Autopilot and Full Self-Driving suites, but a lot of work remains to be done.
As noted by Tesla during its Autonomy Day presentation last year, attaining Full Self-Driving is largely a matter of training the neural networks used by the company. Tesla adopts what could be described as a somewhat organic approach for autonomy, with the company using a system that is centered on cameras and artificial intelligence — the equivalent of a human primarily using the eyes and brain to drive.
Tesla’s camera-centric approach may be quite controversial due to Elon Musk’s strong stance against LiDAR, but it is gaining ground, with other autonomous vehicle companies such as MobilEye developing FSD systems that rely primarily on visual data and a trained neural network. This approach does come with its challenges, as training neural networks requires tons of data. Tesla emphasized this point as much during its Autonomy Day presentation.
With this in mind, it is pertinent for the electric car maker to train its neural networks in a way that is as efficient as possible with zero compromises. To help accomplish this, Tesla seems to be looking into the utilization of augmented data, as described in a recently published patent titled “Systems and Methods for Training Machine Models with Augmented Data.”
Teslas are equipped with a suite of cameras that provide 360-degree visual coverage for the vehicle. In the patent’s description, Tesla noted that images used for neural network training are usually captured by various sensors, which, at times, have different characteristics. An example of this may lie in a Tesla’s three forward-facing cameras, each of which has a different field of view and range as the other two.
Tesla’s recent patent describes a system that allows the company to process these images in an optimized manner. Part of how this is done is through augmentation, which opens the doors to flexible and widespread neural network training, even when it involves vehicles equipped with differently-specced cameras. The electric car maker describes this process as such:
“Augmentation may provide generalization and greater robustness to the model prediction, particularly when images are clouded, occluded, or otherwise do not provide clear views of the detectable objects. These approaches may be particularly useful for object detection and in autonomous vehicles. This approach may also be beneficial for other situations in which the same camera configurations may be deployed to many devices. Since these devices may have a consistent set of sensors in a consistent orientation, the training data may be collected with a given configuration, a model may be trained with augmented data from the collected training data, and the trained model may be deployed to devices having the same configuration.”
Among the most notable aspects of Tesla’s recent patent is the use of “cutouts,” which allow Tesla’s neural networks to be trained using an optimized set of images. This was something that was discussed by former Tesla Autopilot engineer Eshak Mir in a Third Row Podcast interview, where he hinted at a system adopted in the electric car maker’s ongoing Autopilot rewrite that helped lay out “all the camera images” from a vehicle “into one view.” Such a process has the potential to help Tesla with 3D labeling, especially since the images used for neural network training are stitched together. Tesla’s patent seems to reference a system that is very similar to that described by the former Autopilot engineer.
“As a further example, the images may be augmented with a“cutout” function that removes a portion of the original image. The removed portion of the image may then be replaced with other image content, such as a specified color, blur, noise, or from another image. The number, size, region, and replacement content for cutouts may be varied and may be based on the label of the image (e.g., the region of interest in the image, or a bounding box for an object).”
Tesla is aiming to release a feature-complete version of its Full Self-Driving suite as soon as possible. Elon Musk remains optimistic about this, despite the company missing its initial timeline that was set at the end of 2019. That being said, Elon Musk did mention previously that Tesla is working on a foundational rewrite of Autopilot. In a tweet early last month, Musk stated that an essential part of the rewrite involves work on Autopilot’s core foundation code and 3D labeling. Once done, the CEO indicated that additional functionalities could be rolled out quickly. This recent patent, if any, seems to give a glimpse at how these improvements are being done.
Energy
Taiwan power firm bets big on Tesla Megapacks as AI data center thirst grows
Taiwan’s Apollo Power orders 260 Tesla Megapacks, betting AI data centers will need more electricity.
A Taiwanese power company has ordered 260 Tesla Megapack 2XL units, roughly 1GWh of storage, betting that electricity rather than chips will decide how fast AI data centers come online. Apollo Power signed the contract with Tesla’s Taiwan energy arm on October 8, according to the company’s filing with the Taiwan stock exchange.
The filing says the units are meant for behind the meter storage and resilient grid projects, with delivery in batches from the end of 2027 and sites operating in 2028. Apollo and Tesla did not disclose a price, citing a confidentiality agreement. Apollo plans to pay for the equipment through a cash capital increase and bank financing, and it noted the purchase is not revenue and does not mean it holds firm orders from its partners.
Biggo reported that only about 33 units are under signed customer contracts, with roughly 150 more in negotiation. The same report put total project investment near NT$130 billion, but that figure includes installation work, with the equipment itself making up about 40 to 50 percent. In other words, a large share of this order is Apollo reserving supply ahead of demand. Apollo’s chairman told local media global large storage supply is tight, which is why the company locked in the units early, according to Yahoo Taiwan Finance.
Apollo also launched an AI data center and grid resilience alliance with Phison, Gigabyte and Shihlin Electric. The pitch rests on a pilot at Phison’s Zhunan plant, where Apollo and Tesla Energy Taiwan installed grid forming Megapack 2XLs in June. Tesla Energy Taiwan general manager Sun Hsiao-Han said the project went from demo to 1GWh in four months, and that Tesla Energy has deployed more than 50GWh worldwide.
Taiwan’s cabinet has approved amendments that would require large energy users to install their own generation and storage, which gives enterprise storage a regulatory tailwind.
The deal fits a pattern Teslarati has tracked. Tesla’s Megablock targets 1GWh commissioned in 20 business days, a pitch aimed squarely at fast moving data center builds. Megapacks already back xAI’s Colossus in Memphis and a 400MW AI campus in Brazil. Sun Media adds that Megapack 2XL was named among the first products in Nvidia’s DSX Ready qualification for battery storage in AI factories.
Deliveries are more than a year away, so the real test is how much of that 260 unit order turns into signed customer contracts.
For a century, the car has asked the same thing of its owner, which is basically to pay attention. Even the best ride-hailing app like Uber or Lyft only moved the problem into someone else’s hands. You stopped driving, but you still sat in a stranger’s car, while half listening, and wondering whether to take a phone call in the car without being rude. The Cybercab has no wheel, no pedals and no driver. With Grok @bot now live inside it, it becomes something else entirely, and I think that matters more than the fleet count that everyone is tracking.
Grok @bot is an agent, not a voice assistant. A rider signs in once, and from then on can say what they need and watch it happen. Order dinner from a favorite restaurant. Fill the grocery cart. Move money, answer email, adjust a calendar, push a work project forward. Tesla engineer Yun-Ta Tsai confirmed it is enabled across every Cybercab and the rest of the Model Y fleet, and Tesla’s own pitch is that Full Self-Driving handles the commute while Grok handles the errands.
Think about what that does to a thirty minute ride. Uber sells you a trip, while Waymo sells you the same trip in a more private setting. Tesla is selling the thirty minutes themselves, returned to you with an assistant attached. Errands that used to eat a Saturday get done between pickup and drop off. The ride stops being dead time between two places and becomes a place of its own.
Tesla integrates Grok Bot into its vehicles for the ultimate personal assistant
Best of all, your Bots follow you into any Cybercab, and presumably your own Tesla, then logs you out automatically when you step out. Anyone can bolt a chatbot onto a screen, but much harder to own the vehicle, the autonomy stack and the assistant at once. It also quietly ties SpaceX’s Super Intelligence business to Tesla’s fleet, after Tesla converted its xAI stake into SpaceX equity. The relationship between those two companies is getting harder to describe as separate.
This is also where the hard questions start, because an agent that can spend your money in a vehicle other people also ride in needs some answers. How does it handle a voice it shouldn’t trust? What gets recorded, who can see it, and for how long? What happens when a rider leaves halfway through a task?
If Tesla gets those answers right, the scoreboard changes. We have been counting Cybercabs and wait times. The more telling number may be what riders accomplish per trip. That is data almost no one else will ever have, and it is the real reason to watch Grok @bot.
What do you think? Leave me a comment.
News
Why Elon Musk’s target date for buying an Optimus robot is harder than it sounds
Musk says China’s magnet export ban cost Optimus months, and robot hands remain shockingly hard.
Elon Musk says China’s export curbs on magnets used in electric motors set back Tesla’s Optimus program by two to three months, and that the robot’s hands remain the hardest part of the project. He made the comments Saturday during a live interview with Tesla Owners Silicon Valley at the X Takeover event at Giga Texas, in front of roughly 2,000 attendees. The Optimus discussion begins around the 49 minute mark of the stream.
On the hands, Musk was blunt and noted that a general purpose humanoid needs fine motor skills that are “incredibly difficult to get right. Shockingly difficult,” he said in the interview. He pointed to a problem he says no one has demonstrated yet: “One of the hardest things also generally is to do precision and force at the same time,” such as driving a small self tapping screw into wood. Touch sensors add another layer, since it is “actually kind of hard to get touch sensors that elastically deform and can handle high cycle life and it won’t just break after a few thousand cycles.”
He also took aim at rival robot videos. “Any complicated looking robot demo, it’s a canned demo. It’s not real,” Musk said, adding that real AI means showing a robot something it has never seen and watching it still cope, the way FSD handles a city it has never driven.
Musk expects the Optimus ramp to be slower than a new vehicle production ramp. “The initial part of that S curve of production ramp is going to be very slow at the beginning,” Musk said, partly because “unlike cars, there’s no pre existing supply chain for robot stuff.” That echoes his July reply on X that Optimus production “will be extremely slow at first, as everything is new.”
The magnet disclosure puts a number on a problem Musk first raised in 2025. He said China “prevented the export of magnets for use in electric motors” and that it “set back the Optimus program by like two or three months.” Then he added: “We had robots with no arms.” Tesla’s fix is to bring the supply chain home. “You’ve got to be able to make all the parts domestically,” he said. Teslarati reported in April 2025 that Musk tied the issue to permanent magnets in the arm actuators, where motors have to be as small as possible.
Tesla Optimus V3 hand and arm details revealed in new patents
Production is moving regardless, with a reported internal goal of more than 1,000 a week by the end of 2026. Teslarati recently covered how Tesla is certifying supply partners in China for Optimus parts, while the company’s hand design has been the subject of new patent filings.
Musk has said he wants Optimus available to the public by late 2027, though Tesla has not confirmed that date or published updated production figures.
Taiwan power firm bets big on Tesla Megapacks as AI data center thirst grows
Tesla Cybercab and Robotaxi’s real advantage won’t be the ride service
Why Elon Musk’s target date for buying an Optimus robot is harder than it sounds
The Pentagon taps Elon Musk to design the battlefield of the future
Tesla Roadster’s new patent preps white-knuckle speeds, keeping it grounded
