Tesla’s AI Day is here. In a few minutes, Tesla watchers would be seeing executives like Elon Musk provide an in-depth discussion on the company’s AI efforts on not just its automotive business but on its energy business and beyond as well. AI Day promises to be yet another tour-de-force of technical information from the electric car manufacturer. Thus, it is no surprise that there is a lot of excitement from the EV community heading into the event.
Tesla has kept the details of AI Day behind closed doors, so the specifics of the actual event are scarce. That being said, an AI Day agenda sent to attendees indicated that they could expect to hear Elon Musk speak during a live keynote, speak with Andrej Karpathy and the rest of Tesla’s AI engineers, and participate in breakout sessions with the teams behind Tesla’s AI development.
Similar to Autonomy Day and Battery Day, Teslarati would be following along on AI Day’s discussions to provide you with an updated account of the highly-anticipated event. Please refresh this page from time to time, as notes, details, and quotes from Elon Musk’s keynote and its following discussions will be posted here.
Simon 19:40 PT – A question about the use cases for the Tesla Bot was asked. Musk notes that the Tesla Bot would start with boring, repetitive, work, or work that people would least like to do.
Simon 19:25 PT – A question about AI and manufacturing is asked and how it potentially relates to the “Alien Dreadnaught” concept. Musk notes that most of Tesla’s manufacturing today is already automated. Musk also noted that humanoid robots would be done either way, so it would be great for Tesla to do this project, and safely as well. “We’re making the pieces that would be useful for a humanoid robot, so we should probably make it. If we don’t someone else will — and we want to make sure it’s safe,” Musk said.
Simon 19:15 PT – And the Q&A starts. First question involves open-sourcing Tesla’s innovations. Musk notes that it’s pretty expensive to develop all this tech, so he’s not sure how things could be open-sourced. But if other car companies would like to license the system, that could be done.
Simon 19:11 PT – There will really be a “Tesla Bot.” It would be built by humans, for humans. It would be friendly, and it would eliminate dangerous, repetitive, boring tasks. This is still petty darn unreal. It uses the systems that are currently being developed for the company’s vehicles. “There will be profound applications for the economy,” Musk said.
Simon 19:06 PT – New products! A whole Tesla suit?! After a fun skit, Elon says the “Tesla Bot” would eventually be real.
Simon 19:00 PT – What is crazy is that Dojo is not even done. This is just what it is today. Dojo is still evolving, and it is going to be way more powerful in the future. Now, it’s Elon Musk’s turn. What’s next for Tesla beyond vehicles.
Simon 19:00 PT – Venkataramanan teases the ExaPOD. Yet another revolutionary solution from Tesla. With all this, it is evident that Tesla’s approach to autonomy is on a whole other level. It would not be surprising if it takes Wall Street and the market a few days to fully absorb what is happening here.
Simon 18:55 PT – The specs of Dojo are insane. Behind its beastly specs, it seems that Dojo’s full potential lies in the fact that all this power is being used to do one thing: to make autonomous cars possible. Dojo is a pure learning machine, with more than 500,000 training nodes being built together. Nine petaflops of compute per tile, 36 terabytes per second of off-tile bandwidth. But this is just the tip of the iceberg for Dojo.
Simon 18:50 PT – Ganesh Venkataramanan, Project Dojo’s lead, takes the stage. He states that Elon Musk wanted a super-fast training computer to train Autopilot. And thus Project Dojo was born. Dojo is a distributed compute architecture connected by network fabric. It also has a large compute plane, extremely high bandwidth with low latencies, and big networks that are partitioned and mapped, to name a few.
Simon 18:45 PT – Milan Kovac, Tesla’s Director of Autopilot Engineering takes the stage. He notes that he would discuss how neural networks are run in the company’s cars. He notes that Tesla’s systems require supercomputers.
Simon 18:40 PT – Ashok notes that simulations have helped Tesla a lot already. It has, for example, helped the company identify pedestrian, bicycle, and vehicle detection and kinematics. The networks in the vehicles were traded to 371 million simulated images and 480 million cuboids.
Simon 18:35 PT – Ashok notes that these strategies ultimately helped Tesla retire radar from its FSD and Autopilot suite and adopt a pure vision model. A comparison between a radar+camera system and pure vision shows just how much more refined the company’s current strategy is. The executive also touched on how simulations help Tesla develop its self-driving systems. He states that simulations help when data is difficult to source, difficult to label, or in a closed loop.
Simon 18:30 PT – Ashok returns to discuss Auto Labeling. Simply put, there is so much labeling that needs to be done that it’s impossible to be done manually. He shows how roads and other items on the road are “reconstructed” from a single car that’s driving. This effectively allowed Tesla to label data much faster, while allowing vehicles to navigate safely and accurately even when occlusions are present.
Simon 18:25 PT – Karpathy returns to talk about manual labeling. He notes that manual labeling that’s outsourced to third-party firms is not optimal. Thus, in the spirit of vertical integration, Tesla opted to establish its own labeling team. Karpathy notes that in the beginning, that Tesla was using 2D image labeling. Eventually, Tesla transitioned to 4D labeling, where the company could label in vector space. But even this was not enough, and thus, auto labeling was developed.
Simon 18:23 PT – The executive states that traffic behavior is extremely complicated, especially in several parts of the world. Ashok notes that this partly illustrated by parking lots and how they are actually complex. Summoning a car from a parking lot, for example, used to utilize 400k notes to navigate, resulting in a system whose performance left much to be desired.
Simon 18:18 PT – Ashok notes that when driving alongside other cars, Autopilot must not only think about how they would drive, they must also think about how other cars would operate. He shows a video of a Tesla navigating a road and dealing with multiple vehicles to demonstrate this point.
Simon 18:15 PT – Director of Autopilot Software Ashok Elluswamy takes the stage. He starts off by discussing some key problems in planning in both non-convex and high-dimensional action spaces. He also shows Tesla’s solution to these issues, a “Hybrid Planning System.” He demonstrates this by showing how Autopilot performs a lane change.
Simon 18:10 PT – Karpathy’s discussion notes that today, Tesla’s FSD strategy is a lot more cohesive. This is demonstrated by the fact that the company’s vehicles could effectively draw a map in real-time as it drives. This is a massive difference compared to the pre-mapped strategies employed by rivals in both the automotive and software field like Super Cruise and Waymo.
To solve several problems encountered over the last few years with the previous suite, Tesla re-engineered their NN learning from the ground up and utilized a multi-head route, camera calibrations, caching, queues, and optimizations to streamline all tasks.
(heavily simplified) pic.twitter.com/LG2TRgjxip
— Teslascope (@teslascope) August 20, 2021
Simon 18:05 PT – The AI Director discusses how Tesla practically re-engineered their neural network learning from the ground-up and utilized a multi-head route. These include camera calibrations, caching, queues, and optimizations to streamline all tasks. Do note that this is an extremely simplified iteration of Karpathy’s discussion so far.
Simon 18:00 PT – Karpathy covers more challenges that are involved in even the basics of perception. Needless to say, AI Day is quickly proving to be Tesla’s most technical event right off the bat. That said, multi-camera networks are amazing. They’re just a ton of work, but it may very well be a silver bullet for Tesla’s predictive efforts.
Simon 17:56 PT – Karpathy showcases a video of how Tesla used to process its image data in the past. He shows a popular video for FSD that has been shared in the past. He notes that while great, such a system proved to be inadequate, and this is something that Tesla learned when it launched Smart Summon. While per-camera detection is great, the vector space proves inadequate.
Simon 17:55 PT – Karpathy noted that when Tesla designs the visual cortex in its car, the company is modeling it to how a biological vision is perceived by eyes. He also touches on how Tesla’s visual processing strategies have evolved over the years, and how it is done today. The AI Director also touches on Tesla’s “HydraNets,” on account of their multi-task learning capabilities.
Simon 17:51 PT – Karpathy starts off by discussing the visual component of Tesla’s AI, as characterized by the eight cameras used in the company’s vehicles. The AI director notes that AI could be considered like a biological being, and it’s built from the ground up, including its synthetic visual cortex.
Simon 17:48 PT – Elon Musk takes the stage. He apologizes for the event’s delay. He jokes that Tesla probably needs AI to solve these “technical difficulties.” The CEO highlights that AI Day is a recruitment event. He calls Tesla’s head of AI Andrej Karpathy. There’s no better person to discuss AI.
Simon 17:45 PT – We’re here watching the AI Day FSD preview video and we can’t help but notice that… are those Waypoints?!
Simon 17:38 PT – Looks like we’ve got an Elon sighting! And a preview video too! Here we go, folks!
We’ve got an Elon sighting
— Rob Maurer (@TeslaPodcast) August 20, 2021
Simon 17:30 PT – A 30-minute delay. We haven’t seen this much delay in quite a bit.
Simon 17:20 PT – It’s a good thing that Tesla has great taste in music. Did Grimes mix this track?
Simon 17:15 PT – We’re 15 minutes in. “Elon Time” is going strong on AI Day. To be honest, though, this music would fit the “Rave Cave” in Giga Berlin this coming October.
Simon 17:10 PT – A good thing to keep in mind is that AI Day is a recruitment event. Some food for thought just in case the discussions take a turn for the extremely technical. AI Day is designed to attract individuals who speak Tesla’s language in its rawest form. We’re just fortunate enough to come along for the ride.
Tesla Board Member Hiro Mizuno sums it up in this tweet pretty well.
Anybody passionate about real world AI !! https://t.co/ydaWQlkE4O
— HIRO MIZUNO (@hiromichimizuno) August 20, 2021
Simon 17:05 PT – I guess AI Day is starting on “Elon Time?” We’re on to the next track of chill music.
Simon 17:00 PT – And with 5 p.m. PST here, the music is officially live on the AI Day live stream. Looks like we’re in for some wait. Wonder how many minutes it would take before it starts? Gotta love this chill music though.
Simon 16:58 PT – While waiting, I can’t help but think that a ton of TSLA bears and Wall Street would likely not understand the nuances of what Tesla would be discussing today. Will Tesla go three-for-three? It was certainly the case with Battery Day and Autonomy Day.
Made it pic.twitter.com/aAWqxgf0bP
— Johnna (@JohnnaCrider1) August 19, 2021
Simon 16:55 PT – T-minus 5 minutes. Some attendees of AI Day are now posting some photos on Twitter, but it seems like photos and videos are not allowed on the actual venue of the event. Pretty much expected, I guess.
Simon 16:50 PT – Greetings, everyone, and welcome to another Live Blog. This is Tesla’s most technical event yet, so I expect this one to go extremely in-depth on the company’s AI efforts and the technology behind it. We’re pretty excited.
Don’t hesitate to contact us with news tips. Just send a message to tips@teslarati.com to give us a heads up.
Elon Musk
Tesla reveals Optimus Academy, where robots teach themselves
Tesla has begun building production quality Optimus robots in Fremont and revealed its training plan.
Tesla has started building Optimus on the high volume line it set up inside its Fremont factory, and the company says the robots coming off it are the first that meet the bar for a real product. “The bots coming off this line are the first production-quality bots that we have manufactured,” Tesla wrote in its third quarter shareholder letter, adding that it has “proven that the current manufacturing process for Optimus is suitable for a commercial product.”
The line sits in space that built the Model S and Model X until this spring. Tesla cleared the old vehicle line in 46 days, as Teslarati reported in July, to make room for robot production sized for one million units a year. On Wednesday’s earnings call, Elon Musk said most of the equipment for that capacity is already in Fremont, with output set to climb toward roughly 20,000 robots a week as the line ramps.
Musk described the ramp as a classic S curve. “The growth of Optimus is an S curve, but the initial curve is a shallow slope,” he said. The team is now refining the robot’s AI, electromechanical design, reliability and service life, and Musk expects that work to be finished by the end of next year, when Tesla plans to start delivering Optimus to outside companies for dangerous, repetitive and around the clock jobs.
The most interesting part of the call was how Tesla plans to teach the robots. Its cars send real driving data back to the AI team from millions of vehicles, but Optimus has no fleet like that yet. “A big difference with Optimus versus the cars is we don’t have this vast fleet of cars from which we can gather training data,” Musk said. Tesla’s answer is an Optimus Academy, where 10,000 to 20,000 robots will practice a wide range of tasks and feed what they learn into the next version of the model. Every unit off the Fremont line becomes a source of that training data.
Smarter hardware is on the way too. Tesla plans to put its AI5 chip into Optimus next year, and Musk said a new Optimus video should arrive within one or two months showing the robot completing tasks driven by AI rather than preprogrammed motions. He took aim at rival humanoid demos that only run scripted routines, saying those would not hold up in the real world.
The supply chain is lining up behind the program. In September, Tesla certified Tuopu, Joyson and Sanhua as mass production partners for Optimus components. A far larger Optimus factory is also rising at Giga Texas, where drone footage from Joe Tegtmeyer showed the steel frame nearing completion, and Musk has pegged that plant’s long term target at 10 million robots a year.
Tesla executives also said Optimus production costs are trending toward the expected manufacturing cost of Cybercab, a sign the robot is being engineered for volume from the start. The hands remain the toughest piece of the design, a point Musk also made at the X Takeover event earlier this month, as Teslarati covered at the time, and that is where much of the current engineering effort is focused.
Elon Musk
The Starship moon landing number NASA and Elon Musk can’t agree on
Musk disputes NASA estimate of 22 Starship launches for Moon landing, predicting 6 to 8.
Elon Musk is disputing a new NASA safety panel estimate of how many Starship launches it will take to put astronauts on the Moon, saying the real number will be less than half the panel’s figure.
On Thursday, NASA’s Aerospace Safety Advisory Panel held its quarterly public meeting. Panel member Charlie Precourt, a former NASA astronaut, flagged the fueling process for SpaceX’s lunar lander as a major risk for Artemis IV, the mission planned to land a crew on the Moon in 2028. “We need to keep our eyes on numerous required fuelings for the lander,” Precourt said, according to Ars Technica. “I think the last estimate is around 22 flights.”
Ars Technica’s Eric Berger posted the comment on X, and Musk replied. “That’s way too high,” Musk wrote. “Actual number will be more like 6 to 8.”
The disagreement comes down to a basic problem with sending a ship as large as Starship to the Moon. Starship burns nearly all of its propellant just reaching Earth orbit, so it cannot continue on to the Moon without stopping for a refill. SpaceX’s answer is a gas station in space. First, a Starship that works as a storage tank, called a depot, is launched into orbit. Then a series of tanker Starships launch one at a time, dock with the depot and pump their leftover propellant into it. Once the depot is full, the lunar lander version of Starship launches, docks, fills up and heads for the Moon.
Every tanker trip is its own full rocket launch, which is why the count matters. More launches mean more time, more cost and more chances for something to go wrong before astronauts ever leave Earth. Berger estimated that even if SpaceX reaches twice weekly Starship launches by the end of 2027, filling a depot at the panel’s number would still take roughly three months.
How many tankers are needed depends mostly on how much propellant each one can carry to orbit and how much is lost along the way. Starship’s fuel and oxygen are stored at extremely cold temperatures, and some of it slowly boils off while waiting in orbit. More is lost in each transfer. NASA appears to be working from what Starship can lift today, while Musk’s range assumes a more capable rocket.
Musk has pushed back on higher counts before. In 2021, after a government document suggested 16 launches, he called that “extremely unlikely” and said eight tankers would be the most required.
Much of the answer depends on hardware SpaceX has not finished flying. As Teslarati noted earlier this year, the company plans to carry out the refueling process with its upgraded V3 Starship, and Precourt described Artemis IV as carrying “a large number of firsts.” He also praised NASA Administrator Jared Isaacman’s team for working to clear obstacles for both SpaceX and Blue Origin, whose Blue Moon lander does not rely on orbital refueling for its first mission.
Elon Musk
Tesla’s secretive “Project Crystal Sun” gets a big green light in Texas
Fort Bend County approved four incentive agreements Thursday for Tesla’s proposed $10.1 billion solar factory.
Fort Bend County commissioners on Thursday approved a package of tax incentives for Tesla’s proposed $10.1 billion solar cell factory near Needville, Texas, giving the company one of the last local approvals it needs before it decides where to build. All four Tesla agreements passed 4 to 0.
The votes covered a property tax abatement inside Reinvestment Zone No. 34, a Chapter 381 economic development agreement, a foreign trade zone non-objection paired with payments in lieu of certain taxes, and a separate payment in lieu of taxes “true up” agreement, according to agenda notes from the Fort Bend Commissioners Court meeting.
The project, known internally as “Project Crystal Sun,” would make solar cells and finished modules on roughly 3,000 acres near FM 762 and FM 1994, south of the Brazos River. As Teslarati reported when the filing first surfaced in August, Tesla’s application under the state’s JETI Act splits the investment into $1.5 billion in real property and $8.6 billion in equipment, with 9,712 permanent jobs projected by 2033 and commercial operations targeted for the first quarter of 2029. The plant would handle the full chain from ingot and wafer production through cell testing.
Inside Tesla’s secretive $10 Billion “Project Crystal Sun” filing
Thursday’s approval does not lock in the site. Tesla has told the Texas Comptroller it is still weighing other locations, and site selection adviser David Dennison told Lamar Consolidated ISD trustees in September that the race had narrowed to Fort Bend and one out of state rival with a “very compelling package,” according to the Houston Chronicle. Tesla hopes to decide before the end of 2026, and the incentives still need sign off from the Comptroller and the Governor’s office.
Local officials have now done nearly everything within their control. Commissioners created the reinvestment zone in an August 24 special meeting, with Interim County Judge Daniel Wong formally withdrawing himself over his engineering firm’s past work with Tesla. Lamar CISD trustees voted 7 to 0 on September 15 to advance a JETI agreement that would tax Tesla’s qualifying property at 50 percent of its value for maintenance and operations from 2029 through 2038.
The plant lines up with a target Elon Musk set on Tesla’s Q4 2025 earnings call in January, when he said the company would “work toward getting 100 GW a year of solar cell production, integrating across the entire supply chain from raw materials all the way to finished solar panels.” In March, Tesla was reported to be negotiating a $2.9 billion purchase of Chinese solar manufacturing equipment bound for Texas. The Houston area already hosts Tesla’s Megafactory in Brookshire, where Megapack 3 production has started. Tesla said on its Q3 call that Megapack supply for 2027 is nearly sold out, and Musk wrote on X Thursday that the company is “increasing production of Megapacks as much as possible,” as demand from data centers climbs.
Tesla reveals Optimus Academy, where robots teach themselves
The Starship moon landing number NASA and Elon Musk can’t agree on
