Tesla shared a fresh look at its Dojo AI supercomputer at the Hot Chips 34 (HC34) conference, according to to Serve The Home which attended the conference. Tesla held two talks at the conference: one on the microarchitecture of Dojo and another one on Dojo’s System-on-Wafer solution.
Tesla recently upgraded its GPU supercomputer to 7360 A-100(80GB) GPUs making it Top-7 by GPU count. However, this is not enough. This is where Dojo comes in. On AI Day last year, Tesla revealed that it was building the Dojo supercomputer in which each node has its own CPU, memory, and communication interface.
During the conference, Tesla’s software engineers who worked on the Dojo supercomputer shared new architectural features for the first time such as the Tesla Dojo instruction set. Other key features of Dojo mentioned at the conference included Dojo’s arithmetic formats, system network, and its ability to route around dead processing nodes in the software.
According to NextPlatform, one thing we have to look forward to at Tesla AI Day 2 is hearing about the performance of the Dojo system.
NextPlatform also noted that Emil Talpes, an Autopilot Hardware Engineer at Tesla who worked at AMD for almost 17 years on several Opteron processors, gave the Dojo presentation.
“The defining goal of our application is scalability. We have de-emphasized several mechanisms that you find in typical CPUs, like coherency, virtual memory, and global lookup directories just because these mechanisms do not scale very well when we scale up to a very large system. Instead, we have relied on a very fast and very distributed SRAM storage throughout the mesh. And this is backed by an order of magnitude higher speed of interconnect than what you find in a typical distributed system,” Talpes said at the end of the presentation.
For a quick recap on Tesla’s Dojo supercomputer, you can rewatch Tesla’s AI Day presentation from last year below.
Note: Johnna is a Tesla shareholder and supports its mission.