Elon Musk does not want Tesla to be just another automaker. He wants Tesla to be an AI business that has figured out how to make vehicles drive themselves.
Tesla's custom-built supercomputer, Dojo, is critical to the aim of training its Full Self-Driving (FSD) neural networks. FSD is not entirely self-driving; it can do some automated driving activities but still requires a human driver. Tesla, on the other hand, believes that with more data, computation capacity, and training, it will be able to pass the threshold from almost self-driving to fully autonomous.
And this is where Dojo comes in.
Musk has been teasing Dojo for some years, but the CEO has stepped up conversations regarding the supercomputer during 2024. Dojo's importance to Tesla may be existential; with EV sales declining, investors want assurances that Tesla can attain autonomy. Here's a history of Dojo mentions and promises.
2019: First mentions of Dojo.
April 22 – At Tesla's Autonomy Day, the carmaker brought its AI team on stage to discuss Autopilot and Full Self-Driving, as well as the AI that powers both. The business provides details regarding Tesla's custom-built chips, which are specifically developed for neural networks and self-driving automobiles.
During the event, Musk teases Dojo, explaining that it is a supercomputer for training artificial intelligence. He also mentions that all Tesla cars made at the time would have all of the hardware needed for complete self-driving and would just require a software update.
2020
Musk starts the Dojo roadshow.
Feb 2 - Musk claims that Tesla will soon have over a million connected vehicles worldwide, equipped with the sensors and computing power required for full self-driving, and he praises Dojo's capabilities.
"Dojo, our training supercomputer, will be able to process massive amounts of video training data while efficiently running hyperspace arrays with a large number of parameters, ample memory, and ultra-high bandwidth between cores. More about this later."
August 14 - Musk reiterates Tesla's intention to create Dojo, a neural network training computer, "to process truly vast amounts of video data," describing it as "a beast." He also believes the initial version of Dojo is "about a year away," putting its release date at around August 2021.
2021
Tesla makes Dojo official.
August 19 – Tesla officially introduces Dojo at its first AI Day, an event designed to draw engineers to the company's AI team. Tesla also reveals its D1 chip, which the carmaker claims will power the Dojo supercomputer alongside Nvidia's GPU. Tesla said its AI cluster will contain 3,000 D1 chips.
Tesla publishes a Dojo Technology whitepaper titled "a guide to Tesla's configurable floating point formats & arithmetic." The whitepaper describes a technological standard for a new type of binary floating-point arithmetic used in deep learning neural networks. It can be implemented "entirely in software, entirely in hardware, or in any combination of software and hardware."
2022
Tesla announces Dojo development.
August 12 – Musk stated that Tesla will "phase in Dojo." "Won't need to buy as many additional GPUs next year."
September 30 – At Tesla's second AI Day, the firm announces that the first Dojo cabinet has been installed, with 2.2 megawatts of load testing. Tesla claims it was producing one tile every day (which is made up of 25 D1 chips). Tesla shows Dojo on stage using a Stable Diffusion model to make an AI-generated image of a "Cybertruck on Mars."
Importantly, the business announces that a full Hexapod cluster will be finished by Q1 2023, and that it intends to create a total of seven Exapods in Palo Alto.
2023
A 'long-shot bet'
During Tesla's first-quarter earnings call on April 19, Musk tells investors that Dojo "has the potential for an order of magnitude improvement in the cost of training," as well as "the potential to become a sellable service that we would offer to other companies in the same way that Amazon Web Services offers web services."
Musk also says he'd "look at Dojo as kind of a long-shot bet," but it's a "bet worth making."
Nvidia Supply Challenges
July 23 - During Tesla's second-quarter earnings call, Musk claimed that demand for Nvidia hardware is "so high that it's often difficult to get the GPUs."
"I think this therefore requires that we put a lot more effort on Dojo in order to ensure that we've got the training capability that we need," Musk tells me. "And we do see a path to being competitive with Nvidia with Dojo."
A graph in Tesla's investor deck indicates that Tesla AI training capacity would increase to approximately 90,000 H100 equivalent GPUs by the end of 2024, up from around 40,000 in June. Later that day on X, Musk announces that Dojo 1 will have "roughly 8k H100-equivalent of training online by end of year."
XXX: July 30 Musk responds to a post from someone claiming to create a club of "Tesla HW4/AI4 owners angry about being left behind when AI5 comes out" by stating that AI5 is approximately 18 months away from high-volume production.
August 3 - Musk posts on X that he walked through "the Tesla supercompute cluster at Giga Texas (aka Cortex)." He states that it would consist of around 100,000 H100/H200 Nvidia GPUs with "massive storage for video training of FSD & Optimus."