Elon Musk’s xAI is ramping up its presence in Memphis, where it is currently housing its Colossus supercomputer cluster at a former Electrolux facility.
xAI activated Colossus in July. At the time, it was powered by 100,000 Nvidia H100 GPUs. In late October, Elon Musk stated that Colossus will soon be powered by 200,000 Nvidia H100/H200 GPUs.
The key point:
- As per the Greater Memphis Chamber, xAI is set to expand Colossus’ GPU count to at least 1 million units, as noted in an Insider report. The move is expected to bolster xAI’s position against rivals like OpenAI and Google by further enhancing the capabilities of its large language model, Grok.
- Brent Mayo, an xAI engineer, emphasized the project’s role in local and technological advancement. “In Memphis, we’re pioneering development in the heartland of America. We’re not just leading from the front; we’re accelerating progress at an unprecedented pace while ensuring the stability of the grid utilizing megapack technology,” the engineer said.
- Musk has poked fun at the update, joking that at least 1 billion GPUs will be used in the facility.
Important background:
- Colossus is particularly remarkable because it was built in just 122 days. Musk himself stated that “it took 19 days to get Colossus from hardware installation to beginning training.”
- Nvidia CEO Jensen Huang lauded xAI’s speed and effort, calling it a “superhuman” task and acknowledging Musk’s engineering acumen. Huang also stated that a project like Colossus would typically take about three years to plan and one year to set up.
- xAI’s market value has been on a tear, soaring to $50 billion. This is about double its valuation since spring this year. The company has also raised a notable $11 billion in 2024 so far.
Don’t hesitate to contact us with news tips. Just send a message to simon@teslarati.com to give us a heads up.