Nvidia has figured out a way to build AI factories a lot faster.
These massive data centers bring together billions of dollars of computing equipment into a facility where a ton of different tech, power, cooling, and logistical systems have to work together seamlessly. Managing that kind of complexity is one of Nvidia's strengths.
With chip design, Nvidia architects everything virtually in a simulated environment long before the first manufacturing process starts. And since today's chips and processors have become enormously complex, that orchestration is where the magic happens, in terms of speed, efficiency, and innovation.
Nvidia is applying this same mindset to building AI factories by partnering with Siemens and implementing digital twins, or virtual replicas of real-world environments. This allows Nvidia and its partners, such as hyperscalers and neoscalers, to simulate these elaborate facilities ahead of time and rehearse all of the processes that make them work.
"You’re really building a data center scale computer, along with all the technology, it’s not just about the GPU," Vladimir Troy, Nvidia's VP of AI infrastructure, told The Deep View. "This is the fundamental part that I think people miss about accelerated computing. If you just optimize the GPU, then you have the fastest GPU, and your bottlenecks could be everywhere else."
Digital twins are the key to making it all happen by allowing engineers to simulate a facility's physical behavior. Siemens brings the simulation, automation, and industrial engineering expertise while Nvidia brings the accelerated computing, its Omniverse platform, and the experience with synthetic data.
"You can improve the time-to-token by [roughly] 40% if you’re using digital twins," Ales Alajbegovic, global program manager of industrial metaverse for Siemens, told The Deep View. "If you don’t use digital twins, I don’t even know how you can build this."
In this case, time-to-token is referring to the period from the time you break ground on the facility to the time it's serving its first useful AI query for the public. That metric is critical because AI usage continues to scale up, despite public sentiment turning negative on AI. In the past 18 months, we've gone from roughly 15% of the global population regularly using AI to 19%. And in places like Singapore and UAE, two-thirds of the population are already using it regularly.
All of that usage is putting tremendous strain on today's data centers. Without more data center capacity, the industry faces the risk of AI queries taking much longer to run, the most powerful models being limited to those who can afford to pay a premium, and slowing down some of the most promising uses of AI for science and healthcare. That's why so many of the world's leading cloud and data center companies are building out future capacity, despite limitations on power and component shortages for RAM and data storage.
"The demand is huge," said Alajbegovic. "Right now the whole industry … is running to build them."
Nvidia estimated that there are well over 100 projects underway globally.
Our Deeper View
Of course, the only thing less popular than AI among the general public right now might be data centers. However, as Nat Rubio-Licht has written, the backlash against data centers might also just be the public's easiest way to register its larger concerns about AI. It doesn't help that there are false narratives proliferating about AI using up community water supplies, for example. But the leaders of the AI industry also need to do a much better job of explaining how AI will benefit humanity, without resorting to unbelievable hyperboles about AI curing all diseases or leading to unlimited abundance and prosperity. People just aren't buying that. The work that Nvidia and Siemens are doing to build data centers faster is also making them enormously more efficient. However, with only so much power, water, and construction capacity, those efficiency gains are out of pure necessity. Still, it's also a great story about building responsibly and not wasting resources, while also making sure there's enough compute to power the AI of the future and keep it broadly accessible and affordable.