Nvidia creates open server spec to deal with its personal {hardware}, and the occasional x86

Computex Nvidia has created an open design for servers to deal with its accelerators, arguing that CPU-centric server designs aren’t up the job of housing a number of GPUs and SmartNICs .

Nvidia CEO Jensen Huang introduced the design, known as “MGX”, on the Computex 2023 convention in Taiwan on Monday, and mentioned it’s needed as a result of present server designs weren’t created to deal with the warmth produced, and energy consumed, by Nvidia’s accelerators.

The chassis design was introduced in a keynote that noticed Huang certain onto stage for his first public speech in 4 years, main him to ask the viewers to want him luck.

The gang was prepared to do this, and way more: punters actually ran to the entrance of the room to be nearer to Huang. Carrying his trademark black leather-based jacket, the CEO delivered a two-hour speech peppered with jokes in Chinese language and even an expression of appreciation for a infamous Taiwanese snack known as “stinking tofu”. The gang ate it up. A Taiwanese software program challenge supervisor who cheekily occupied a seat reserved for the press so she might see Huang extra clearly informed us she attended as a result of the Nvidia boss was positive to supply unparalleled insights on AI was rapt all through.

Huang’s pitch was that an period of computing historical past that began with the 1965 debut of IBM’s System 360 has come to an finish. In Huang’s telling the System 360 gave the world the primacy of the CPU and the flexibility to scale methods.

That structure has dominated the world since, he opined, however CPU efficiency enchancment has plateaued, and accelerator-assisted computing is the longer term.

That argument is, in fact, Nvidia’s core tenet.

However Huang backed it up with information in regards to the effort required to provide a big language mannequin (LLM) citing a hypothetical 960-server system that price $10 million and consumed 11Gwh to coach one LLM.

The CEO asserted only a pair of Nvidia-powered servers costing $400,000 and packing GPUs can do the identical job whereas consuming simply 0.13 GWh. He additionally prompt a $34m Nvidia-powered rig of 172 might produce 150 LLMs whereas consuming 11GWh.

Forthcoming servers primarily based on Nvidia’s MGX spec- Click on to enlarge

Huang’s concept is that this type of rig will quickly be on many group’s procuring lists as a result of whereas datacenters are being constructed at a livid fee, competitors for area on racks and electrical energy will stay fierce and lots of customers will due to this fact look to rearchitect their datacentres for better effectivity and density.

Which is the place the MGX spec is available in, by providing a design that may use extra of Nvidia’s wares in a smaller footprint than can be the case in the event that they have been shoehorned right into a machine powered by a CPU alone. Huang prompt MGX delivers the density customers require. And it might fortunately home boring outdated x86 CPUs if required, alongside all these beautiful Nvidia accelerators.

ASRock Rack, ASUS, GIGABYTE, Pegatron, QCT and Supermicro have all signed as much as produce the servers, which Huang mentioned might be deployed in over 100 configurations and in 1U, 2U, and 4U kind components.

In August QCT will ship an MGX design named the S74G-2U that can supply the GH200 Grace Hopper Superchip . In the identical month Supermicro’s ARS-221GL-NR will embody the Grace CPU Superchip.

Huang additionally introduced that Grace Hopper manufacturing is now in full swing. The CEO prompt it might discover a dwelling working 5G networks, and a few generative AI on the similar time, to groom video chats as they go via the community as a substitute of leaving consumer gadgets to do all of the work of compressing and decompressing video.

He additionally outlined an enormous change, the Spectrum-X Networking Platform, to enhance the effectivity of ethernet in large-scale clouds working AI workloads.

And he made imprecise references to Nvidia bettering assist for its AI software program stack so it’s extra appropriate for enterprise use, with Crimson Hat supplied because the mannequin Nvidia intends to emulate.

Robotics have been additionally on the CEO’s thoughts, as Nvidia has opened a platform known as Isaac that mixes software program and silicon to allow the creation of autonomous bots for industrial use, particularly in warehouses the place Huang mentioned they’ll roll round to maneuver items.

These bots shall be ready to take action after being created on digital twins of real-world areas developed utilizing servers packed stuffed with Nvidia acceleration {hardware}.

Which once more illustrated why Nvidia has created MGX. ®