Air cooling fails at these densities
The GB200 NVL72 draws 120kW per rack, a figure that crushes the 10 to 15 kW air-cooling ceiling of well-engineered data halls. I see facility teams at Equinix struggle because they underestimate the thermal physics of Blackwell. Because the GB200 GPU die heat flux reaches 500 to 600 W/cm2, standard air-cooled heatsinks that top out at 15 W/cm2 cannot prevent the system from hitting the maximum junction temperature threshold during continuous operation. This heat flux exceeds the capacity of air cooling by a factor of 40. The GB200 NVL72 rack pulls 120kW continuously through four 30kW power shelves, requiring 480V three-phase input to manage the massive electrical load without causing immediate instability during the most intensive training sessions.
The architecture achieves 97% conversion efficiency, yet it generates 3.6kW of waste heat from power conversion alone. The rack requires 2.4 megawatts of cooling capacity. Equinix expands capacity through xScale projects, but its DA12 facility only achieves 18 kVA per cabinet density. The GB200 NVL72 connects 36 Grace-Blackwell Superchips. Each Superchip combines an Arm-based Grace CPU with two Blackwell GPUs via NVLink-C2C at 900GB/s bidirectional bandwidth. The system delivers 1.4 exaflops of AI compute and provides 13.5 terabytes of HBM3e memory with 576 terabytes per second of aggregate bandwidth.
Reinforcing floors for 3,000kg racks
A single GB200 NVL72 rack weighs 3,000kg. This concentrates 1,500kg in just 0.8 square meters, creating point loads of 1,875 kg/m2. Standard raised floors only handle 1,000 kg/m2. I would avoid these deployments in older facilities without pouring reinforced concrete pads or using steel reinforcement plates. Standard data center doors cannot accommodate the width of the rack, which requires removing door frames or walls.
Installation requires military precision. The system arrives in four components: a 1,500kg compute rack, an 800kg NVLink Switch rack, a 400kg CDU, and a 300kg power distribution unit. Teams use specialized hydraulic lifts rated for 2,000kg to move these parts. Deployment teams also spend up to 80 hours on cable management. They must manage 5,000 individual connections, including 144 copper NVLink cables, 288 optical cables, and 72 liquid cooling tubes. They use augmented reality headsets to ensure every cable matches the exact routing diagrams.
| Component | Specification |
|---|---|
| Rack Power Draw | 120kW |
| Rack Weight | 3,000kg |
| Floor Point Load | 1,875 kg/m2 |
| GPU Heat Flux | 500-600 W/cm2 |
| Required Flow Rate | 2-3 L/min |
| Max Inlet Temperature | 45°C |
The system provides 13.5 terabytes of HBM3e memory. This memory is accessible at 576 terabytes per second.
Liquid cooling requirements and operational risks
Managing 72 GPUs as a single unit creates massive telemetry volumes. Because each GPU produces 10,000 metrics per second, a single rack generates 1 million metrics per second. Traditional monitoring systems cannot handle this volume. Hardware failures happen constantly. During the training of Llama 3, Meta experienced 8.6 hardware failures every day. One failed GPU can destabilize the entire system due to NVLink topology dependencies. This failure reduces performance by 15 to 20% per failed GPU. A single GB300 NVL72 rack costs $25,000 per day in lost productivity if downtime occurs.
Operators design around failure by keeping aside 10% of pools to failover safely. This allows training to continue. However, a faulty network showing up on one node can indicate a bad switch that takes out the whole rack. Thermal throttling on one node can indicate a failed cooling tray, while power throttling can indicate a faulty supply instead of a bad node.
Liquid cooling loops require extreme cleanliness. A single contaminant particle can clog microchannel cold plates. Installation teams must flush the system three times with deionized water before introducing coolant. This process takes 12 to 16 hours. Pressure drops must not exceed 1.5 bar. Does Equinix have the specialized personnel to manage these pharmaceutical-grade cleanliness requirements?
