Electrical, thermal, and deployment planning for Cerebras CS-3 clusters

Cerebras CS-3 clusters require massive power density and custom water-cooling solutions to manage the intensive thermal needs of wafer-scale hardware. A single CS-3 system delivers significant AI compute, while the Scale Datacenter in Oklahoma City manages over 300…

Electrical, thermal, and deployment planning for Cerebras CS-3 clusters

Electrical infrastructure planning for Cerebras clusters requires accounting for massive power density. Training large models on legacy equipment requires tens of megawatts of power. Cerebras targets 600 megawatts of computing capacity by the end of 2027. The Scale Datacenter in Oklahoma City, which houses over 300 Cerebras CS-3 systems, utilizes triple redundant power stations and custom water-cooling solutions to manage the intensive requirements of large scale wafer-scale hardware deployments. A single CS-2 system consumes approximately 15kW. A cluster of 2048 CS-3 systems delivers 256 exaflops of AI compute. The Condor Galaxy 3 supercomputer in Dallas uses 64 CS-3 systems to provide 8 exaflops of performance. The CS-3 supports clusters of 2048 systems, which is a 10x improvement over the 192 systems supported by the CS-2. The CS-3 provides memory options including 1.5TB, 12TB, 24TB, 36TB, 120TB, and 1,200TB configurations. The CS-3 can train Llama2-70B from scratch in less than a day. Training Llama2-70B on a GPU cluster took approximately a month, but a CS-3 cluster completes the task in less than a day. Cerebras also signed a multi-year agreement with OpenAI covering 750 MW of capacity.

Thermal Management and Hardware Requirements

Thermal management presents specific difficulties for wafer-scale integration. Scale Datacenter in Oklahoma City uses custom water-cooling solutions to power large scale deployments of Cerebras Wafer Scale systems. This cooling handles the unique needs of the hardware. Research indicates that thermal management and reliability present difficulties for wafer-scale technologies. The WSE-3 chip measures 46,225 mm2, making it over 50 times larger than a high-end Nvidia H100 GPU. The WSE-3 integrates 4 trillion transistors and 900,000 AI-optimized cores. The WSE-3 contains 44 GB of on-chip SRAM. Each CS-3 core has 8-wide FP16 SIMD units, which is a 2x increase over the CS-2. This configuration provides 21 petabytes per second of memory bandwidth. The WSE-3 uses the TSMC 5nm process and provides 214 petabits per second of network fabric bandwidth. The WSE-2 used 7nm technology and had 2.6 trillion transistors and 850,000 cores. The WSE-1 used 14nm technology and had 1.2 trillion transistors and 400,000 cores. The WSE-2 achieved 20 petabytes per second of memory bandwidth, while the WSE-3 achieves 21 petabytes per second.

Physical Deployment and Complexity

Physical deployment complexity affects how quickly facilities bring AI computing systems online. The CS-4 uses 50% fewer components than its predecessor to reduce the complexity involved in bringing AI computing systems online and to simplify the installation process for data center operators. This redesign simplifies installation. The CS-4 uses the Nexus server architecture and integrates three WSE-3 Turbo processors in a rack-scale system. The system uses upgraded networking technology to improve communication between components. Independent reports suggest the CS-4 delivers up to 30 times the speed of comparable GPU-based solutions for certain workloads. The CS-4 focuses on the AI inference market to accelerate chatbot queries and generative AI applications. The CS-4 is designed for the rapidly expanding AI inference market. Can facilities scale to these densities without upgrading existing cooling infrastructures?

airtrain.ai
airtrain.ai

The airtrain.ai newsroom covers AI research, models and the tools built on them.

More on this topic

Stay ahead of AI

Get the week's most important AI stories delivered to your inbox every Monday.

No spam. Unsubscribe anytime.

More Stories