OpenAI signed a $10 billion check to Cerebras. This deal secures 750 megawatts of compute through 2028. OpenAI uses NVIDIA for training, AMD for diversity, and Broadcom for custom silicon. They use Cerebras for speed. This investment helps OpenAI build independence. The focus shifts from training to inference. Models like O3 and GPT-5 require rapid response times. Standard GPUs act like a Ferrari in heavy traffic when running these heavy models. Cerebras solves this by keeping data on the chip.
Andromeda and the Wafer Architecture
Andromeda uses 16 Cerebras CS-2 systems. It uses 18,176 3rd Gen AMD EPYC processors. It has 13.5 million AI cores. This is more cores than 1,953 Nvidia A100 GPUs. It has 1.6 times more cores than the Frontier supercomputer, which has 8.7 million cores. Andromeda delivers 120 Petaflops of dense compute at 16-bit half precision. It delivers more than 1 Exaflop of AI compute. The WSE-2 processor provides 22PB/s of fabric bandwidth. It has 40GB of on-chip SRAM. The WSE-1 used 14nm technology, 400,000 cores, 18GB SRAM, and 9PB/s bandwidth. The WSE-2 uses 7nm technology, 850,000 cores, 40GB SRAM, and 20PB/s bandwidth. The WSE-3 uses 5nm technology, 900,000 cores, 44GB SRAM, and 21PB/s bandwidth.
| Component | WSE-1 | WSE-2 | WSE-3 |
|---|---|---|---|
| Process Node | 14nm | 7nm | 5nm |
| AI Cores | 400,000 | 850,000 | 900,000 |
| On-chip SRAM | 18 GB | 40 GB | 44 GB |
| Memory Bandwidth | 9 PB/s | 20 PB/s | 21 PB/s |
| Transistors | 1.2 Trillion | 2.6 Trillion | 4 Trillion |
Scaling without the Pain
Argonne National Laboratory used GPT3-XL to put the entire COVID-19 genome into the sequence window, and Andromeda ran their unique genetic workload with long sequence lengths across 16 CS-2 nodes while delivering 15.87X throughput compared to a single CS-2 system. A user achieved near perfect scaling on GPT-J with 25 billion parameters and a sequence length of 10,240. They could not achieve this on the Polaris cluster, which uses 2,000 Nvidia A100 GPUs. This capability allows for easier model distribution. Andromeda delivers near-perfect scaling across the largest language models. It works through simple data parallelism.
The PLMR model describes the hardware needs for these systems. Massive Parallelism requires partitioning LLMs across millions of cores. Highly Non-uniform Memory Access Latency means requests to distant cores take 1,000 times longer than local access. Constrained per-core Local Memory means data must fit into small fragments of tens of KBs or MBs. Constrained Routing Resources means the network on chip can only support a limited number of routing paths.
The Inference Bottleneck
Inference has two phases: prefill and decode. Prefill processes input tokens and spends most cycles on General Matrix Multiply (GEMM). Decode generates tokens one by one and primarily performs General Matrix-Vector Product (GEMV). LLM inference is memory-bandwidth-bound. WaferLLM achieves 10-20x speedups over A100 GPU clusters running SGLang and vLLM. It achieves 30-40x speedup compared to a single A100 GPU. WaferLLM achieves 606x faster GEMV operations than a single A100 GPU. The shift-based KV cache method in WaferLLM provides 360-385x more token capacity than GPU-based methods.
| Metric | Single A100 GPU | WaferLLM (WSE-2) |
|---|---|---|
| GEMV Speedup | 1x | 606x |
| End-to-End Speedup | 1x | 10-20x (vs A100 cluster) |
| Energy Efficiency | 1x | 2.5x |
| Token Capacity | Standard | 360-385x increase |
The Networking Problem
OpenAI and Microsoft developed Multipath RC (MRC) to prevent congestion. MRC distributes traffic across hundreds of network paths. It uses packet spraying and adaptive load balancing. This avoids the straggler effect in large clusters. MRC uses 800Gb/s network interfaces. It uses 8x100Gb/s ports. The design increases node reachability to 256 nodes in one hop. This is an improvement over the 32 nodes seen in traditional designs. MRC uses SRv6-based source routing to avoid conflicts between adaptive mechanisms.
As clusters scale to 100,000 GPUs, network noise creates problems. A single slow transfer can stall the entire synchronous pretraining job. MRC implements the protocol in 400 and 800Gb/s RDMA NICs. These include NVIDIA ConnectX-8, AMD Pollara and Vulcano, and Broadcom Thor Ultra. This architecture allows the system to bypass failed links.
Software and Reliability Gaps
The software ecosystem for Cerebras is thin. As of May 2026, Cerebras supports four models. Llama-3.1 8B and Llama-3.3 70B were scheduled for deprecation on May 27, 2026. Most open-source frameworks like vLLM do not run on Cerebras hardware. Compilation for custom operators takes over 3 hours. Every wafer has a unique pattern of 70,000 disabled cores. This makes binaries non-portable across a fleet. If a voltage regulator module fails, the system creates a silent-data-corruption risk. The system cannot dynamically recompile around a localized power brown-out during operation. I find this lack of resilience unacceptable for enterprise scale.
The Graph Compiler calculates all core placement and message routing at compile time. A binary compiled for one unit cannot run on another unit because the functional core maps do not line up. The pool of CSL experts is non-existent compared to the thousands of CUDA developers. Additionally, Cerebras operates its multi-tenant concurrency model as a black box. This hides capacity limits and queue behaviors from enterprise buyers.
Economics and Supply Chain Risk
86 percent of 2025 revenue came from two Abu Dhabi entities. MBZUAI provided 62 percent and G42 provided 24 percent. The 2025 net income of $237.8 million came from a $363.3 million accounting gain from the G42 contract. Without this, the core business had a $75.7 million operating loss. Fulfilling the OpenAI order requires 32,600 systems by 2028. This is a 170-fold increase from the 192 systems shipped through the end of 2024.
Cerebras must compete for limited cleanroom allocations at TSMC. The company possesses no committed long-term wafer allocation from TSMC. They must compete against Apple, AMD, Qualcomm, and NVIDIA. The critical thermal-expansion connector is manufactured under a single-source agreement. This leaves the scale-up exposed to supply chain dependencies.
The Future of Inference Hardware
The CS-4 platform uses the Nexus architecture. It delivers twice the power to the wafer. It delivers twice the interconnect bandwidth. It provides half the latency of the previous generation. GPT-OSS runs at over 4,400 tokens per second on CS4. Projections for CS5 suggest 10,000 tokens per second. You already know that NVIDIA dominates training. This deal shows OpenAI seeks independence. If HBM improves enough in three years to solve the decode bottleneck, Cerebras remains a transitional technology. Will the software ecosystem grow fast enough to support this hardware?
