Cerebras generates 4,400 tokens per second on the GPT-OSS 120B model using a single CS-4 rack. This speed exceeds SambaNova’s 708 tokens per second on the same benchmark by a wide margin. The CS-4 system uses the WSE-3 Turbo chip, which contains 4 trillion transistors and 900,000 AI cores. Each WSE-3 Turbo chip provides 125 PFLOPS of compute and uses 44 GB of SRAM. I find the raw throughput of the Cerebras architecture superior for high-speed text generation. A single CS-3 system provides 125 peak AI petaflops. This system is scalable up to 2,048 CS-3s in a single cluster to reach 256 exaflops of AI compute. The CS-4 rack is a pluggable backpack design that attaches the compute subsystem vertically to the power array. This design moves power conversion closer to the wafers to reduce power losses. Cerebras’s cloud business grew 281% year over year, reaching $126 million in the second quarter. The WSE-3T provides 43.2 PB/s of bandwidth and 250 PFLOPS of compute with sparsity. Cerebras is also building a third Condor Galaxy machine featuring CS-3s as part of a $900 million deal with G42.
SambaNova density and memory
SambaNova achieves high performance for the Llama 3.1 70B model using 16 SN40L chips. This configuration produces 457 tokens per second. In contrast, Cerebras needs four wafers containing 336 chips to reach 445 tokens per second for the same model. You should look at the footprint if you care about rack space. SambaNova’s 16-chip setup is a 19-inch form factor with air cooling. The SN40L architecture uses a three-tier memory system to overcome the memory wall.
| Component | Cerebras CS-3 (WSE-3) | SambaNova SN40L |
|---|---|---|
| AI Cores | 900,000 | 1,040 (PCUs) |
| On-chip SRAM | 44 GB | 520 MiB |
| Memory Bandwidth | 43.2 PB/s | Hundreds of TBps |
| Peak Compute | 125 PFLOPS | 638 BF16 TFLOPS |
| Memory Capacity | 44 GB SRAM | 64 GiB HBM / 1.5 TiB DDR |
The SN40L RDU utilizes a three-tier memory system containing 520 MiB of on-chip PMU SRAM, 64 GiB of co-packaged HBM, and up to 1.5 TiB of DDR DRAM through pluggable DIMMs. This architecture combines streaming dataflow parallelism with high-capacity memory to solve the compute-to-memory ratio problem. The RDU architecture fuses hundreds of complex operations into a single kernel call without programmer intervention. This capability delivers speedups ranging from 2x to 13x on various benchmarks. The SN40L reduces machine footprint by up to 19x for CoE inference deployments. It also speeds up model switching time by 15x to 31x. The Samba-CoE system uses 150 expert models and a trillion total parameters. The SN50 provides 5x more compute per accelerator and 4x more network bandwidth. It supports clusters of up to 256 accelerators and models exceeding 10 trillion parameters. SambaNova and Intel plan to develop high-performance, cost-efficient AI inference solutions for enterprises and governments.
Training versus inference reality
Cerebras’s approach to inference relies on hosting all weights in SRAM. This method creates scalability challenges for larger models. To run Llama 3.1 70B, Cerebras requires 184,900 mm2 of silicon across four racks. SambaNova uses 16 chips to serve the Llama 3.1 405B model at 129 output tokens per second. The SN40L provides 64 GiB of HBM and up to 1.5 TiB of DDR DRAM. The SN50 provides 5x more compute per accelerator and 4x more network bandwidth than the SN40L. For the Llama 3.1 8B model, Cerebras delivers 1837 tokens per second, while SambaNova delivers 1042 tokens per second. I find the Cerebras deployment model inefficient for large-scale data centers because of the massive silicon area required for single models. The power draw for a single CS-3 is 23kW. The SN50 averages 10kW. Can SambaNova maintain this lead as Cerebras scales its CS-4 production? I recommend the SambaNova SN40L for organizations prioritizing data center density and multi-model flexibility.
