The throughput claim and the transformer bet
Etched claims one 8-chip Sohu server delivers 500,000 tokens per second on Llama 70B. This throughput equals 62,500 tokens per second per chip. I find this 20x advantage over an 8-GPU H100 system, which produces 23,000 tokens per second, impossible to ignore. NVIDIA’s Blackwell architecture delivers 43,000 to 45,000 tokens per second on the same model. Because Sohu hard-codes transformer attention into fixed-function silicon, it reaches these levels. The chip avoids general programmability. This design removes overhead from kernel launch latency, memory allocation, and scheduler decisions. You know the scale of these models. A single H100 SXM5 has 80 GB of HBM3 with 3.35 TB/s of bandwidth. At batch 256, a single H100 can generate 45,000 tokens per second. The throughput gap widens when comparing memory bandwidth limits. While an H100 is limited by its 3.35 TB/s bandwidth, Sohu provides higher capacity via its 144GB of HBM3E. Etched’s 500,000 tokens per second figure for Llama 70B comes from their own published materials and appears to be measured at or near batch 1. This chip focuses only on inference and lacks any backward pass implementation for training.
Specifications and the TSMC production ramp
| Specification | Etched Sohu Value |
|---|---|
| Process Node | TSMC N4P (4nm) |
| Memory Capacity | 144 GB HBM3E |
| Memory Bandwidth | ~4,800 GB/s |
| Llama 70B Throughput | 500,000 tokens/sec (8-chip) |
| Peak FLOP Utilization | 80%+ for sparse MoEs |
Etched uses TSMC’s 4nm process to manufacture the Sohu chip. The hardware pairs the compute die with 144GB of HBM3E. This provides 4,800 GB/s of memory bandwidth, which is 1.8x the bandwidth of an H100. The company uses a Low Voltage Inference architecture to run math blocks at under half the voltage of conventional AI chips, which increases FLOP density because it avoids the thermal throttling that causes conventional chips to downregulate their clock speeds. The design also includes Cluster Scale Memory, which uses a proprietary interconnect to create a shared, low-latency memory pool across the scale-up domain. This interconnect addresses the memory access bottlenecks found in standard HBM-based systems. Unlike Groq, which uses 500 MB of on-chip SRAM to achieve 150 TB/s bandwidth, Sohu uses HBM3E to maintain higher memory capacity. Etched has opened a Taiwan factory and a data center to support the production ramp. Their team of 400 engineers includes experts from NVIDIA, Google TPUs, Broadcom, SK Hynix, and TSMC. They co-design the entire system from the PCB and cold plates to the interconnects. Notable investors like Jane Street, which invested more than $100 million, and VentureTech Alliance provide significant backing.
The MoE incompatibility problem
The fixed-function design creates a massive bottleneck for modern workloads. Sohu cannot run Mixture-of-Experts (MoE) models with dynamic expert routing, vision models, or diffusion models. DeepSeek V4 and Qwen3-235B-A22B both utilize MoE architectures, which makes them incompatible with the Sohu chip. These models require irregular memory access patterns that the fixed-function transformer circuits cannot accommodate. Models like LLaVA and Stable Diffusion fail on this hardware. I see this as the single biggest failure of the Etched strategy. If the industry continues to move toward MoE, Sohu becomes expensive fiberglass. The company raised $800 million across four rounds, including a $500 million round at a $5 billion valuation, to support this specialization. They also secured over $1 billion in signed customer contracts. Notable backers include Peter Thiel, Ribbit Capital, and Stanley Druckenmiller. First racks should ship this summer. Will the current transformer dominance last long enough to justify the lack of programmability?
