SoftBank acquired Graphcore, turning the UK chipmaker into a wholly owned subsidiary. While reports suggested a deal worth $500 million to $600 million, Nigel Toon, Graphcore CEO, stated that the $500 million figure lacks accuracy. Other reports placed the deal at $400 million. This acquisition follows a period of financial decline where Graphcore reported a 46 percent revenue drop in 2022, falling to just $2.7 million. The company previously raised more than $700 million in total funding and reached a valuation of $2.77 billion in 2020. I view the entire resurgence of Graphcore through SoftBank as a fragile attempt to salvage a struggling architecture. The company also faced significant job losses and closed operations in Norway, Japan, and South Korea. Despite these setbacks, SoftBank remains interested in the AI chip market. Vikas J. Parekh, Managing Partner at SoftBank Investment Advisers, noted that next generation semiconductors remain essential for the AGI journey.
Inference and MoE throughput
SambaNova’s SN40L targets large-scale AI inference by utilizing a three-tier memory hierarchy. This system combines High Bandwidth Memory, DDR DRAM, and distributed on-die Static Random-Access Memory. For sparse Mixture of Experts (MoE) workloads, SambaNova’s 16 RDU configuration delivers between 198 and 255 tokens per second on the DeepSeek-R1 671B model. You already know that Nvidia holds the market, so focus on how these specialized architectures handle sparsity. The SN40L chip, manufactured by TSMC, supports models with up to 5 trillion parameters. SambaNova claims its architecture allows for much higher operational intensity for activations through on-chip pipeline parallelism.
Graphcore’s Intelligence Processing Unit also targets complex data structures. The architecture supports group separable convolutions, which legacy processors often struggle to process efficiently. In inference tests, the IPU-M2000 achieved 40x higher throughput and 10x lower latency for ResNeXt-101 compared to the Nvidia A100. For EfficientNet-B0 inference, the IPU-M2000 delivered 60x higher throughput at its lowest latency. For BERT-Large inference, the IPU-M2000 provides 3.4x higher throughput at the lowest latency, while for LSTM 2-layer inference, it achieved 600x higher throughput at the lowest latency.
| Specification | Graphcore IPU-M2000 | SambaNova SN40L |
|---|---|---|
| EfficientNet-B0 Inference | 60x higher throughput | N/A |
| ResNeXt-101 Inference | 40x higher throughput | N/A |
| DeepSeek-R1 671B Throughput | N/A | 198 – 255 tokens/s |
| Power per Rack | N/A | 8kW to 15kW |
| Max Model Parameters | N/A | 5 trillion |
Scaling and survival
SambaNova struggles to maintain its momentum. The company explored a sale after failing to complete a new fundraising round, and its valuation fell significantly from its 2021 peak of $5.1 billion. I find the lack of recent funding for SambaNova as telling as the revenue drop seen at Graphcore. SambaNova’s RDU architecture, internally called Cardinal, uses 7nm technology to manage dataflow. The SN40L provides a power-efficient alternative with an average consumption of 10kW per rack.
Graphcore maintains its Bristol headquarters and offices in Cambridge, London, Gdansk, and Hsinchu. The company’s Colossus MK2 GC200 processor contains 1,472 independent IPU cores capable of managing 8,832 parallel threads, providing an aggregated bandwidth of 47.5 TB/s per chip. The IPU-M2000, built around four Colossus GC200 processors, provides one PetaFlop of AI compute.
The SambaNova SN40L handles models twice the size of advanced ChatGPT versions by using a tightly coupled three-tier memory system comprising HBM, DDR DRAM, and distributed on-die SRAM to overcome the massive memory wall. While SambaNova emphasizes efficiency, the company faces stiff competition from Groq, which uses a Language Processing Unit. Groq’s LPU relies on a programmable assembly line architecture but suffers from memory constraints due to an over-reliance on expensive SRAM. For Llama 3.3 70B, Groq requires 576 chips across nine racks to achieve 350 tokens per second. SambaNova’s 16-chip configuration for a 70B model offers a much smaller footprint. Will the infusion of SoftBank capital allow Graphcore to finally overcome its revenue instability?
