The IPU architecture and early growth
Simon Knowles and Nigel Toon founded Graphcore in 2016 after two years of development inside XMOS. Knowles, who holds an MA in Electrical Science from Cambridge and authored 14 patents, previously co-founded Element14 and Icera. Toon, who led Icera before its sale to Nvidia, previously managed the European business unit at Altera. The company secured $30 million in its first funding round, which Robert Bosch Venture Capital led. Other backers included Samsung and Amadeus Capital Partners. In July 2017, Graphcore secured a Series B round led by Atomico, which was followed by $50 million from Sequoia Capital.
The company aims to increase the speed of AI training by 10 to 100 times compared to current computer servers. Graphcore attracted interest from several technology leaders. Amadeus Capital and Samsung Catalyst Fund participated in early rounds. The company attracted investment from Dell Technologies Capital and Microsoft. Demis Hassabis, the founder of Google DeepMind, also invested in the firm. These high-profile backers helped push the company toward its unicorn status after a $200 million Series D round in 2018, which established a valuation of $1.7 billion. By December 2020, the company raised $222 million, which established a valuation of $2.77 billion. The company aims to address a market for deep learning hardware that Tractica predicted would grow from $436 million in 2015 to $41.5 billion by 2024.
Scaling against the giants
The Intelligence Processing Unit (IPU) architecture uses massively parallel resources to process machine learning workloads. Graphcore developed the Poplar software stack to manage memory through computation graphs rather than traditional cache hierarchies. This approach targets the irregular data access patterns found in high-dimensional AI models. The IPU architecture emphasizes low-precision floating-point compute and high compute density. By using the Poplar framework, developers interface with standard machine learning libraries like TensorFlow and MXNet. Poplar provides a C++ and Python interface for application development.
| Processor Generation | Process Node | Cores | Threads | Transistors | Performance (FP16) |
|---|---|---|---|---|---|
| Colossus MK1 (GC2) | 16 nm TSMC | 1216 | 7296 | – | ~100-125 teraFLOPS |
| Colossus MK2 (GC200) | 7 nm TSMC | 1472 | 8832 | 59 billion | ~250-280 teraFLOPS |
| Bow IPU | 3D package | – | – | – | – |
The Bow-2000 IPU Machine uses four Bow IPU processors to provide 1.4 petaflops of processing capacity. Customers combine these machines into Bow Pods containing between 16 and 1,025 processors. A 16-processor Bow Pod provides five times the performance of an Nvidia DGX A100 at half the cost. You already know that Nvidia dominates this space, yet Graphcore attempted to challenge that dominance with these specialized architectures. The Bow IPU utilizes a wafer-on-wafer design to manage high clock rates at lower core voltages. The GC200 chip contains 1,472 computational cores and 900 Mbyte of local memory within an 823 square-millimeter integrated circuit. The architecture uses MIMD (Multiple Instruction, Multiple Data) parallelism and distributed, local memory. The machine relies on scratchpad memory for performance instead of traditional cache hierarchies. The IPU uses IEEE FP16, with stochastic rounding, and also single-precision FP32, at lower performance. The company aims for a "Good machine" to enable AI models with more parameters than the human brain has synapses. This hardware aims to improve performance and efficiency by between 10x to 100x compared to current solutions.
The SoftBank salvage
Graphcore struggled to keep up with new developments in AI chip design because its capital reserves were insufficient. While the company aimed to reduce data center costs, its capital reserves failed to match the massive investment requirements of continuous AI chip development, leaving the startup vulnerable to the rapid release cycles of incumbents like Nvidia. By February 2024, reports indicated the company discussed selling itself to cover heavy losses. Chrysalis Investments, a stakeholder, revised the company’s valuation to $528 million after a prior write-down. This valuation fell from $2.77 billion in late 2020 to $528 million in early 2024. The company also failed to disclose a new funding round it expected to close in the third quarter of last year.
In July 2024, SoftBank Group agreed to acquire the company for approximately $500 million. The UK government reviews the deal because the investment security unit flagged concerns regarding AI technology. This acquisition follows the loss of Graphcore’s AI networking technology team to Meta Platforms in early 2023. The SoftBank deal salvages the Bristol-based engineering talent after the company faced a liquidity crisis. Potential buyers previously included firms such as Arm Holdings and SoftBank. Will the integration of IPU technology into SoftBank’s ecosystem change the trajectory of British semiconductor design?
