Stacking silicon for speed
Graphcore uses wafer-on-wafer technology to bind two semiconductor die together through hybrid bonding. This process joins the metal circuitry of each wafer and creates extremely high density of interconnects between the two wafers. Graphcore developed this method in close collaboration with Taiwan Semiconductor Manufacturing. The top wafer undergoes shaving to a thin, film-like quality that Simon Knowles describes as "floppy." This top wafer sits on the IPU logic and memory circuits to provide power through deep trench capacitor cells. These cells act as charge reservoirs to speed up the logic transistors. This technique increases the clock speed of transistors from 1.325 gigahertz to 1.85 gigahertz, resulting in a 40% speed-up. The Bow chip delivers 350 trillion floating point per second of mixed-precision AI arithmetic, making it the highest-performing AI processor in the world today according to Knowles. The chip moves data at 65 trillion bytes per second and utilizes 900 megabytes of on-chip SRAM primary cache. A special "via" tunnel connects the stacked die to the rest of the system. While the Cerebras WSE-2 delivers between 1,700 and 3,400 teraflops for 32-bit or 16-bit floating point arithmetic, Graphcore maintains that the Cerebras device consists of an entire 12-inch wafer rather than a single chip.
Cost efficiency in the BOW POD
The BOW POD-16 delivers a five times speed up to train the EfficientNet neural network versus a comparable DGX-A100 server from Nvidia, while the BOW POD-16 costs only $149,995 compared to the $299,000 price of the DGX. You already know that NVIDIA owns the data center, so look at the raw numbers instead. Graphcore assembles the Bow into IPU-POD machines that scale from 16 to 1,024 Bow chips to provide as much as 358.4 petaflops of compute alongside up to 64 companion CPU processors.
| Specification | Graphcore Bow IPU |
|---|---|
| AI Arithmetic | 350 trillion floating point per second |
| On-chip SRAM | 900 megabytes |
| Data Movement | 65 trillion bytes per second |
| Clock Speed | 1.85 gigahertz |
Pacific Northwest National Labs uses these machines to run Transformer-based neural networks and graph neural networks for tasks like computational chemistry and cyber-security. Other partners like Kingsoft Cloud and Cirrascale provide the Pods as cloud services. The hardware provides chip-to-chip link speeds of 320 gigabytes per second. Earlier models like the IPU-M2000, which contains four Colossus GC200 IPU processors, provided one petaflop of AI compute. The IPU-POD64 provides the ability to run large models across as many as 64 IPU processors in parallel or to share the compute resource across multiple users and tasks.
The 500 trillion parameter goal
Graphcore works on a computer design called The Good Computer to handle neural network models that use 500 trillion parameters. This design seeks to realize the vision of 1960s scientist Jack Good, who described a computer exceeding the information capacity of a human brain. This machine will combine multiple wafers of logic circuitry to achieve 10 exaflops of compute and 10 billion billion floating point operations per second. The system requires four petabytes of memory and costs approximately $120 million, which Knowles says is less than many of today’s supercomputers. SoftBank now owns Graphcore as a wholly owned subsidiary following an acquisition. However, the company previously faced material uncertainty regarding its ability to remain a going concern and laid off 21 percent of its workers. Graphcore also closed operations in Norway, Japan, and South Korea. It reported pre-tax losses of $204.6 million, while revenue dropped 46 percent to $2.7 million. Graphcore previously raised over $710 million in total funding, including a $222 million Series E round that set a post-money valuation of $2.77 billion. This round, led by the Ontario Teachers’ Pension Plan Board, included funds managed by Fidelity International and Schroders, along with existing investors like Baillie Gifford and Draper Esprit. Other investors include Samsung Catalyst Fund, Robert Bosch Venture Capital, Amadeus Capital Partners, and Pitango. The $222 million investment supports the company’s global expansion and future IPU silicon, systems, and software development. Will the industry shift from GPUs to these stacked IPUs fast enough to prevent NVIDIA from widening its lead?
