Graphcore’s Colossus GC200 IPU contains 59.4 billion transistors per die, which exceeds the 54 billion transistors found in the Nvidia A100 GPU. The GC200 uses a 7-nanometer manufacturing process from TSMC and includes 1,472 cores that support 8,832 parallel computing threads. These cores use 900 megabytes of onboard memory spread across the die. Graphcore CEO Nigel Toon claims the IPU achieves its strongest performance over GPU-based systems when processing extremely complex AI models like ResNeXt, EfficientNet, and sparse natural language processing.
The IPU-Machine M2000 server provides a four-socket, single-rack unit. Each M2000 server delivers a petaFLOP of AI compute performance and 450 gigabytes of memory with 180 Tb/s of bandwidth. Customers can connect up to 64,000 IPUs using the IPU-Fabric, a network system using standard quad small form factor pluggable transceivers. This fabric allows communications performance to scale linearly to several petabits per second in IPU-Pod systems. While the M2000 serves as a competitor to the A100, Nvidia still maintains a lead in hardware and software. The M2000 was available for pre-order at $32,450 and began shipping to some customers for evaluation purposes.
| Component | Graphcore Colossus GC200 | Nvidia A100 |
|---|---|---|
| Transistor Count | 59.4 billion | 54 billion |
| Onboard Memory | 900 MB | Not specified |
| Manufacturing Process | TSTS 7-nanometer | Not specified |
| System Performance | 1 petaFLOP per server | Not specified |
Intel Gaudi 3 deployment and training metrics
Intel submitted results for the Gaudi 2 accelerator chip to MLPerf benchmarks, showing that enabling 8-bit floating-point (FP8) capabilities provided a 103 percent reduction in time-to-train for a 384-accelerator cluster. This performance improvement put the Gaudi 2 system at a little less than one-third the speed of an Nvidia system on a per-chip basis. The Gaudi 2 system also ran three times as fast as Google’s TPUv5e. In image-generation benchmarks, the Gaudi 2 ran at about half the speed of the H100. Intel expects the performance advantage to grow with the Gaudi 3 chip.
The Gaudi 3 deployment options include a PCIe card version and a rack-level reference system. The PCIe card version targets scalable AI inference within existing server infrastructure and supports models ranging from LLaMA 3.1 8B to LLaMA 4 Scout and Maverick. The rack-level reference system accommodates 64 accelerators per rack with 8.2TB of high-bandwidth memory. This architecture uses wired backplanes and blind-mate 2D cabling to simplify installation.
| Intel Product | Target Application | Key Memory/Scale Detail |
|---|---|---|
| Gaudi 3 PCIe Card | Scalable AI inference | Supports LLaMA 3.1 8B to LLaMA 4 |
| Gaudi 3 Rack System | Flexible, scalable architecture | 64 accelerators; 8.2TB memory |
| Arc Pro B60 | AI inference | 24GB memory; 456 GB/s bandwidth |
| Project Battlematrix | Medium-scale deployment | Up to 192GB VRAM; 8 GPUs |
Machine intelligence learning vs training
Modern machine learning systems often require massive amounts of labeled data to train classification systems for narrow domains. These systems typically do not continue to learn from new experiences. In contrast, next generation machine intelligence systems will use cognitive memory to understand context and reuse learned knowledge. Humans do not store complete pixel bit-maps of every scene, but instead build representations through foveal vision systems and re-compute scenes when a trigger causes a memory recall.
Hardware developers are designing processors to support this continuous learning rather than just training or inference. Machine intelligence in the core, such as in the cloud, needs to scale up and scale out to support many different users and tasks in parallel. Machine intelligence at the edge, such as in an autonomous car, requires local intelligence processing because a mobile internet connection to the cloud is not practical for real-time reactions. A floor cleaning robot might use the edge for local intelligence and connect to the cloud to learn about a new object, then update its model to avoid that object later.
The distinction between training and inference will disappear as systems evolve into learning systems that improve after deployment. For example, an autonomous car seeing a ball roll into the road could learn that a ball implies a child is likely to follow. This knowledge could be shared via a cloud-based learning system overnight to update the models for all cars by the next morning. Can hardware designers balance the high-performance requirements of the core with the power constraints of the edge? Intel’s Arc Pro B50 uses 70W of power to provide 170 TOPS (INT8), while the Arc Pro B60 uses between 120W and 200W to provide 197 TOPS (INT8).
