The Tesla P100 architecture in 2016 used NVLink 1.0 to bypass PCIe 3.0 x16 bandwidth limitations. This first generation provided 160 GB/s of bidirectional aggregate bandwidth through four links. Before NVLink, GPUs communicated through PCIe switches, which required CPU intervention for data distribution and scheduling. This added latency and limited system performance as GPU chip performance grew. The P100 used four NVLink connections between GPUs, with each link consisting of eight channels.
The Volta transition to NVSwitch
The 2017 Volta architecture introduced NVLink 2.0, which increased bidirectional bandwidth to 300 GB/s per V100 GPU through six links. Each link in this generation contained eight channels operating at 25 GB/s. NVIDIA reached the scaling limit of direct peer-to-peer connections in 2018 with the DGX-2, which housed 16 V100 GPUs. The DGX-2 system contains two baseboards, each holding six first-generation NVSwitch ASICs, which allow 16 V100 GPUs to exchange data at full NVLink speed across the entire large-scale switch fabric. This NVSwitch technology changed the topology from a small mesh to a switched fabric where any GPU pair communicates at full speed.
| NVLink Generation | Bandwidth per GPU | Max Links per GPU | Supported Architecture |
|---|---|---|---|
| Fourth Generation | 900 GB/s | 18 | NVIDIA Hopper |
| Fifth Generation | 1,800 GB/s | 18 | NVIDIA Blackwell |
| Sixth Generation | 3,000 GB/s | 36 | NVIDIA Vera Rubin |
Ampere and the 8-GPU standard
NVIDIA standardized the 8-GPU building block with the Ampere A100 in 2020. NVLink 3.0 provides 600 GB/s of bidirectional bandwidth per GPU via 12 links. Second-generation NVSwitch chips in the DGX A100 connect eight GPUs into a full-mesh domain. The A100 also contains 80 GB of HBM2e memory. You likely know that this architecture became the standard for large language model training due to its high-speed interconnect. Multi-Instance GPU technology in Ampere also allows a single A100 to partition into seven isolated instances.
Hopper and the GH200 Superchip
The Hopper H100 introduced NVLink 4.0 in 2022, which uses PAM4 signaling to reach 900 GB/s of bidirectional bandwidth across 18 links. This generation provides 14 times the bandwidth of a PCIe 5.0 x16 slot. Third-generation NVSwitch chips include the SHARP in-network reduction engine to handle collective operations within the fabric. The GH200 Grace Hopper Superchip connects an Arm-based Grace CPU and a Hopper GPU using the NVLink-C2C interconnect. This link provides 900 GB/s of bidirectional bandwidth, which is 7x higher than PCIe Gen5. The Hopper GPU features 96 GB of HBM3 memory with 3000 GB/s bandwidth. The GH200 allows CPU and GPU threads to access both CPU and GPU memory.
Blackwell and the 72-GPU rack
Blackwell moves NVLink from a node-level interconnect to a rack-level fabric. The GB200 NVL72 rack configuration packages 36 Grace CPUs and 72 Blackwell GPUs into a single liquid-cooled rack with 130 TB/s of all-to-all GPU bandwidth to support the training of massive trillion-parameter models. NVLink 5.0 provides 1.8 TB/s per GPU across 18 links. This technology enables a 72-GPU NVLink domain within a single rack. The B200 GPU has 192 GB of memory.
| NVLink Switch Generation | GPU Domains | GPU-to-GPU Bandwidth | Total Aggregate Bandwidth | Supported Architecture |
|---|---|---|---|---|
| NVLink 4 Switch | 8 | 900 GB/s | 7.2 TB/s | NVIDIA Hopper |
| NVLink 5 Switch | 8 | 1,800 GB/s | 130 TB/s (NVL72) | NVIDIA Blackwell |
| NVLink 6 Switch | 72 | 3,000 GB/s | 216 TB/s (NVL72) | NVIDIA Vera Rubin |
Blackwell Ultra and the B300
Blackwell Ultra serves as the mid-cycle enhancement for data center AI in 2026. The B300 GPU delivers 288 GB of HBM3e memory, which is 50% more capacity than the B200. This capacity increase helps serve large language models that do not fit in B200 memory without tensor parallelism. The B300 provides 1.1 EXAFLOPS of FP4 compute per rack in the GB300 NVL72 configuration. This generation provides 35% higher training throughput than the B200 on GPT-4-class model benchmarks. The B300 is available on AWS via P6-B300 instances.
Vera Rubin and NVLink 6
The Vera Rubin platform introduces NVLink 6.0, which provides 3.6 TB/s of bidirectional bandwidth per GPU through 36 links. This generation increases rack-scale fabric bandwidth to 260 TB/s for the Vera Rubin NVL144. The NVLink 6.0 architecture doubles the per-GPU link count to 36 and moves to bidirectional SerDes, the same high-speed electrical signaling class that now appears across 1.6T Ethernet optics. This platform uses the Vera Rubin architecture for high-scale AI training and inference. Will the Vera Rubin NVL144 satisfy the demand for even larger training clusters?
Reliability and sovereign AI deployments
The reliability of these large-scale clusters is a significant concern. A study of the Delta system using 1,168 NVIDIA GPUs shows NVLink errors occur with a mean time between errors of 6.9 hours. These errors cause job failure in 66% of cases. The GSP error stalls GPU control functions and requires a node reboot, a process that takes 23 hours. I find the instability of these large-scale clusters concerning.
NVIDIA also supports sovereign AI infrastructure in India and Australia. Yotta Data Services and Larsen & Touhm in India are building gigawatt-scale NVIDIA powered data centers. Australia’s first Cisco Secure AI Factory runs on NVIDIA GPUs for local enterprise and government workloads. These projects use Blackwell Ultra GPUs for secure AI capacity.
| System Name | GPU Type | Target Application |
|---|---|---|
| Solstice | NVIDIA Blackwell | Scientific discovery |
| Equinox | NVIDIA Blackwell | Scientific discovery |
| Mission | NVIDIA Vera Rubin | Classified applications |
| Vision | NVIDIA Vera Rubin | Unclassified research |
| Lux AI | AMD Instinct MI355X | AI for science |
| Discovery | AMD Instinct MI430X | AI and scientific research |
