Hardware & Compute
Chips, data centres and the infrastructure behind AI.
-
VRAM Waste and Routing Failures in MI300X MoE Fine-Tuning
Engineers provisioning AMD Instinct MI300X clusters often face memory failures by using inference-only sizing for MoE training. Proper provisioning for a 235B…
-
H200 memory bandwidth inefficiencies in distributed training
Improper NUMA settings and hardware selection can waste the 4.8 TB/s memory bandwidth of the NVIDIA H200. Communication errors with the Power…
-
The cost of scaling AI interconnects
Marvell’s $3.25 billion acquisition of Celestial AI targets the 16 Tbps Photonic Fabric platform to scale XPU connections. This optical interconnect offers…
-
Graphcore acquisition by SoftBank and AI chip inference performance
SoftBank acquired Graphcore amid reports of a $500 million deal following a 46 percent revenue drop. While SambaNova’s SN40L delivers up to…
-
Scaling Llama 3.3 70B on Lambda Labs 8x H100 instances
Training Llama 3.3 70B requires significant VRAM, with full fine-tuning demanding 1,158 GB. Lambda Labs provides 8x H100 SXM nodes with 3200…
-
NVIDIA Blackwell Ultra supply chain shifts toward memory and substrate constraints
NVIDIA’s Blackwell supply chain faces new bottlenecks in organic substrates from Ajinomoto and Ibiden following TSMC yield recoveries. While TSMC reports stable…
-
The Stargate $500 billion AI infrastructure buildout
Sam Altman, Larry Ellison, and Masayoshi Son announced the Stargate joint venture to invest $500 billion in US AI infrastructure. The project…
-
Cerebras CS-3 wafer-scale engine and sparse MoE training explained
The Cerebras WSE-3 integrates 4 trillion transistors and 900,000 AI cores onto a single 300mm silicon wafer. This monolithic design delivers 125…
-
Why the SambaNova SN40L RDU beats the NVIDIA H200 on inference economics
SambaNova’s SN40L RDU delivers 9x greater performance and 5.6x more energy efficiency than the NVIDIA H200 for Llama 3.3 70B at low…
-
Scaling inference with Positron and Intel hardware
Positron AI’s Atlas hardware delivers 70% faster inference and 66% lower power consumption than H100 setups. The upcoming Titan system utilizes Asimov…
-
Groq secures $6.9B valuation as LPU inference outpaces NVIDIA
Groq has reached a $6.9 billion valuation as its LPU architecture delivers 500 to 750 tokens per second on Llama 3 70B…
-
Lightmatter scales AI interconnects with Passage M1000
Lightmatter’s Passage M1000 photonic interposer overcomes electrical I/O bottlenecks in AI clusters by delivering 114 Tbps of aggregated bandwidth. This 3D photonic…