Groq secured $350 million in a funding round that sets its valuation at $3.5 billion. The Dallas firm Disruptive led the round, and Nvidia intends to participate. This follows a $650 million raise in June 2026, which brings Groq’s recent total funding to $1 billion. Founder and CEO Jonathan Ross, the inventor of Google’s TPU, manages the company as it scales data center capacity to 200 megawatts by 2027. I find the performance gap between Groq and Nvidia the most telling aspect of the current hardware race. Groq’s Language Processing Unit uses on-chip SRAM to bypass the memory bandwidth bottlenecks that plague Nvidia GPUs during token generation. While Nvidia GPUs handle training and inference tasks across many different frameworks, the Groq LPU focuses solely on running large language models with deterministic, ultra-low latency through its specialized, compiler-scheduled architecture. This approach allows Groq to deliver over 750 tokens per second on Llama 3 70B, whereas standard GPU clusters achieve only 40 to 80 tokens per second. Groq places massive amounts of on-chip SRAM directly inside the LPU die to avoid the latency found when accessing external memory. This trade-off makes the chip expensive to manufacture at scale. Groq extracts more inference output per joule of energy consumed compared to Nvidia’s H100, which draws up to 700 watts under full load. The LPU is not a general-purpose chip. It does not render graphics. It does exactly one thing: run inference on large language models as fast as physically possible.
| Feature | Groq LPU (via GroqCloud API) | Nvidia GPU (A100/H100 Cloud) |
|---|---|---|
| Llama 3 70B Tokens/sec | ~800 – 1,000 t/s | ~60 – 150 t/s |
| First Token Latency | <100ms | 200 – 800ms |
| Custom/Fine-tuned Models | No | Yes |
| Self-hosted Option | No | Yes |
| Context Length (max) | 8K – 128K | 128K – 1M |
Meta’s tiered AI strategy
Meta spends between $115 billion and $135 billion on infrastructure in 2026 to support its Meta Superintelligence Labs. Alexandr Wang leads this superintelligence effort after Meta took a 49% stake in Scale AI for $14.8 billion in June 2025. Mark Zuckerberg argues that open-source AI prevents power concentration, but he also acknowledges that advanced systems require caution regarding what they release openly. Meta released Muse Spark, part of the Avocado model family, through a private preview rather than an immediate open release. Muse Spark includes modes designed around deeper reasoning and multi-agent execution, such as its Contemplating Mode. The deployment of Muse Spark via the app and website is a departure from Meta’s earlier patterns of immediate open access. I see this as a clear shift in how Meta handles its frontier models. The company plans to offer open versions later, but these later versions might be smaller or have capability limits to manage safety risks. Internal codenames like Mango also suggest a focus on image and video generation. This hybrid strategy allows Meta to maintain a frontier edge while keeping developer mindshare through open weights. To lead this effort, Chirantan Desai joined Meta as Chief Enterprise Platform Officer. Desai previously served as CEO and President at MongoDB and held leadership roles at Cloudflare and ServiceNow. Gemma 4 from Google released its models under the Apache 2.0 license, which provides commercial flexibility. In contrast, Llama 4 uses a community license with restrictions, such as the EU-related limitations described in Microsoft’s model catalogue.
Infrastructure and deployment
Microsoft integrated Llama 4 Scout and Maverick into Azure AI Foundry this quarter. Maverick handles interactive applications, such as chatbots for customer support, which process images uploaded by users. Maverick contains over 400 billion parameters, supports twelve different languages, and has a context length of one million parameters. Scout focuses on writing reports and summaries from large amounts of data. The leadership of the Meta Enterprise Platform aims to help businesses use AI to grow and change in new ways. You should check if your current provider runs standard acceptance tests before you commit to a large-scale deployment. Meta reports that its largest AI clusters outgrow single regions, making cable distance and network buffer management a physical design constraint. The Prometheus site reaches roughly 1 gigawatt in 2026, and the Hyperion site scales toward 5 gigawatts. Meta uses a layer called BAG to join smaller fabrics into one cluster across buildings and regions. To handle the distance, the FBOSS software measures each cable in place to set the buffer. This requires using IceCube switch blades, which provide 102.4 terabits per second of bandwidth and 128 ports of 800 gigabits. This hardware is built on either Broadcom or Nvidia chips. Meta also uses Tahan, an Ethernet switch that links accelerators inside a rack, to reduce costs. Meta works with partners through the Open Switch Platform Architecture project to standardize the switch platform layer. How will companies manage the cost of these massive, multi-region clusters?
