The talent migration and the $20B deal
NVIDIA’s $20 billion non-exclusive licensing agreement with Groq in December 2025 fundamentally altered the startup’s business model. The deal, which functioned as an IP license and acqui-hire, brought Groq’s low-latency inference technology into NVIDIA’s ecosystem, and Jonathan Ross, the founder who helped design Google’s Tensor Processing Unit, joined NVIDIA as Chief Software Architect. Simon Edwards took over as Groq CEO to lead the company’s shift from chip design to running an AI inference cloud. The $20 billion deal structured as a non-exclusive IP license and acqui-hire allows NVIDIA to integrate Groq’s high-speed, low-latency inference technology into its own AI factory architecture without triggering massive antitrust reviews. This move follows the company’s early days in 2016 when it raised $10.3 million from investors like Chamath Palihapitiya. Since its founding, the company has raised an estimated $3 billion from backers including BlackRock, Neuberger Berman, Deutsche Telekom Capital Partners, Samsung, Cisco, D1, Altimeter, 1789 Capital, and Infinitum. Douglas Wightman, a former Google X engineer, also played a role in the company’s early formation. I find the sudden departure of the core TPU engineering team to a direct competitor a blatant contradiction to Groq’s claim of remaining an independent entity.
LPU architecture and performance benchmarks
The LPU architecture bypasses the memory bandwidth bottleneck of general-purpose GPUs by using on-chip SRAM instead of external HBM. Groq uses patented technology around a matrix engine to efficiently calculate linear algebra operations like convolution. This deterministic processing design allows the LPU to deliver higher token throughput for large language models. During benchmarks with Meta AI’s Llama 2 70b, Groq achieved 241 tokens per second, more than double the speed of other hosting providers. ArtificialAnalysis.ai noted that Groq took only 0.8 seconds to receive 100 output tokens using prompts of 100 tokens.
| Spec Category | LPU Performance / Capacity |
|---|---|
| Llama 2 70b Throughput | 241 tokens per second |
| GPT OSS 20B Speed | 1,000 tokens per second |
| GPT OSS 120B Speed | 500 tokens per second |
| Energy Efficiency | 10x better per token than GPUs |
| Current Power Capacity | 54 megawatts |
| 2027 Power Target | 200 megawatts |
The LPU delivers speeds of 500 tokens per second on GPT OSS 120B models and reaches 1,000 tokens per second on GPT OSS 20B models. For users with latency-sensitive applications like real-time voice agents, the raw speed of the LPU provides a massive advantage. Recent tests show Groq hitting 750 tokens per second in ChatGPT-style responses, while conventional GPUs manage only 30 to 60 tokens. Groq’s partnerships with Meta for Llama 4 inference and Bell Canada for large scale infrastructure show the utility of these deployments. The technology supports over two million developers and Fortune 500 companies. The LPU offers an API that is OpenAI-compatible, providing a drop-in replacement for existing codebases.
The neocloud shift and market competition
Groq’s recent $350 million Series A round, announced in August 2026 and led by Disruptive, values the company at $3.5 billion. This figure is a significant drop from the $6.9 billion valuation the company reached in September 2025. The lower valuation follows the strategic pivot to a neocloud model that uses NVIDIA systems. Groq’s revenue forecast for 2025 was revised downward by more than $1 billion. The company currently operates 13 data centers across North America, Europe, the Middle East, and Asia Pacific to serve over six million developers. Groq also secured a $1.5 billion commitment from Saudi Arabia, which should generate $500 million in revenue this year.
The inference market is growing, with estimates suggesting it will reach $253.75 billion by 2030. Groq faces competition from several specialized players in this space. Baseten is reportedly raising $1.5 billion at a $13 billion valuation, while Modal Labs raised $355 million at a $4.65 billion valuation. Cerebras raised $5.5 billion in its IPO, and TensorWave reached unicorn status with a $350 million Series B. You should look at these capacity numbers closely if you are tracking the scale of the inference race. Competitors like CoreWeave, Lambda, and Nebius also operate within the NVIDIA ecosystem. Groq’s ability to scale depends on the success of its partnership with NVIDIA and its ability to manage huge capital costs. Does the company’s move into a neocloud model ultimately make it just another service provider for NVIDIA’s hardware?
