People shaping Groq’s tensor streaming architecture for real-time AI inference

Jonathan Ross and Douglas Wightman founded Groq in 2016 to build a compiler-first chip that eliminates hardware guesswork. Ross, who designed Google’s Tensor Processing Unit as a side project, left the search giant after other companies asked him to prevent AI compute from concentrating in one place. He wanted a world where anyone could access high-speed inference, not just giants with massive capital budgets. The company’s first product was the Tensor Streaming Processor, later rebranded as the Language Processing Unit by Ross and Mark Heaps to clarify its function. This shift happened after ChatGPT made large language models mainstream, making the processor’s purpose obvious to developers and investors alike.

Architectural choices driven by deterministic control

The LPU uses a functionally sliced microarchitecture where memory units are interleaved with vector and matrix computation units. This design exploits dataflow locality in AI compute graphs, improving execution performance and efficiency. The chip avoids traditional reactive components like branch predictors, arbiters, reordering buffers, and caches. Instead, the compiler explicitly controls all execution, guaranteeing determinism in every program run. The first generation TSP achieved a computational density of more than 1 teraop per square millimeter on a 14nm chip operating at 900 MHz. That speed allows open-source models like Llama and Mixtral to generate text at several hundred tokens per second, far faster than many GPU-based services. I find this deterministic approach clearly superior for latency-sensitive tasks because it removes the variability inherent in general-purpose processors.

Samsung manufactures the next-generation LPU v2 on its 4-nanometer process node at the Taylor, Texas foundry. This facility handled the first order for Groq’s chips, signaling a major shift in semiconductor supply chains. The company raised $300 million in a series C round led by Tiger Global Management and D1 Capital Partners, reaching unicorn status with a valuation over $1 billion. Later, BlackRock Private Equity Partners led a $640 million series D round, valuing Groq at $2.8 billion. These funds supported the expansion of GroqCloud, which launched in February 2024 to let developers rent access to the chips. The platform hosts demos of open-source large language models for public access, showcasing the raw speed of the hardware.

Leadership transitions and the Nvidia licensing deal

In December 2025, Nvidia agreed to purchase assets from Groq for approximately $20 billion, a record for the chipmaker. This non-exclusive licensing deal included the transfer of senior executives, including founder Jonathan Ross and president Sunny Madra, to Nvidia. Ross now serves as Chief Software Architect at Nvidia, while Simon Edwards became Groq’s CEO. The company stated it would continue to operate as an independent entity, focusing on its cloud business. By mid-2026, Douglas Wightman had become CEO, leading the shift toward running an AI inference cloud. He oversaw a $650 million raise from existing investors, backstopped by Disruptive and Infinitum, to fund this transition. The net income for 2023 stood at negative $88 million, highlighting the financial strain of building custom silicon before revenue scales. Revenue reached $500 million in 2025, showing strong demand for accelerator chips.

I see the leadership change as a critical test for Groq’s independence. While the Nvidia deal provides massive validation and cash flow, losing Ross and Madra removes the original architects from day-to-day operations. Wightman must now prove he can scale the infrastructure without their direct guidance. The question remains whether a company stripped of its founding engineers can maintain the same architectural edge against established competitors. The answer will depend on how effectively the new team executes the compiler-first vision that defined the early years.