Tenstorrent Blackhole AI accelerator: myths vs facts

Tenstorrent Galaxy Blackhole achieves 350 tokens per second per user on DeepSeek R1 671B using 512 chips. This architecture offers a 5x TCO advantage over NVIDIA by utilizing commodity GDDR6 memory to reach a cost of $6 per…

Tenstorrent Blackhole AI accelerator: myths vs facts

Tenstorrent Galaxy Blackhole achieves 350 tokens per second per user on DeepSeek R1 671B using 16 Galaxy units containing 512 chips. This configuration supports batch sizes from 8 to 64 and context lengths up to 128k. The architecture uses 140 tensor cores and 16 SiFive X280 RISC-V cores per chip. Unlike NVIDIA systems that use HBM, Blackhole relies on 32 GB of commodity GDDR6 memory providing 512 GB/s bandwidth. This design helps Tenstorrent reach a cost of $6 per million tokens, which provides a 5x TCO advantage over the $30 per million token cost seen with NVIDIA. The 140 tensor cores per chip provide more compute per dollar at rack scale. You already know the price of NVIDIA H100 units can reach $40,000. The system handles both prefill and decode tasks on the same hardware, allowing for sub-4-second time-to-first-token on 100K context. This unified approach avoids the need for disaggregated, specialized inference chips. Each server provides 210 MB of on-chip SRAM and 6.2 GB of aggregate SRAM across 32 chips.

Hardware tiers and technical specifications

Tenstorrent sells multiple hardware tiers to reach different developers. The P100A PCIe card provides 120 tensor cores and 28 GB of memory for $999. The P150A/B adds scale-out ports and 32 GB of memory for $1,399. The QuietBox workstation costs $11,999 and contains four Blackhole chips with 32 GB GDDR6 and 4x 800G networking. This QuietBox machine has insane 300W TDPs and weighs 36 kg. The Galaxy 6U server contains 32 Blackhole chips that deliver 23 PFLOPS of FP8 compute while providing 1 TB of DRAM with 16 TB/s of aggregate bandwidth across its 56 x 800G Ethernet ports.

Product Tensor Cores Memory Price
Blackhole P100A 120 28 GB GDDR6 $999
Blackhole P150 140 32 GB GDDR6 $1,399
QuietBox 560 32 GB GDDR6 $11,999
Galaxy 6U 4,480 1 TB DRAM $110,000

The previous Wormhole generation caused frustration among developers because the hardware worked poorly for anything outside of development. Blackhole improves this by using 140 tensor cores and 16 SiFive X280 RISC-V cores per card. Each core carries five RISC-V controllers for fine-grained scheduling. This setup allows for much faster video generation than leading GPU systems. Prodia Labs reported a 10x improvement in video generation speed when using a Tenstorrent Galaxy supercluster. The system includes Regular Mode for efficient text generation and Blitz Mode for maximum inference speed suited to code generation. The 6U server provides 56 x 800G Ethernet ports for 11.2 TB/s of scale-out bandwidth. EE Times recorded 255 tokens per second per user for short chatbot prompts during testing.

Deployment and market adoption

Hyundai and LG use Tenstorrent IP for automotive and smart TV applications. LG participated in Tenstorrent’s Series D funding round, and Hyundai Mobis appointed Tenstorrent COO Keith Witek to its board. Equinix uses Galaxy Blackhole servers in its Distributed AI Hub in Ashburn, Virginia, which they built in partnership with BetterBrain and OrionVM. The company targets sovereign AI requirements for the UAE via Infinia. This arrangement for sovereign compute was formalized at Abu Dhabi Finance Week 2025. Turiam and Cirrascale also deploy these systems for image generation and cloud services. The company also works with the Japanese government on its compute sovereignty program via the Leading-edge Semiconductor Technology Center and Rapidus. SingularityNet and UnsungFields represent other strategic partners.

The software stack includes TT-Forge and TT-Lang. The company provides a full stack through its open-source software, which includes the LLK, Metal, and TTNN libraries. Developing kernels in TT-Metalium requires significant engineering effort. Will the software reach CUDA parity? Blackhole provides the best cost per token for large-scale inference. Buy the Galaxy 6U if you run high-volume MoE models. Virtu Financial uses 16 servers for low-latency trading.

airtrain.ai
airtrain.ai

The airtrain.ai newsroom covers AI research, models and the tools built on them.

More on this topic

Stay ahead of AI

Get the week's most important AI stories delivered to your inbox every Monday.

No spam. Unsubscribe anytime.

More Stories