I find the MTIA v2 hardware specifications more practical than generic GPU marketing. Meta designed the MTIA v2 to serve ranking and recommendation models by balancing compute, memory bandwidth, and memory capacity. The architecture uses an 8×8 grid of processing elements. These elements provide 3.5x more dense compute performance and 7x more sparse compute performance than the first generation. Meta tripled the local PE storage, doubled the on-chip SRAM to 128 MB, and increased the SRAM bandwidth by 3.5x. The MTIA v2 chip runs at 1.35 GHz and uses 90W of power. This improvement helps Instagram Reels process high-complexity models with a 100x difference in compute per input sample. Because the architecture provides outsized SRAM capacity relative to typical GPUs, the MTIA v2 maintains high utilization even when batch sizes are low.
| Spec | MTIA v2 Value |
|---|---|
| Clock Rate | 1.35 GHz |
| Vector Core (INT8) | 11.06 TFLOPS/s |
| Vector Core (FP16/BF16) | 5.53 TFLOPS/s |
| Vector Core (FP32) | 2.76 TFLOPS/s |
| SIMD (INT8/FP16/BF16) | 5.53 TFLOPS/s |
| SIMD (FP32) | 2.76 TFLOPS/s |
| Local Memory per PE | 1 TB/s |
| On-chip Memory | 2.7 TB/s |
| Off-chip LPDDR5 | 204.8 GB/s |
| On-chip SRAM | 128 MB |
The MTIA v2 architecture improves model serving throughput by 6x over the v1 system. The v2 design includes 11.06 TFLOPS/s of INT8 performance and 5.53 TFLOPS/s of FP16/BF16 performance per vector core. The local memory provides 1 TB/s per processing element, while the on-chip memory delivers 2.7 TB/s. To support these accelerators, Meta developed a rack-based system that holds up to 72 accelerators using three chassis, each containing 12 boards with two accelerators each. The system uses PCIe Gen5 to increase bandwidth and scalability between accelerators and the host. This density allows the system to accommodate a broad range of model complexities.
Software integration and deployment speed
The team uses the Triton-MTIA compiler backend to generate high-performance code for the MTIA hardware. This backend maximizes hardware utilization and supports high-performance kernels. Because Meta integrates the software stack with the silicon, engineers use PyTorch 2.0 with TorchDynamo and TorchInductor. The Triton-MTIA backend allows developers to scale compute kernel authoring. This development helps Meta deploy chips more rapidly than traditional hardware cycles.
Deployment speeds show the effectiveness of the hardware-software co-design. Meta moved from first silicon to production in 16 regions in less than nine months. The company targets a six-month release cadence for new MTIA generations. You should remember that the MTIA 400 already finished testing in labs and moves toward deployment. The MTIA 400 provides 400% higher FP8 FLOPS and 51% higher HBM bandwidth than the MTIA 300. The MTIA 400 uses two compute chiplets to double compute density and supports advanced versions of MX8 and MX4.
The MTIA roadmap expands from ranking and recommendation to generative AI. The MTIA 300 includes one compute chiplet, two network chiplets, and several HBM stacks. Each compute chiplet contains two RISC-V vector cores, a Dot Product Engine for matrix multiplication, a Special Function Unit for activations, and a Reduction Engine for inter-PE communication. MTIA 450 provides 6x the MX4 FLOPS of FP16/BF16 and adds hardware acceleration for attention and feed-forward network computation. MTIA 500 provides 50% more HBM bandwidth and 80% more HBM capacity compared to the MTIA 450. The MTIA 500 also provides a 43% increase in MX4 FLOPS over the MTIA 450 and uses a 2×2 configuration of smaller compute chiplets.
CapEx efficiency and the NVIDIA competition
Meta plans to spend over $65 billion on capital expenditure this year to build AI infrastructure. This investment helps Meta reduce reliance on NVIDIA hardware. I see that Meta currently uses almost 50,000 NVIDIA H100 GPUs to train Llama 3. NVIDIA’s Selene platform, based on the DGX SuperPOD architecture, set eight records in the latest MLPerf benchmarks. Google Cloud uses NVIDIA Vera Rubin NVL72 systems to target 10x lower cost per token, but Meta relies on its own custom designs for specific workloads. Meta also acquired over 2 gigawatts of clean energy from solar, wind, and nuclear sources to power its data centers.
The MTIA v2 provides 1.5x better performance per watt than the first generation. While Meta invests billions to reduce its reliance on external providers, the company previously discarded its most advanced chip and faced significant challenges matching the performance of established hardware like NVIDIA GPUs.
Can Meta maintain this pace of innovation without hitting a wall with custom silicon design? The MTIA 450 is scheduled for mass deployment in early 2027.
