The infrastructure play for open models
Together AI closed an $800 million Series C at an $8.3 billion valuation on July 1, 2026. Aramco Ventures led this round, and NVIDIA and Salesforce Ventures also participated. The company reported annualized bookings exceeding $1.15 billion in its most recent quarter. Decagon cut its inference costs sixfold after it moved workloads to Together. Zomato also uses the platform to run Meta’s Llama models for a customer support bot that handles more than 1,000 messages per minute. The company manages a power grid for open models and holds a 500-megawatt commitment to expand its computing capacity. This infrastructure is the foundation for companies like Cursor and Cognition. The platform includes fine-tuning for Llama, Mistral, and Qwen through LoRA and full fine-tune methods. Together AI also provides dedicated GPU clusters with H100 SXM and A100 options.
The pricing gap between Together AI and AWS
The cost for NVIDIA H100 hardware varies significantly among cloud providers. Together AI lists on-demand H100 clusters at $3.99 per GPU per hour, and its reserved term for up to six months starts from $3.19 per GPU per hour. In contrast, AWS charges $6.88 per hour for an H100 instance. zCLOUD offers H100 starting rates of $3.75 per hour and $2.79 for a reserved term. Organizations that move large training pipelines from AWS to dedicated private clouds avoid the heavy egress fees that reach $0.09 per gigabyte for outbound internet data transfer. Because the AWS Nitro hypervisor architecture shares resources in multi-tenant environments, it causes performance jitter that degrades Model Flops Utilization by 15% to 35% when compared to dedicated bare metal clusters. A training run that generates 200 Terabytes of periodic checkpoint synchronization and dataset streaming incurs thousands of dollars in pure network transit surcharges each month on AWS.
| H100 Option | Together AI | zCLOUD (Starting) | AWS |
|---|---|---|---|
| Hourly | $3.99 | $3.75 | $6.88 |
| Reserved | $3.19 | $2.79 | N/A |
zCLOUD provides a competitive alternative for price-first jobs. When comparing 256 H100 GPUs over 30 days, zCLOUD’s starting rates save about $44,200 by the hour and $73,700 on a reserved term compared to Together AI. However, the final rate for zCLOUD gets set at the moment of purchase based on live supply. Modal offers H100 compute at $3.95 per hour with per-second billing. Baseten charges $6.50 per hour for an H100 using a per-minute billing model. Replicate also bills $5.49 per hour for an H100. AWS SageMaker adds a 20% to 40% cost premium over standard EC2 instances. You should evaluate your own bills before you decide to migrate.
Performance and the transition to dedicated hardware
As traffic grows, the economics that made an API attractive at launch change. Per-token pricing grows linearly with usage, while dedicated GPU cost stays fixed once capacity is provisioned. Every production team passes a crossover point where their monthly token bill covers committed GPUs at high utilization. At this stage, dedicated infrastructure is the strongest alternative for steady production traffic. Dedicated capacity removes the contention and the noisy neighbors that appear in shared-capacity serverless platforms under load. Dedicated infrastructure also puts the data path on known hardware in a chosen data center, which matters for healthcare, financial, and government-adjacent workloads. This architecture provides isolation and logging that the team controls.
Many teams keep an API for bursts and experiments while they serve the baseline on dedicated hardware. This hybrid model allows businesses to capture the benefits of both low-cost burst capacity and high-performance stability. Performance benchmarks for production coding agent workloads show Together’s engine delivers 31% more TPS than industry competitors. It also provides 2 times better TTFT. These gains lead to 76% cheaper per request compared to a leading open-source solution. This saves an estimated $440,000 annually for users. Does the reduction in latency justify the increase in operational responsibility?
