Together AI moves toward algorithmic intelligence and managed deployment

Together AI utilizes the ATLAS software engine to increase workload speeds by 400 percent through speculative decoding. The platform offers per-token pricing for over 200 models and recently secured $800 million in Series C funding to expand its…

Together AI moves toward algorithmic intelligence and managed deployment

Software-driven inference speeds

I see the market move from renting raw GPU hours to buying specific outcomes through software-driven orchestration. Together AI provides per-token pricing for over 200 models across text, code, image, and multimodal categories. The ATLAS software engine uses speculative decoding to speed up workloads by 400 percent. This engine integrates a lightweight neural network with the main model to draft responses. The system then refines those drafts to maintain accuracy. The company gives 50 percent discounts on batch inference and 50 percent savings on prompt caching for repetitive queries. You already know that frontier-model pricing consumes profit margins. The platform enables companies to train and run workloads on models like DeepSeek, Nemotron, MiniMax, and Kimi at a fraction of the cost of closed systems while maintaining comparable or even superior performance. Researchers at the company developed ThunderAgent, which provides 2x faster agentic inference for synthetic data generation. Other research includes cache-aware prefill-decode disaggregation for 40% faster long-context LLM serving. Consistency diffusion language models achieve 14x faster inference without sacrificing quality. Medusa provides a framework for accelerating LLM generation using multiple decoding heads. StripedHyena-7B offers a glimpse into architectures beyond Transformers. Scale can reach 30 billion tokens per model with any serverless model or private deployment. The platform hosts over 200 models, including LLaMA, DeepSeek, and Qwen. Some users report problems with prepaid billing and unreachable support on self-serve accounts.

Dedicated GPU availability and pricing

The infrastructure depends on high-end GPU clusters including H100, H200, and B200. Together AI provides dedicated endpoints for teams that require speed and control. These dedicated models bill per ready GPU-replica minute. For those who need scale, the company provides self-serve instant clusters. These clusters use the Together Kernel Collection to optimize performance. The company provides three service tiers: Build, Scale, and Enterprise. The Build tier is for early-stage developers with access to base models. The Scale tier provides higher throughput and HIPAA compliance. The Enterprise tier includes geo-redundant deployment and a 99.9% SLA. OpenAI-compatible endpoints allow for a drop-in replacement for proprietary APIs.

Model On-demand Price 91-180 Day Commitment
HGX H100 $3.99/hr $3.19/hr
HGX H200 $5.99/hr $3.99/hr
HGX B200 $8.19/hr $6.79/hr

Compared to Runpod, Together AI does not have a tier below the H100. Runpod provides cards at or under $1.09/hr starting from $0.27/hr. I would choose Together AI if I need managed fine-tuning or a broad per-token model library. Runpod is the better choice if I need a GPU environment for development or if I want per-second billing. Provisioned throughput carries a 99% uptime SLA. Together AI is a more flexible alternative to Fireworks AI because of its fine-tuning pipeline and broader model selection.

Scaling through training and investment

Together AI raised $800 million in Series C funding to increase its infrastructure capacity. The investment, led by Aramco Ventures and including NVIDIA and Salesforce Ventures, pushed the company valuation to $8.3 billion. This capital helps the company plan a fiftyfold increase in its footprint over five years. The company also provides managed fine-tuning for production workloads. This service uses LoRA or full training methods to improve accuracy and control behavior. The founders, including Vipul Ved Prakash, Ce Zhang, Percy Liang, and Chris Re, aim to make intelligence abundant. Vipul Ved Prakash previously co-founded Topsy, which Apple acquired in 2013.

The platform serves thousands of paying customers, including Decagon and Cursor. Decagon, a prominent customer, cut inference costs sixfold after moving to Together AI. Other customers report cost savings between 6x and 60x. The platform provides access to training clusters with thousands of graphics cards. Developers manage these through Kubernetes or Slurm. The system also includes automated fault detection to prevent errors in complex training workflows. The company provides high-performance managed storage with zero egress fees. The annual bookings crossed $1.15 billion last quarter. How will the company manage the massive scaling of infrastructure without increasing the complexity of its tiered pricing?

airtrain.ai
airtrain.ai

The airtrain.ai newsroom covers AI research, models and the tools built on them.

More on this topic

Stay ahead of AI

Get the week's most important AI stories delivered to your inbox every Monday.

No spam. Unsubscribe anytime.

More Stories