Getting Started with Together AI’s Serverless Inference and Fine-Tuning

Together AI’s serverless inference endpoints bill per million tokens, matching the model maker’s list price for most open-weight models. On 19 October 2026, DeepSeek V4.1 Flash cost $0.30 for input and $1.20 for output, while GLM-5.3 ran $1.40 in and $4.40 out. These rates are identical to the providers’ own APIs, making Together a fair choice if you want a single US-based account for many models. It is not a discount host; cheaper hosts exist for every model tested, often using quantized versions like fp8 or fp4. The platform also supports fine-tuning via LoRA and full fine-tune across major model families, allowing teams to customize behavior without moving weights between vendors. This workflow continuity means you can start with serverless APIs, test multiple models behind an OpenAI-compatible interface, and move heavier traffic to dedicated endpoints when needed.

Serverless Inference: Speed Over Price

The primary advantage of Together AI is speed. In tests conducted on 19 October 2026, Together was the fastest host for all six models compared, beating competitors by wide margins. DeepSeek V4.1 Flash answered in 0.76 seconds, with the first word arriving after 0.38 seconds, whereas DeepInfra took 2.12 seconds at the same or lower price. GLM-5.3 completed in 2.80 seconds on Together against 5.36 seconds on Z.ai’s endpoint, despite identical pricing. Qwen3.8 finished in 2.95 seconds compared to 11.46 seconds on Novita, a difference that matters significantly for chat windows where visitors wait for complete answers. Automatic prompt caching further reduces costs on supported models; for DeepSeek V4.1 Flash, caching halved the bill from $1.41 to $0.73 per 1,000 answers by reading about 2,300 of 4,200 input tokens from cache. However, caching does not work uniformly. GLM-5.3 Flash had zero cache hits across six calls, billing the full $0.77, while Llama 3.3 70B has no cached rate at all, costing ten times more than the cheapest host for the same answer.

For small business sites handling a few thousand answers monthly, the cost difference between Together and the cheapest host is only a few dollars, making the speed advantage worth it. At higher volumes, such as 100,000 answers on GLM-5.3, the gap grows to $270 per month. If your workload requires custom models rather than catalog entries, you will likely need dedicated endpoints, which cost $5.49 per H100-hour. A single H100 running continuously costs about $4,008 a month, a sum that buys roughly 5.5 million DeepSeek V4.1 Flash answers on the serverless API. Unless you handle millions of conversations or require specific fine-tuned models, serverless remains the more economical option for most teams.

Fine-Tuning and SDK Integration

Fine-tuning on Together AI allows you to send data and receive a tuned model back, with options for LoRA and full fine-tune. Larger model and longer context support were added in September 2025, expanding the range of customizable models. The process integrates seamlessly with the platform’s serving layer, so you do not need to move weights to another vendor for deployment. For developers, the Python SDK v2.0 Release Candidate provides a type-safe, OpenAPI-driven client that improves editor support and reduces edge-case inconsistencies. The new SDK uses httpx instead of requests, resulting in about 20ms faster performance per request on average. Migration from v1 is straightforward for common workloads like chat completions, embeddings, and image generation, which are drop-in compatible. More complex APIs, such as batches and evaluations, require updates to method names and parameter structures, with client.files.retrieve_content replaced by client.files.content. The error hierarchy has been redesigned into specific classes like RateLimitError and NotFoundError, replacing the older mixed-bag exceptions. This structure makes debugging more predictable and helps large codebases maintain confidence during development. You should verify your specific use case against the migration guide, as some legacy CLI commands have been removed or reshaped in v2.