Customizing Llama models via API
I recommend Together AI for developers who need to fine-tune open-weight models like Llama and DeepSeek. The platform hosts over 200 models across eight categories, including chat, vision, image, audio, embeddings, rerank, and moderation. You can implement fine-tuning using either full fine-tuning or LoRA. LoRA fine-tuning trains faster and costs less than a full fine-tune. It is the default choice for most production use cases where a model needs to adopt a specific tone or domain vocabulary. Full fine-tuning provides deeper behavior changes but costs more per million training tokens for models in the 70-100B range.
You connect to the platform using an OpenAI-compatible REST API. This means most existing code that communicates with the OpenAI API works against Together AI with just a base URL and API key change. You should store your API key in a .env file to maintain security, as you already know how to manage environment variables. For a custom project, you can organize your Python directory with files like chat.py for language models, vision_chat.py for vision language models, and image_gen.py for FLUX models. You can also build a Gradio application to manage text streaming and chat history. New accounts receive $25 in free credit to experiment with paid models. The company has raised $500 million in total funding and reached an $8.3 billion valuation in July 2026.
Cost analysis and model selection
I suggest using the Llama 3.3 70B serverless endpoint for high-performance chat tasks. This model costs $1.04 per million input and output tokens. If you require a reasoning model, Kimi K3 costs $3 per million input tokens and $15 per million output tokens. I find that the Batch API provides a significant advantage for non-urgent workloads because it allows you to take up to 50% off standard serverless rates for selected models. Always check the specific model page before assuming a context length, as context windows vary widely across the catalog.
| Model/Service | Rate Type | Cost |
|---|---|---|
| Llama 3.3 70B | Serverless | $1.04 / 1M tokens |
| Kimi K3 | Input | $3.00 / 1M tokens |
| Kimi K3 | Output | $15.00 / 1M tokens |
| H100 | On-demand | $3.99 / hour |
| H100 | Dedicated | $5.49 / hour |
| H200 | On-demand | $6.60 / hour |
| L40 | On-demand | $1.49 / hour |
| L40S | On-demand | $2.10 / hour |
I skip Groq for fine-tuning workflows because Groq does not support fine-tuning. Groq provides faster inference for latency-sensitive tasks like live voice agents. I also skip Together AI if you need reranking models in your serverless pipeline, as the catalog currently lacks rerank models. For teams with high volume, I suggest evaluating the dedicated H100 rates at $5.49 per hour against the $3.99 on-demand rate. For small, efficient models, training costs can start as low as $0.48 per million tokens. I note that for models in the 70-100B range, full fine-tuning runs several dollars per million tokens.
I skip Fireworks AI if you prioritize a narrower catalog with strong latency optimization, although they provide good speed for certain models. I recommend Replicate if you need a massive variety of niche community models through their pay-per-call model. For users seeking a single API for dozens of providers with automatic failover, OpenRouter provides a good alternative, but it adds a 5.5% fee on prepaid credits.
Infrastructure and deployment architecture
The decision between serverless and dedicated infrastructure depends on your specific traffic patterns. Serverless inference works well for bursty workloads, but I suggest moving to dedicated or reserved capacity once your usage patterns become predictable enough to justify the higher hourly cost of the specialized hardware, given that dedicated inference provides total isolation from noisy neighbors. Dedicated inference provides a dedicated URL endpoint and prevents noisy neighbors from affecting your inference speed or VRAM availability. You can rent H100 GPUs at $3.99 per hour for on-demand clusters or $5.49 per hour for dedicated inference.
If you need more capacity, the H200 costs $6.60 per hour, while the L40 costs $1.49 per hour and the L40S costs $2.10 per hour. You can also access A100 SXM 80GB GPUs for $2.40 per hour. For long-term capacity, reserved clusters with 30-day or 180-day commitments drop the per-hour rate. I recommend using the dedicated clusters for production workloads that require high isolation.
I wonder if the shift toward specialized hardware like the NVIDIA Blackwell GPUs will eventually make serverless prices too volatile for long-term enterprise budgeting? The company is already deploying optimized clusters of NVIDIA Blackwell GPUs across multiple data centers in North America, and it has partnered with Hypertec to co-build a cluster of 36,000 NVIDIA GB200 NVL72 GPUs.
