Scaling Llama 3.3 70B on Lambda Labs 8x H100 instances

Deployment and VRAM demands

Lambda charges $3.99 per hour for an H100 SXM GPU within an 8-GPU node. These nodes use NVLink and NVIDIA Quantum-2 InfiniBand to provide 3200 Gbps of bandwidth between GPUs. I find this interconnect performance necessary when I train the Llama 3.3 70B model. The model has 80 layers and uses grouped-query attention with 8 KV heads and a 128k-token context window. The model weights consume 141 GB at 16-bit precision. A LoRA fine-tune on a 16-bit base requires 173 GB of VRAM, while a full fine-tune with Adam in mixed precision demands 1,158 GB. If I use 4-bit quantization for a QLoRA training run, I only need 67 GB of VRAM. You know the drill: if you train a 70B model, you must watch your VRAM carefully. For context, serving this model at FP8 with an 8k-token context and one sequence takes about 78 GB of total memory once I include the KV cache and runtime overhead. Unsloth allows me to train these models faster by using specific methods like train_on_completions to ignore loss on user inputs. I can also use the get_chat_template function to ensure the model uses the correct conversation style. I can save the final model as LoRA adapters to Hugging Face using push_to_hub or save it locally with save_pretrained.

Limitations in billing and availability

Lambda’s billing model creates unnecessary expenses for researchers running iterative experiments. The platform rounds all usage up to the full hour, meaning a job finishing in 47 minutes costs as much as a 60-minute run. I calculate that ten such 47-minute jobs a day at $3.29 per hour per GPU creates $213 in monthly rounding overhead. Lambda also lacks spot instances and serverless GPU products, so I must pay full on-demand rates even for workloads that could tolerate interruption. This constraint makes Lambda a poor choice for teams needing cost flexibility or inference-first infrastructure. When I run a job on an 8x H100 SXM node, the inter-GPU bandwidth reaches 3200 Gbps through NVIDIA Quantum-2 InfiniBand, which helps mitigate the communication bottlenecks that often slow down distributed training for 70B parameter models. I also notice that Lambda operates primarily in US regions like California and Texas, which limits my options if I require data residency in Europe or Asia. For European workloads, I look to Nebius or Hyperstack instead. I also see performance issues with AWQ quantization on 8x H100 GPUs; specifically, performance degrades for Llama 3.3 70B once the batch size rises above 4. Lambda’s H100 PCIe inventory goes out of stock regularly, leaving teams to wait or scramble for alternatives. Does Lambda plan to improve H100 PCIe availability during peak demand periods?

Comparison of H100 cloud providers

Thunder Compute provides H100 PCIe at $3.20 per hour, which undercuts Lambda by 3%. Thunder Compute includes 100GB of storage per GPU at no extra charge. For users needing more flexibility, Spheron offers H100 on-demand for $2.98 per hour with no commitment. Spheron bills per minute, which eliminates the rounding overhead I encounter on Lambda. For the lowest possible prices, I look toward marketplaces like Vast.ai where H100 instances appear from $1.87 per hour. RunPod offers H100 community cloud instances for around $1.99 per hour, though I must tolerate some variability in host quality. If I need to escape the constraints of single-node deployments, Hyperstack offers H100 SXM on-demand at $2.40 per hour and scales up to 16,384 GPUs. For large-scale HPC workloads, CoreWeave provides the necessary infrastructure, though I pay negotiated rates typically between $2.75 and $4.00 per hour for H100 spot capacity. Jarvislabs also provides H100 SXM at $3.49 per hour, which beats Lambda’s $4.29 per hour rate for a single-GPU SXM instance. GMI Cloud provides H100 GPUs at $2.00 per hour with serverless inference that scales to zero.