The economics of Lambda Labs GPU scaling and pricing

Lambda Labs offers H100 SXM GPUs at $4.29 per hour for single units, while competitors like Vast.ai and RunPod provide lower rates starting at $1.87. Pricing varies significantly between on-demand single instances and multi-node clusters for large-scale AI…

The economics of Lambda Labs GPU scaling and pricing

On-demand rates and cluster pricing

Lambda charges $4.29 per hour for a single on-demand H100 SXM GPU, while an 8-GPU configuration reduces the rate to $3.99 per GPU hour. The PCIe variant of the H100 costs $3.29 per hour for a single unit. For the A100 40GB model, the price stays at a flat $1.99 per hour regardless of whether a user selects one or eight GPUs. The B200 SXM6 hardware costs $6.99 per hour for a single GPU and drops to $6.69 per hour in an 8-GPU setup. Lambda 1-Click Clusters, which allow for multi-node training, cost more than on-demand instances. A 16-GPU H100 cluster costs $6.16 per GPU hour, which is 44 percent higher than the $4.29 on-demand rate. A 16-GPU B200 cluster costs $9.86 per GPU hour. For larger scale needs, a 256-GPU H100 cluster costs $5.54 per GPU hour, while a 256-plus GPU B200 cluster costs $8.87 per GPU hour. Even the 64-GPU H100 cluster at $5.85 per GPU hour remains higher than the single-unit on-demand price. Older hardware remains available, with the Tesla V100 at $0.79 per hour and the A10 at $1.29 per hour.

GPU Model Configuration Price/GPU/Hour
H100 SXM 1x $4.29
H100 SXM 8x $3.99
H100 PCIe 1x $3.29
A100 (40GB) 1x to 8x $1.99
B200 SXM6 1x $6.99
B200 SXM6 8x $6.69

Marketplace competition and single-node costs

Single-node developers often search for alternatives to Lambda to reduce compute spend. Jarvislabs lists H100 SXM at $3.49 per hour, which saves users 19 percent compared to Lambda. Thunder Compute prices H100 PCIe at $3.20 per hour, which is 16 percent less than Lambda. For A100 80GB workloads, Thunder Compute charges $1.09 per hour per GPU, whereas Lambda’s 8-GPU SXM node costs $3.19 per GPU. This 66 percent difference makes Thunder Compute a much cheaper option for A100 users. Thunder Compute also includes 100GB of storage per GPU at no extra charge. Marketplace options like Vast.ai and RunPod also present lower rates, with H100 prices at $1.87 and $1.99 per hour respectively. GMI Cloud has H100 GPUs at $2.00 per hour and H200 GPUs at $2.60 per hour. Spheron lists H100 SXM5 at $2.64 per hour and H200 at $5.54 per hour. RunPod’s secure cloud H100s cost between $2.39 and $2.69 per hour. You should check the live rates if your project relies on specific hardware availability.

Provider Model Price/Hour
Lambda H100 SXM $4.29
Jarvislabs H100 SXM $3.49
Thunder Compute H100 PCIe $3.20
GMI Cloud H100 $2.00
RunPod H100 $1.99
Vast.ai H100 $1.87

Infrastructure and operational constraints

Lambda builds its infrastructure around AI training using NVIDIA Quantum-2 InfiniBand to deliver 3200 Gbps of bandwidth per cluster. This setup supports distributed training for large-scale models across up to 8 GPUs per instance. The platform includes the Lambda Stack, which pre-installs PyTorch, TensorFlow, NVIDIA CUDA, and cuDNN. While Lambda provides high-bandwidth networking through Quantum-2 InfiniBand with 3200 Gbps bandwidth, the platform lacks the flexibility required for users who need to scale inference workloads using serverless or spot-priced instances. The service is a poor choice for inference-first teams because it lacks serverless GPU offerings. Users pay full on-demand rates for every instance because the platform has no spot or preemptible instances. Lambda also restricts its service to specific regions, primarily in the US, which prevents users from meeting certain data residency requirements in Europe or Asia. Availability also remains a problem, as H100 inventory frequently sells out during periods of high demand. Lambda also lacks a published rate for H200 instances. Lambda is the correct choice for teams requiring large-scale, managed InfiniBand clusters. Will the increasing demand for Blackwell architecture force Lambda to change its cluster pricing model?

airtrain.ai
airtrain.ai

The airtrain.ai newsroom covers AI research, models and the tools built on them.

More on this topic

Stay ahead of AI

Get the week's most important AI stories delivered to your inbox every Monday.

No spam. Unsubscribe anytime.

More Stories