/blog / guide
Self-hosting Llama 3 70B: cheapest providers per million tokens
Eight providers, the same prompt distribution, ranked by cost per million output tokens.
- guide
- gpu
- llama
- inference
If you’re running Llama-3 70B yourself rather than paying an API, the question is which provider gets you the lowest cost per million tokens. We measured.
The setup
Single H100 80GB, vLLM, batch size tuned per host (since shared hosts vary), 4-bit AWQ quantisation. Prompt mix derived from a real production traffic sample, ~512 input tokens and ~256 output on average.
The ranking
- Runpod Community Cloud H100: $0.41 / 1M output tokens
- Vast.ai 4090 (3 in tensor-parallel): $0.49
- Hetzner GPU box (RTX 6000 Ada): $0.62
- Lambda H100 on-demand: $0.71
- Modal Serverless H100: $0.84
- Paperspace A100 80GB: $1.04
- AWS p5 H100 on-demand: $1.83
- Replicate (managed): $2.40+ depending on quota
The takeaway: rolling your own on Runpod is ~5× cheaper than the managed APIs at meaningful volume. Below ~50M tokens/month it’s probably not worth your engineering time. Above that, it’s an obvious move.
comparison
Nvidia L40 48GB vs A100 40GB: better value for LLM inference?
Compare Nvidia L40 48GB vs A100 40GB for LLM inference. We break down pricing, performance, and which GPU offers better value for your specific AI workloads.
7 min
comparison
Modal vs Replicate vs Runpod: cheapest Llama 3 vLLM inference
Compare Modal, Replicate, and Runpod for Llama 3 inference with vLLM. See our measured cost-per-token, latency, and cold start times to find the cheapest option for your LLM workloads.
5 min
comparison
Modal vs Replicate for Llama 3 Inference: A Cost and Latency Showdown
We pitted two serverless GPU platforms against each other for Llama 3 8B inference, tracking cold starts, throughput, and the real cost per token.
5 min