Which GPU is cheapest at your answer time?
Measured, not estimated: vLLM serving real requests on rented GPUs, cost per 1M output tokens at the load that still meets your answer-time target. By Qvunex.
Cost scales with price: a card at twice the price costs twice as much per token at the same load.
| GPU | at once | p50 s | p95 s | $ / 1M out |
|---|
Bold rows meet your target.
What this is. Qwen2.5-7B-Instruct, vLLM 0.27.1 with default flags, prompts of about 512 tokens and 256-token answers, 60 s at each load, one run per load. Cost = GPU price x the measured window. Request and token counts were checked against vLLM's own /metrics counters and matched exactly.
What it is not. Only measured points are used: nothing is interpolated between loads or carried over to other models, prompt lengths or GPUs. Costs assume the GPU is busy the whole time; idle hours in a real day raise them, and only your own traffic shows by how much.
Raw data, server logs and method: benchmarks/corpus. Same answer from the command line: python3 cheapest.py --p95 8.
Want this for your own model and settings? Send your model, vLLM flags and typical prompt and answer length, and I run a short mini-report on a rented GPU for free. Full report: paid work. qvunexaudit@gmail.com