VRAM is the gate: it decides which open-weight models a server can run at all. This guide maps model-size classes to their VRAM needs — VRAM ≈ parameters × bytes-per-weight × 1.2 overhead — and resolves each bar against our live GPU plan database.
| Model class | VRAM @ FP16 | VRAM @ 4-bit | Cheapest qualifying server (4-bit) | Monthly |
|---|---|---|---|---|
| 7–9B (Llama 3 8B class)Chatbots, RAG over small corpora, coding assistants for one user | ~20GB | ~5GB | Basic GPU Dedicated Server - T1000DatabaseMart · Nvidia Quadro T1000 · 8GB | $83.30/mo |
| 13–14BBetter reasoning than 7B at modest extra cost | ~34GB | ~9GB | Basic GPU Dedicated Server - K80DatabaseMart · Nvidia Tesla K80 · 24GB | $129.00/mo |
| ~34BStrong coding models, small-team internal assistants | ~82GB | ~21GB | Basic GPU Dedicated Server - K80DatabaseMart · Nvidia Tesla K80 · 24GB | $129.00/mo |
| 70B (Llama 3.3 70B class)Near-frontier open quality; production self-hosting sweet spot | ~168GB | ~42GB | Multi-GPU Dedicated Server - 2xRTX A4000DatabaseMart · 2 x Nvidia RTX A4000 · 32GB ×2 | $419.00/mo |
Rule of thumb: parameters × 2 bytes (FP16) or × 0.5 bytes (4-bit quantized), plus ~20% overhead for KV-cache and activations. A 8B model needs ~20GB at FP16 but only ~5GB at 4-bit; a 70B model needs ~168GB FP16 / ~42GB 4-bit.
For most chat and RAG workloads, modern 4-bit quantization (GPTQ/AWQ/GGUF Q4) loses little measurable quality and cuts VRAM need 4×. Precision-critical tasks (math, code generation at scale) may justify 8-bit or FP16.
Estimate your monthly token volume, multiply by API prices, and compare with a flat GPU rental running an open-weight model. Around-the-clock inference at moderate volume usually crosses over; spiky or low traffic rarely does.
Yes — inference engines like vLLM support tensor parallelism, so two 24GB cards can serve a model needing ~40GB. Multi-GPU plans are listed with their total VRAM in our comparison.
Comparing against API costs instead? Full LLM API pricing table →