⚑ LLMVRAMCalc.com
EN δΈ­ζ–‡ ES DE
VRAM-Rechner β€Ί Modelle β€Ί Llama-3.1-405B-Instruct

Llama-3.1-405B-Instruct VRAM Requirements & GPU Benchmarks

Llama-3.1-405B-Instruct (405B total, 405B active, 126L, 8 KV heads, vocab 128256) requires 233.02 GB in 4-bit Q4_K_M, 393.55 GB in FP8, and 758.82 GB in FP16 at 8K context.

CLOUD-GPU PREISVERGLEICH & MIETEN-VS-KAUFEN RECHNER

Nicht genug lokaler VRAM? Cloud-GPU in 60 Sekunden starten (Llama-3.1-405B-Instruct: 233.02 GB in 4-bit)

Recommended Cloud Tier: Multi-GPU H100 / H200 Node Β· Full FP16 Needs: 758.82 GB

Amortisationszeit (Kauf vs. Miete): ~32 months (vs $1,699 hardware)
Cloud Provider RTX 4090 (24GB) A100 (80GB) H100 (80GB) Est. Monthly Cost (4h/day) Free Credit / Trial Deploy
RunPod $0.44 / hr $1.39 / hr $2.39 / hr $52.80 / mo Instant PyTorch / vLLM Templates + Serverless GPU Auf RunPod starten β†’
Vast.ai $0.29 / hr $0.95 / hr $1.79 / hr $34.80 / mo Save 50–70% on RTX 4090 / 5090 / A100 rentals Spot-GPU mieten β†’
Vultr Cloud GPU β€” $1.79 / hr $2.69 / hr $214.80 / mo (A100) $300 Free Credit $300 Guthaben sichern β†’
DigitalOcean GPU β€” β€” $2.99 / hr $358.80 / mo (H100) $200 Free Credit (60d) $200 Startguthaben β†’
GPU Hardware VRAM Bandwidth FP16 (758.82GB) Q4_K_M (233.02GB) Decode / Prefill Speed Report
NVIDIA RTX 5090 32 GB 1792 GB/s Nein Nein ~1.2 / 134 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 4090 24 GB 1008 GB/s Nein Nein ~1.2 / 106 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 5080 16 GB 960 GB/s Nein Nein ~1.2 / 72 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 4080 Super 16 GB 736 GB/s Nein Nein ~1.2 / 67 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 4070 Ti Super 16 GB 672 GB/s Nein Nein ~1.2 / 57 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 3090 24 GB 936 GB/s Nein Nein ~1.2 / 46 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 4060 Ti 16GB 16 GB 288 GB/s Nein Nein ~1.2 / 45 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 3060 12GB 12 GB 360 GB/s Nein Nein ~1.2 / 45 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX 6000 Ada 48 GB 960 GB/s Nein Nein ~1.2 / 117 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA RTX A6000 48 GB 768 GB/s Nein Nein ~1.2 / 50 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA L40S 48GB 48 GB 864 GB/s Nein Nein ~1.2 / 232 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA A100 80GB SXM 80 GB 2039 GB/s Nein Nein ~1.2 / 200 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA H100 80GB SXM 80 GB 3350 GB/s Nein Nein ~1.2 / 635 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA H200 141GB SXM 141 GB 4800 GB/s Nein Nein ~1.2 / 635 tok/s Exakten VRAM & Speed berechnen β†’
NVIDIA Blackwell B200 192GB 192 GB 8000 GB/s Nein Nein ~1.3 / 1444 tok/s Exakten VRAM & Speed berechnen β†’
AMD Instinct MI300X 192GB 192 GB 5300 GB/s Nein Nein ~1.3 / 839 tok/s Exakten VRAM & Speed berechnen β†’
2x NVIDIA RTX 3090 (48GB) 48 GB 936 GB/s Nein Nein ~1.2 / 91 tok/s Exakten VRAM & Speed berechnen β†’
2x NVIDIA RTX 4090 (48GB) 48 GB 1008 GB/s Nein Nein ~1.2 / 212 tok/s Exakten VRAM & Speed berechnen β†’
4x NVIDIA A100 80GB (320GB) 320 GB 2039 GB/s Nein Ja (73%) ~5.7 / 801 tok/s Exakten VRAM & Speed berechnen β†’
8x NVIDIA H100 80GB (640GB) 640 GB 3350 GB/s Nein Ja (36%) ~9.4 / 5079 tok/s Exakten VRAM & Speed berechnen β†’
Apple M4 Max (128GB Unified) 128 GB 546 GB/s Nein Nein ~1.2 / 45 tok/s Exakten VRAM & Speed berechnen β†’
Apple M3 Ultra Mac Studio (512GB) 512 GB 819 GB/s Nein Ja (46%) ~2.5 / 45 tok/s Exakten VRAM & Speed berechnen β†’
Apple M4 Pro Mac Mini (64GB) 64 GB 273 GB/s Nein Nein ~1.2 / 45 tok/s Exakten VRAM & Speed berechnen β†’