⚑ LLMVRAMCalc.com
EN δΈ­ζ–‡ ES DE
VRAM-Rechner β€Ί GPUs β€Ί 4x NVIDIA A100 80GB (320GB)

What LLMs Run on 4x NVIDIA A100 80GB (320GB) (320GB VRAM)?

4x NVIDIA A100 80GB (320GB) (320GB VRAM, 2039 GB/s bandwidth, 1248 TFLOPS FP16).

Modellname Parameter (Aktiv) Q4_K_M (4-bit) FP16 VRAM Status on 4x NVIDIA A100 80GB (320GB) Decode / Prefill Report
DeepSeek-R1 671B (MoE) 671B 378.7 GB 1250.87 GB Nein ~13.9 / 8770 tok/s Calculate β†’
DeepSeek-V3-0324 671B (MoE) 671B 378.7 GB 1250.87 GB Nein ~13.9 / 8770 tok/s Calculate β†’
DeepSeek-R1-Distill-Llama-70B 70.55B 43.09 GB 134.41 GB FP16 & Q4 ~32.9 / 4599 tok/s Calculate β†’
DeepSeek-R1-Distill-Qwen-32B 32.76B 21.23 GB 63.51 GB FP16 & Q4 ~70.8 / 9905 tok/s Calculate β†’
DeepSeek-R1-Distill-Qwen-14B 14.77B 10.61 GB 29.5 GB FP16 & Q4 ~157.1 / 18000 tok/s Calculate β†’
DeepSeek-R1-Distill-Llama-8B 8.03B 6.22 GB 16.45 GB FP16 & Q4 ~289 / 18000 tok/s Calculate β†’
DeepSeek-R1-Distill-Qwen-7B 7.62B 5.43 GB 15.12 GB FP16 & Q4 ~304.6 / 18000 tok/s Calculate β†’
Qwen3-235B-A22B (MoE) 235.1B 134.4 GB 439.87 GB Fits Q4_K_M ~104.5 / 14616 tok/s Calculate β†’
Qwen3-30B-A3B (MoE) 30.53B 18.53 GB 58.1 GB FP16 & Q4 ~350 / 18000 tok/s Calculate β†’
Qwen3-32B 32.76B 21.22 GB 63.51 GB FP16 & Q4 ~70.8 / 9905 tok/s Calculate β†’
Qwen3-14B 14.77B 10.36 GB 29.25 GB FP16 & Q4 ~157.1 / 18000 tok/s Calculate β†’
Qwen3-8B 8.19B 6.47 GB 16.87 GB FP16 & Q4 ~283.4 / 18000 tok/s Calculate β†’
Qwen3-4B 4.02B 3.61 GB 8.68 GB FP16 & Q4 ~350 / 18000 tok/s Calculate β†’
Qwen2.5-72B-Instruct 72.71B 44.38 GB 138.43 GB FP16 & Q4 ~31.9 / 4463 tok/s Calculate β†’
Qwen2.5-Coder-32B-Instruct 32.76B 21.23 GB 63.51 GB FP16 & Q4 ~70.8 / 9905 tok/s Calculate β†’
Qwen2.5-14B-Instruct 14.77B 10.61 GB 29.5 GB FP16 & Q4 ~157.1 / 18000 tok/s Calculate β†’
Qwen2.5-7B-Instruct 7.62B 5.43 GB 15.12 GB FP16 & Q4 ~304.6 / 18000 tok/s Calculate β†’
QwQ-32B Reasoning 32.76B 21.23 GB 63.51 GB FP16 & Q4 ~70.8 / 9905 tok/s Calculate β†’
Llama-4-Maverick-17B-128E (400B MoE) 400B 227.32 GB 747.05 GB Fits Q4_K_M ~134.9 / 18000 tok/s Calculate β†’
Llama-4-Scout-17B-16E (109B MoE) 109B 63.7 GB 205.02 GB FP16 & Q4 ~134.9 / 18000 tok/s Calculate β†’
Llama-3.3-70B-Instruct 70.55B 43.09 GB 134.41 GB FP16 & Q4 ~32.9 / 4599 tok/s Calculate β†’
Llama-3.1-405B-Instruct 405B 233.02 GB 758.82 GB Fits Q4_K_M ~5.7 / 801 tok/s Calculate β†’
Llama-3.1-70B-Instruct 70.55B 43.09 GB 134.41 GB FP16 & Q4 ~32.9 / 4599 tok/s Calculate β†’
Llama-3.1-8B-Instruct 8.03B 6.22 GB 16.45 GB FP16 & Q4 ~289 / 18000 tok/s Calculate β†’
Llama-3.2-3B-Instruct 3.21B 3.33 GB 7.34 GB FP16 & Q4 ~350 / 18000 tok/s Calculate β†’
Mistral-Large-2-123B 122.6B 72.84 GB 231.62 GB FP16 & Q4 ~18.9 / 2647 tok/s Calculate β†’
Mistral-Small-3.1-24B-Instruct 23.6B 15.28 GB 45.7 GB FP16 & Q4 ~98.3 / 13749 tok/s Calculate β†’
Mixtral-8x22B-Instruct (MoE) 140.6B 81.38 GB 264.13 GB FP16 & Q4 ~59.4 / 8299 tok/s Calculate β†’
Mixtral-8x7B-Instruct (MoE) 46.7B 27.8 GB 88.47 GB FP16 & Q4 ~179.9 / 18000 tok/s Calculate β†’
Codestral-22B 22.2B 14.81 GB 43.59 GB FP16 & Q4 ~104.5 / 14616 tok/s Calculate β†’
Google Gemma-3-27B-IT 27.4B 20.34 GB 55.4 GB FP16 & Q4 ~84.7 / 11842 tok/s Calculate β†’
Google Gemma-3-12B-IT 12.2B 10.75 GB 26.21 GB FP16 & Q4 ~190.2 / 18000 tok/s Calculate β†’
Google Gemma-3-4B-IT 4.3B 4.24 GB 9.56 GB FP16 & Q4 ~350 / 18000 tok/s Calculate β†’
Microsoft Phi-4 14B 14.66B 10.5 GB 29.36 GB FP16 & Q4 ~158.3 / 18000 tok/s Calculate β†’
Cohere Command-A 111B 111B 66.19 GB 209.26 GB FP16 & Q4 ~20.9 / 2923 tok/s Calculate β†’
GLM-4-32B-0414 32B 20.77 GB 62 GB FP16 & Q4 ~72.5 / 10140 tok/s Calculate β†’