LLM VRAM, KV Cache, Throughput & Cloud GPU Sizing Calculator
Select an open-weight model, quantization format, and context length to calculate exact weight memory in GiB, GQA / sliding-window / hybrid / MLA KV cache, realistic token throughput, and the lowest-cost local or cloud GPU.
Fits with Safe Headroom
Measured GGUF shard size
HF config.json ↗
Required GPU Memory
17.56 GiB
18.85 GB (decimal)
Usable Hardware Memory
22.8 GiB
24 GB (physical)
Safe Headroom
+5.24 GiB
73% utilization
Est. Decode Throughput
~36.7 tok/s
27.78B active params
Est. Prefill Throughput
~1544 tok/s
4K Prompt TTFT: ~2652 ms
Est. Cost / 1M Tokens
$1.43
Hybrid Linear + Full Attention · Apache-2.0
Recommended Hardware for Current Configuration
Best Single GPU
Arc Pro B60 24GB (24GB)
~$599
Best Value / Used GPU
RTX 3090 (24GB)
~$750
Apple Unified Memory
Ryzen AI Max+ 395 128GB
~$1,999
Cloud From
Vast.ai · $0.19 / hr
1x RTX 3090
Production Launch Commands
llama.cpp (GGUF)
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL \ --ctx-size 8192 -ngl 999 --flash-attn
vLLM
vllm serve Qwen/Qwen3.8-27B \ --max-model-len 8192 --gpu-memory-utilization 0.92
Live Token Speed Simulator (TTFT ~2652ms · ~36.7 tok/s)
[Qwen3.8 27B · 17.56 GiB] allocated on target GPU. Prefill completed in TTFT window; streaming autoregressive tokens at memory-bandwidth-bound rate...
Cloud GPU Comparison & Rent-vs-Buy TCO
Verified public rates across RunPod, Vast.ai, Vultr, and DigitalOcean.
Purchase break-even
~26.6 months
Est. local power: $4.26 / mo
| Provider | Recommended Instance | Total VRAM | Hourly Rate | Monthly at Selected Hours | Billing Mode | Deploy |
|---|---|---|---|---|---|---|
| RunPod | 1x RTX 3090 (Community Cloud) | 24 GB | $0.22 / hr | $26.75 / mo | Per-second | Deploy GPU ↗ |
| Vast.ai | 1x RTX 3090 (Verified Host) | 24 GB | $0.19 / hr | $22.98 / mo | Spot / On-demand | Deploy GPU ↗ |
| Vultr ($300 Free Credit*) | 1x L40S 48GB (Dedicated Instance · $300 Credit*) | 48 GB | $1.67 / hr | $203.19 / mo | Hourly | Deploy GPU ↗ |
| Vultr Cloud GPU | 1x L40S 48GB (Dedicated Instance) | 48 GB | $1.67 / hr | $203.19 / mo | Hourly | Deploy GPU ↗ |
| DigitalOcean | 1x RTX 6000 Ada (Dedicated Instance) | 48 GB | $1.57 / hr | $190.91 / mo | Per-second | Deploy GPU ↗ |
Rates reflect public listings on the verification date and fluctuate with regional inventory.
Open-Weight LLM Memory Reference Matrix (76)
| Model | Total / Active | Max Context | 4-bit @ 8K | 4-bit @ 32K | Recommended Local GPU (32K) | Cloud From |
|---|---|---|---|---|---|---|
|
Qwen3.8 Flash Next (125B-A6B)
Qwen · Hybrid Linear/Mamba + MoE
|
125B / 6B | 256K | 71.65 GiB (Q4_K_M) | 72.21 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $1.67 / hr ↗ |
|
Qwen3.8 27B
Qwen · Hybrid Linear + Full Attention
|
27.78B / 27.78B | 256K | 17.56 GiB (UD-Q4_K_XL) | 19.06 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Qwen3.8 2.4T-A95B
Qwen · Hybrid Linear/Mamba + MoE
|
2400B / 95B | 256K | 1357.58 GiB (Q4_K_M) | 1359.74 GiB | Multi-GPU Required | DigitalOcean $20.72 / hr ↗ |
|
Qwen3.6 35B-A3B
Qwen · Hybrid Linear/Mamba + MoE
|
35.95B / 3B | 256K | 21.6 GiB (UD-Q4_K_XL) | 22.07 GiB | Arc Pro B60 24GB (24GB) | RunPod $0.33 / hr ↗ |
|
Qwen3.5 397B-A17B
Qwen · Hybrid Linear/Mamba + MoE
|
397B / 17B | 256K | 225.47 GiB (Q4_K_M) | 226.17 GiB | Instinct MI325X 256GB (256GB) | DigitalOcean $3.80 / hr ↗ |
|
Qwen3.5 122B-A10B
Qwen · Hybrid Linear/Mamba + MoE
|
122B / 10B | 256K | 70.03 GiB (Q4_K_M) | 70.59 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Qwen3.5 35B-A3B
Qwen · Hybrid Linear/Mamba + MoE
|
35.95B / 3B | 256K | 21.28 GiB (Q4_K_M) | 21.75 GiB | Arc Pro B60 24GB (24GB) | RunPod $0.33 / hr ↗ |
|
Qwen3.5 27B
Qwen · Hybrid Linear + Full Attention
|
27.78B / 27.78B | 256K | 17.4 GiB (Q4_K_M) | 18.9 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Qwen3.5 9B
Qwen · Hybrid Linear + Full Attention
|
9B / 9B | 256K | 6.36 GiB (Q4_K_M) | 7.11 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Qwen3.5 4B
Qwen · Hybrid Linear + Full Attention
|
4B / 4B | 256K | 3.37 GiB (Q4_K_M) | 4.12 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Qwen3 Coder 480B-A35B
Qwen · Sparse MoE + GQA
|
480B / 35B | 256K | 273.9 GiB (Q4_K_M) | 279.71 GiB | 2x B200 180GB | RunPod $11.96 / hr ↗ |
|
Qwen3 Coder Next 80B-A3B
Qwen · Hybrid Linear/Mamba + MoE
|
79.67B / 3B | 256K | 46.02 GiB (Q4_K_M) | 46.58 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Qwen3 Coder 30B-A3B
Qwen · Sparse MoE + GQA
|
30.53B / 3.3B | 256K | 18.59 GiB (Q4_K_M) | 20.84 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Qwen3 30B-A3B Instruct 2507
Qwen · Sparse MoE + GQA
|
30.53B / 3.3B | 256K | 18.59 GiB (Q4_K_M) | 20.84 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Qwen3 235B-A22B Instruct 2507
Qwen · Sparse MoE + GQA
|
235.09B / 22B | 256K | 134.42 GiB (Q4_K_M) | 138.83 GiB | Instinct MI300X 192GB (192GB) | DigitalOcean $2.59 / hr ↗ |
|
Qwen3 32B
Qwen · Dense GQA
|
32.76B / 32.76B | 128K | 20.96 GiB (Q4_K_M) | 26.96 GiB | Radeon AI PRO R9700 32GB (32GB) | RunPod $0.33 / hr ↗ |
|
Qwen3 14B
Qwen · Dense GQA
|
14.77B / 14.77B | 128K | 10.19 GiB (Q4_K_M) | 13.94 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.12 / hr ↗ |
|
Qwen3 8B
Qwen · Dense GQA
|
8.19B / 8.19B | 128K | 6.37 GiB (Q4_K_M) | 9.74 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Qwen3 4B Instruct 2507
Qwen · Dense GQA
|
4.02B / 4.02B | 256K | 4.02 GiB (Q4_K_M) | 7.39 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Qwen3 1.7B
Qwen · Dense GQA
|
1.72B / 1.72B | 40K | 2.54 GiB (Q4_K_M) | 5.16 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Qwen3 0.6B
Qwen · Dense GQA
|
0.75B / 0.75B | 40K | 1.92 GiB (Q4_K_M) | 4.54 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Qwen2.5 72B Instruct
Qwen · Dense GQA
|
72.71B / 72.71B | 128K | 44.62 GiB (Q4_K_M) | 52.12 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Qwen2.5 Coder 32B Instruct
Qwen · Dense GQA
|
32.76B / 32.76B | 128K | 21.37 GiB (Q4_K_M) | 27.37 GiB | Radeon AI PRO R9700 32GB (32GB) | RunPod $0.33 / hr ↗ |
|
QwQ 32B Reasoning
Qwen · Dense GQA
|
32.76B / 32.76B | 128K | 21.37 GiB (Q4_K_M) | 27.37 GiB | Radeon AI PRO R9700 32GB (32GB) | RunPod $0.33 / hr ↗ |
|
DeepSeek V4.1 Flash (552B-A16B)
DeepSeek · Sparse MoE + GQA
|
552B / 16B | 1M | 313.12 GiB (Q4_K_M) | 314.99 GiB | 2x B200 180GB | RunPod $11.96 / hr ↗ |
|
DeepSeek V4 Flash
DeepSeek · Sparse MoE + GQA
|
284B / 11B* | 1M | 145.06 GiB (UD-Q4_K_XL) | 145.22 GiB | Instinct MI300X 192GB (192GB) | DigitalOcean $2.59 / hr ↗ |
|
DeepSeek V3.2 (685B)
DeepSeek · Multi-head Latent Attention (MLA) MoE
|
685.36B / 37B | 160K | 378.79 GiB (Q4_K_M) | 380.75 GiB | 4x H200 141GB SXM | Vast.ai $3.54 / hr ↗ |
|
DeepSeek R1-0528 671B
DeepSeek · Multi-head Latent Attention (MLA) MoE
|
671B / 37B | 160K | 378.11 GiB (Q4_K_M) | 379.71 GiB | 4x H200 141GB SXM | Vast.ai $3.54 / hr ↗ |
|
DeepSeek V3-0324 671B
DeepSeek · Multi-head Latent Attention (MLA) MoE
|
671B / 37B | 160K | 378.11 GiB (Q4_K_M) | 379.71 GiB | 4x H200 141GB SXM | Vast.ai $3.54 / hr ↗ |
|
DeepSeek R1 Distill Llama 70B
DeepSeek · Dense GQA
|
70.55B / 70.55B | 128K | 43.32 GiB (Q4_K_M) | 50.82 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
DeepSeek R1 Distill Qwen 32B
DeepSeek · Dense GQA
|
32.76B / 32.76B | 128K | 21.05 GiB (Q4_K_M) | 27.05 GiB | Radeon AI PRO R9700 32GB (32GB) | RunPod $0.33 / hr ↗ |
|
DeepSeek R1 Distill Qwen 14B
DeepSeek · Dense GQA
|
14.77B / 14.77B | 128K | 10.71 GiB (Q4_K_M) | 15.21 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
DeepSeek R1 Distill Llama 8B
DeepSeek · Dense GQA
|
8.03B / 8.03B | 128K | 6.3 GiB (Q4_K_M) | 9.3 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
gpt-oss-20b
OpenAI · Sparse MoE + GQA
|
20.91B / 3.6B | 128K | 11.58 GiB (Q4_K_M) | 12.14 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.12 / hr ↗ |
|
gpt-oss-120b
OpenAI · Sparse MoE + GQA
|
116.83B / 5.1B | 128K | 59.31 GiB (Q4_K_M) | 60.15 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Gemma 4 31B IT
Gemma · Dense GQA
|
31.27B / 31.27B | 256K | 19.04 GiB (Q4_K_M) | 20.91 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Gemma 4 26B-A4B IT
Gemma · Sparse MoE + GQA
|
25.81B / 3.8B | 256K | 16.75 GiB (UD-Q4_K_XL) | 17.22 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Gemma 4 12B IT
Gemma · Dense GQA
|
11.96B / 11.96B | 256K | 7.63 GiB (Q4_K_M) | 8 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Gemma 3 27B IT
Gemma · Dense GQA
|
27.43B / 27.43B | 128K | 17 GiB (Q4_K_M) | 18.88 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Gemma 3 12B IT
Gemma · Dense GQA
|
12.19B / 12.19B | 128K | 8.65 GiB (Q4_K_M) | 10.15 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Gemma 3 4B IT
Gemma · Dense GQA
|
4.3B / 4.3B | 128K | 3.56 GiB (Q4_K_M) | 4.12 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
GLM-5.3 Flash (320B-A18B)
GLM · Multi-head Latent Attention (MLA) MoE
|
320B / 18B | 1M | 181.84 GiB (Q4_K_M) | 182.9 GiB | Instinct MI325X 256GB (256GB) | DigitalOcean $3.80 / hr ↗ |
|
GLM-5.3 (753B)
GLM · Multi-head Latent Attention (MLA) MoE
|
753.33B / 40B* | 1M | 436.61 GiB (UD-Q4_K_XL) | 439.12 GiB | 4x H200 141GB SXM | Vast.ai $3.54 / hr ↗ |
|
GLM-5 (744B)
GLM · Multi-head Latent Attention (MLA) MoE
|
744B / 40B | 198K | 421.71 GiB (Q4_K_M) | 423.76 GiB | 4x H200 141GB SXM | Vast.ai $3.54 / hr ↗ |
|
GLM-4.5 (355B-A32B)
GLM · Sparse MoE + GQA
|
355B / 32B | 128K | 204.19 GiB (Q4_K_M) | 212.81 GiB | Instinct MI325X 256GB (256GB) | DigitalOcean $3.80 / hr ↗ |
|
GLM-4.5 Air (110B-A12B)
GLM · Sparse MoE + GQA
|
110.47B / 12B | 128K | 69.96 GiB (Q4_K_M) | 74.27 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $1.67 / hr ↗ |
|
GLM-4 32B-0414
GLM · Dense GQA
|
32B / 32B | 128K | 20.92 GiB (Q4_K_M) | 26.64 GiB | Radeon AI PRO R9700 32GB (32GB) | RunPod $0.33 / hr ↗ |
|
Kimi K3 (2.8T-A104B)
Kimi · Multi-head Latent Attention (MLA) MoE
|
2780B / 104B | 1M | 1571.49 GiB (Q4_K_M) | 1573.94 GiB | Multi-GPU Required | DigitalOcean $30.40 / hr ↗ |
|
Kimi K2 Instruct (1T-A32B)
Kimi · Multi-head Latent Attention (MLA) MoE
|
1000B / 32B | 128K | 566.19 GiB (Q4_K_M) | 567.79 GiB | 4x B200 180GB | Vast.ai $3.54 / hr ↗ |
|
Kimi-Dev 72B
Kimi · Dense GQA
|
72.7B / 72.7B | 128K | 44.61 GiB (Q4_K_M) | 52.11 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
MiniMax M3.1 Flash (428B-A23B)
MiniMax · Sparse MoE + GQA
|
428B / 23B | 1M | 243.65 GiB (Q4_K_M) | 246.46 GiB | Instinct MI355X 288GB (288GB) | RunPod $6.94 / hr ↗ |
|
MiniMax M2.5 (229B-A10B)
MiniMax · Sparse MoE + GQA
|
228.69B / 10B | 192K | 131.34 GiB (Q4_K_M) | 137.15 GiB | Instinct MI300X 192GB (192GB) | DigitalOcean $2.59 / hr ↗ |
|
Step-3.7 Flash (198B-A11B)
StepFun · Sparse MoE + GQA
|
198B / 11B | 256K | 112.91 GiB (Q4_K_M) | 113.76 GiB | Instinct MI300X 192GB (192GB) | DigitalOcean $2.59 / hr ↗ |
|
Step-3.5 Flash (197B-A11B)
StepFun · Sparse MoE + GQA
|
196.81B / 11B | 256K | 112.24 GiB (Q4_K_M) | 113.09 GiB | Instinct MI300X 192GB (192GB) | DigitalOcean $2.59 / hr ↗ |
|
ERNIE 4.5 300B-A47B (文心)
ERNIE · Sparse MoE + GQA
|
300B / 47B | 128K | 171.98 GiB (Q4_K_M) | 177.04 GiB | Instinct MI325X 256GB (256GB) | DigitalOcean $3.80 / hr ↗ |
|
ERNIE 4.5 21B-A3B (文心)
ERNIE · Sparse MoE + GQA
|
21B / 3B | 128K | 12.96 GiB (Q4_K_M) | 14.27 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.12 / hr ↗ |
|
Hunyuan A13B Instruct (混元 80B)
Hunyuan · Sparse MoE + GQA
|
80B / 13B | 32K | 46.94 GiB (Q4_K_M) | 49.94 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
MiniCPM5 2B (面壁)
MiniCPM · Dense GQA
|
2.3B / 2.3B | 128K | 2.3 GiB (Q4_K_M) | 3.28 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Llama 4 Maverick 17B-128E (400B)
Llama · Sparse MoE + GQA
|
400B / 17.2B | 128K | 228.32 GiB (Q4_K_M) | 232.82 GiB | Instinct MI325X 256GB (256GB) | RunPod $6.94 / hr ↗ |
|
Llama 4 Scout 17B-16E (109B)
Llama · Sparse MoE + GQA
|
109B / 17.2B | 128K | 64.01 GiB (Q4_K_M) | 68.51 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Llama 3.1 405B Instruct
Llama · Dense GQA
|
405B / 405B | 128K | 234.05 GiB (Q4_K_M) | 245.86 GiB | Instinct MI355X 288GB (288GB) | RunPod $6.94 / hr ↗ |
|
Llama 3.3 70B Instruct
Llama · Dense GQA
|
70.55B / 70.55B | 128K | 42.67 GiB (Q4_K_M) | 50.17 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Llama 3.1 8B Instruct
Llama · Dense GQA
|
8.03B / 8.03B | 128K | 6.14 GiB (Q4_K_M) | 9.14 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Llama 3.2 3B Instruct
Llama · Dense GQA
|
3.21B / 3.21B | 128K | 3.41 GiB (Q4_K_M) | 6.03 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |
|
Mistral Large 4 (1T-A52B)
Mistral · Multi-head Latent Attention (MLA) MoE
|
1000B / 52B | 1M | 566.32 GiB (Q4_K_M) | 568.22 GiB | 4x B200 180GB | Vast.ai $3.54 / hr ↗ |
|
Mistral Large 2 123B
Mistral · Dense GQA
|
122.6B / 122.6B | 128K | 73.19 GiB (Q4_K_M) | 81.44 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $1.67 / hr ↗ |
|
Mixtral 8x22B Instruct
Mistral · Sparse MoE + GQA
|
140.6B / 39.1B | 64K | 81.78 GiB (Q4_K_M) | 87.03 GiB | RTX PRO 6000 Blackwell (96GB) | DigitalOcean $2.59 / hr ↗ |
|
Mixtral 8x7B Instruct
Mistral · Sparse MoE + GQA
|
46.7B / 12.9B | 32K | 27.98 GiB (Q4_K_M) | 30.98 GiB | A40 48GB (48GB) | RunPod $0.33 / hr ↗ |
|
Mistral Small 3.2 24B
Mistral · Dense GQA
|
24.01B / 24.01B | 128K | 15.16 GiB (Q4_K_M) | 18.91 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Devstral Small 24B
Mistral · Dense GQA
|
23.6B / 23.6B | 128K | 15.4 GiB (Q4_K_M) | 19.15 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Nemotron 3 Ultra 550B-A55B
NVIDIA · Hybrid Linear/Mamba + MoE
|
550B / 55B | 256K | 311.95 GiB (Q4_K_M) | 312.23 GiB | 2x B200 180GB | RunPod $11.96 / hr ↗ |
|
Nemotron 3 Super 120B-A12B
NVIDIA · Hybrid Linear/Mamba + MoE
|
123.61B / 12B | 256K | 78.81 GiB (UD-Q4_K_XL) | 78.99 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $1.67 / hr ↗ |
|
Nemotron 3 Nano 30B-A3B
NVIDIA · Hybrid Linear/Mamba + MoE
|
31.58B / 3.5B | 256K | 23.54 GiB (Q4_K_M) | 23.68 GiB | Radeon AI PRO R9700 32GB (32GB) | RunPod $0.33 / hr ↗ |
|
Cohere Command A 111B
Cohere · Dense GQA
|
111B / 111B | 256K | 65.75 GiB (Q4_K_M) | 67.25 GiB | RTX PRO 6000 Blackwell (96GB) | Vast.ai $0.44 / hr ↗ |
|
Phi-4 14B
Microsoft · Dense GQA
|
14.66B / 14.66B | 16K | 10.4 GiB (Q4_K_M) | 15.09 GiB | Arc Pro B60 24GB (24GB) | Vast.ai $0.19 / hr ↗ |
|
Liquid LFM3 1.7B
LiquidAI · Hybrid Linear + Full Attention
|
1.7B / 1.7B | 256K | 1.73 GiB (Q4_K_M) | 2.01 GiB | RTX 4060 Ti 16GB (16GB) | Vast.ai $0.06 / hr ↗ |