What LLMs Run on NVIDIA RTX 3060 12GB (12GB VRAM)?
NVIDIA RTX 3060 12GB (12GB VRAM, 360 GB/s bandwidth, 25.4 TFLOPS FP16).
| 模型名称 | 总参数 (激活参数) | Q4_K_M (4位推荐) | FP16 无损显存 | Status on NVIDIA RTX 3060 12GB | Decode / Prefill | Report |
|---|---|---|---|---|---|---|
| DeepSeek-R1 671B (MoE) | 671B | 378.7 GB | 1250.87 GB | 显存不足 | ~2.7 / 178 tok/s | Calculate → |
| DeepSeek-V3-0324 671B (MoE) | 671B | 378.7 GB | 1250.87 GB | 显存不足 | ~2.7 / 178 tok/s | Calculate → |
| DeepSeek-R1-Distill-Llama-70B | 70.55B | 43.09 GB | 134.41 GB | 显存不足 | ~1.8 / 94 tok/s | Calculate → |
| DeepSeek-R1-Distill-Qwen-32B | 32.76B | 21.23 GB | 63.51 GB | 显存不足 | ~5.2 / 202 tok/s | Calculate → |
| DeepSeek-R1-Distill-Qwen-14B | 14.77B | 10.61 GB | 29.5 GB | Fits Q4_K_M | ~27.7 / 447 tok/s | Calculate → |
| DeepSeek-R1-Distill-Llama-8B | 8.03B | 6.22 GB | 16.45 GB | Fits Q4_K_M | ~51 / 822 tok/s | Calculate → |
| DeepSeek-R1-Distill-Qwen-7B | 7.62B | 5.43 GB | 15.12 GB | Fits Q4_K_M | ~53.8 / 867 tok/s | Calculate → |
| Qwen3-235B-A22B (MoE) | 235.1B | 134.4 GB | 439.87 GB | 显存不足 | ~4.7 / 297 tok/s | Calculate → |
| Qwen3-30B-A3B (MoE) | 30.53B | 18.53 GB | 58.1 GB | 显存不足 | ~57.8 / 1977 tok/s | Calculate → |
| Qwen3-32B | 32.76B | 21.22 GB | 63.51 GB | 显存不足 | ~5.2 / 202 tok/s | Calculate → |
| Qwen3-14B | 14.77B | 10.36 GB | 29.25 GB | Fits Q4_K_M | ~27.7 / 447 tok/s | Calculate → |
| Qwen3-8B | 8.19B | 6.47 GB | 16.87 GB | Fits Q4_K_M | ~50 / 806 tok/s | Calculate → |
| Qwen3-4B | 4.02B | 3.61 GB | 8.68 GB | FP16 & Q4 | ~101.9 / 1643 tok/s | Calculate → |
| Qwen2.5-72B-Instruct | 72.71B | 44.38 GB | 138.43 GB | 显存不足 | ~1.7 / 91 tok/s | Calculate → |
| Qwen2.5-Coder-32B-Instruct | 32.76B | 21.23 GB | 63.51 GB | 显存不足 | ~5.2 / 202 tok/s | Calculate → |
| Qwen2.5-14B-Instruct | 14.77B | 10.61 GB | 29.5 GB | Fits Q4_K_M | ~27.7 / 447 tok/s | Calculate → |
| Qwen2.5-7B-Instruct | 7.62B | 5.43 GB | 15.12 GB | Fits Q4_K_M | ~53.8 / 867 tok/s | Calculate → |
| QwQ-32B Reasoning | 32.76B | 21.23 GB | 63.51 GB | 显存不足 | ~5.2 / 202 tok/s | Calculate → |
| Llama-4-Maverick-17B-128E (400B MoE) | 400B | 227.32 GB | 747.05 GB | 显存不足 | ~5.9 / 384 tok/s | Calculate → |
| Llama-4-Scout-17B-16E (109B MoE) | 109B | 63.7 GB | 205.02 GB | 显存不足 | ~6.6 / 384 tok/s | Calculate → |
| Llama-3.3-70B-Instruct | 70.55B | 43.09 GB | 134.41 GB | 显存不足 | ~1.8 / 94 tok/s | Calculate → |
| Llama-3.1-405B-Instruct | 405B | 233.02 GB | 758.82 GB | 显存不足 | ~1.2 / 45 tok/s | Calculate → |
| Llama-3.1-70B-Instruct | 70.55B | 43.09 GB | 134.41 GB | 显存不足 | ~1.8 / 94 tok/s | Calculate → |
| Llama-3.1-8B-Instruct | 8.03B | 6.22 GB | 16.45 GB | Fits Q4_K_M | ~51 / 822 tok/s | Calculate → |
| Llama-3.2-3B-Instruct | 3.21B | 3.33 GB | 7.34 GB | FP16 & Q4 | ~127.6 / 2057 tok/s | Calculate → |
| Mistral-Large-2-123B | 122.6B | 72.84 GB | 231.62 GB | 显存不足 | ~1.2 / 54 tok/s | Calculate → |
| Mistral-Small-3.1-24B-Instruct | 23.6B | 15.28 GB | 45.7 GB | 显存不足 | ~10.3 / 280 tok/s | Calculate → |
| Mixtral-8x22B-Instruct (MoE) | 140.6B | 81.38 GB | 264.13 GB | 显存不足 | ~2.8 / 169 tok/s | Calculate → |
| Mixtral-8x7B-Instruct (MoE) | 46.7B | 27.8 GB | 88.47 GB | 显存不足 | ~11.3 / 512 tok/s | Calculate → |
| Codestral-22B | 22.2B | 14.81 GB | 43.59 GB | 显存不足 | ~11.5 / 297 tok/s | Calculate → |
| Google Gemma-3-27B-IT | 27.4B | 20.34 GB | 55.4 GB | 显存不足 | ~6.5 / 241 tok/s | Calculate → |
| Google Gemma-3-12B-IT | 12.2B | 10.75 GB | 26.21 GB | Fits Q4_K_M | ~33.6 / 541 tok/s | Calculate → |
| Google Gemma-3-4B-IT | 4.3B | 4.24 GB | 9.56 GB | FP16 & Q4 | ~95.3 / 1536 tok/s | Calculate → |
| Microsoft Phi-4 14B | 14.66B | 10.5 GB | 29.36 GB | Fits Q4_K_M | ~28 / 450 tok/s | Calculate → |
| Cohere Command-A 111B | 111B | 66.19 GB | 209.26 GB | 显存不足 | ~1.2 / 59 tok/s | Calculate → |
| GLM-4-32B-0414 | 32B | 20.77 GB | 62 GB | 显存不足 | ~5.5 / 206 tok/s | Calculate → |