What LLMs Can Run on NVIDIA RTX 4060 Ti 16GB (16GB VRAM)?
NVIDIA RTX 4060 Ti 16GB features 16 GB VRAM and 288 GB/s memory bandwidth (Consumer GPU). Below is the exact compatibility and token/sec speed table across all 36 open-source LLMs.
| LLM Model | Params | Q4_K_M VRAM (8K) | FP16 VRAM (8K) | Status on NVIDIA RTX 4060 Ti 16GB | Est. Speed | Exact Config |
|---|---|---|---|---|---|---|
| DeepSeek-R1 671B (MoE) | 671B | 420.2 GB | 1383.46 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| DeepSeek-V3 671B (MoE) | 671B | 420.2 GB | 1383.46 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| DeepSeek-R1-Distill-Llama-70B | 70.6B | 47.25 GB | 148.6 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| DeepSeek-R1-Distill-Qwen-32B | 32.8B | 23.14 GB | 70.23 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| DeepSeek-R1-Distill-Qwen-14B | 14.8B | 11.4 GB | 32.65 GB | Fits 4-bit Q4_K_M | ~19.9 tok/s | Calculate → |
| DeepSeek-R1-Distill-Llama-8B | 8.03B | 6.67 GB | 18.2 GB | Fits 4-bit Q4_K_M | ~36.7 tok/s | Calculate → |
| DeepSeek-R1-Distill-Qwen-7B | 7.62B | 5.85 GB | 16.79 GB | Fits 4-bit Q4_K_M | ~38.7 tok/s | Calculate → |
| Qwen3-235B-A22B (MoE) | 235B | 148.87 GB | 486.23 GB | Exceeds 16GB | ~1.7 tok/s | Calculate → |
| Qwen3-30B-A3B (MoE) | 30.5B | 20.45 GB | 64.23 GB | Exceeds 16GB | ~11.5 tok/s | Calculate → |
| Qwen3-32B | 32.8B | 23.14 GB | 70.23 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Qwen3-14B | 14.8B | 11.15 GB | 32.4 GB | Fits 4-bit Q4_K_M | ~19.9 tok/s | Calculate → |
| Qwen3-8B | 8.2B | 6.9 GB | 18.67 GB | Fits 4-bit Q4_K_M | ~35.9 tok/s | Calculate → |
| Qwen2.5-72B-Instruct | 72.7B | 48.56 GB | 152.93 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Qwen2.5-Coder-32B-Instruct | 32.8B | 23.14 GB | 70.23 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Qwen2.5-14B-Instruct | 14.8B | 11.4 GB | 32.65 GB | Fits 4-bit Q4_K_M | ~19.9 tok/s | Calculate → |
| Qwen2.5-7B-Instruct | 7.62B | 5.85 GB | 16.79 GB | Fits 4-bit Q4_K_M | ~38.7 tok/s | Calculate → |
| QwQ-32B Reasoning | 32.8B | 23.14 GB | 70.23 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Llama-4-Maverick-17B-128E (400B MoE) | 400B | 251.93 GB | 826.16 GB | Exceeds 16GB | ~2.2 tok/s | Calculate → |
| Llama-4-Scout-17B-16E (109B MoE) | 109B | 70.22 GB | 226.7 GB | Exceeds 16GB | ~2.2 tok/s | Calculate → |
| Llama-3.3-70B-Instruct | 70.6B | 47.25 GB | 148.6 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Llama-3.1-405B-Instruct | 405B | 257.51 GB | 838.92 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Llama-3.1-70B-Instruct | 70.6B | 47.25 GB | 148.6 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Llama-3.1-8B-Instruct | 8.03B | 6.67 GB | 18.2 GB | Fits 4-bit Q4_K_M | ~36.7 tok/s | Calculate → |
| Llama-3.2-3B-Instruct | 3.21B | 3.54 GB | 8.14 GB | Fits FP16 & Q4 | ~91.8 tok/s | Calculate → |
| Mistral-Large-2-123B | 123B | 80.23 GB | 256.8 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Mistral-Small-3.1-24B-Instruct | 24B | 16.9 GB | 51.35 GB | Exceeds 16GB | ~1.6 tok/s | Calculate → |
| Mixtral-8x22B-Instruct (MoE) | 141B | 90.46 GB | 292.87 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Mixtral-8x7B-Instruct (MoE) | 46.7B | 30.82 GB | 97.86 GB | Exceeds 16GB | ~2.9 tok/s | Calculate → |
| Codestral-22B | 22.2B | 16.27 GB | 48.14 GB | Exceeds 16GB | ~1.7 tok/s | Calculate → |
| Google Gemma-3-27B-IT | 27.4B | 21.64 GB | 60.98 GB | Exceeds 16GB | ~1.4 tok/s | Calculate → |
| Google Gemma-3-12B-IT | 12.2B | 11.28 GB | 28.79 GB | Fits 4-bit Q4_K_M | ~24.1 tok/s | Calculate → |
| Google Gemma-3-4B-IT | 4.3B | 4.4 GB | 10.58 GB | Fits FP16 & Q4 | ~68.5 tok/s | Calculate → |
| Microsoft Phi-4 14B | 14.7B | 11.4 GB | 32.5 GB | Fits 4-bit Q4_K_M | ~20 tok/s | Calculate → |
| Cohere Command-A 111B | 111B | 71.99 GB | 231.33 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| Yi-1.5-34B-Chat | 34.4B | 24.02 GB | 73.4 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |
| GLM-4-32B-0414 | 32B | 22.55 GB | 68.49 GB | Exceeds 16GB | ~1.2 tok/s | Calculate → |