All 36 Supported LLM Architectures
DeepSeek-R1 671B (MoE)
671B params (37B active) · MLA-MoE
4-bit VRAM: 420.2 GB
DeepSeek-V3 671B (MoE)
671B params (37B active) · MLA-MoE
4-bit VRAM: 420.2 GB
DeepSeek-R1-Distill-Llama-70B
70.6B params (70.6B active) · Dense-GQA
4-bit VRAM: 47.25 GB
DeepSeek-R1-Distill-Qwen-32B
32.8B params (32.8B active) · Dense-GQA
4-bit VRAM: 23.14 GB
DeepSeek-R1-Distill-Qwen-14B
14.8B params (14.8B active) · Dense-GQA
4-bit VRAM: 11.4 GB
DeepSeek-R1-Distill-Llama-8B
8.03B params (8.03B active) · Dense-GQA
4-bit VRAM: 6.67 GB
DeepSeek-R1-Distill-Qwen-7B
7.62B params (7.62B active) · Dense-GQA
4-bit VRAM: 5.85 GB
Qwen3-235B-A22B (MoE)
235B params (22B active) · MoE-GQA
4-bit VRAM: 148.87 GB
Qwen3-30B-A3B (MoE)
30.5B params (3.3B active) · MoE-GQA
4-bit VRAM: 20.45 GB
Qwen3-32B
32.8B params (32.8B active) · Dense-GQA
4-bit VRAM: 23.14 GB
Qwen3-14B
14.8B params (14.8B active) · Dense-GQA
4-bit VRAM: 11.15 GB
Qwen3-8B
8.2B params (8.2B active) · Dense-GQA
4-bit VRAM: 6.9 GB
Qwen2.5-72B-Instruct
72.7B params (72.7B active) · Dense-GQA
4-bit VRAM: 48.56 GB
Qwen2.5-Coder-32B-Instruct
32.8B params (32.8B active) · Dense-GQA
4-bit VRAM: 23.14 GB
Qwen2.5-14B-Instruct
14.8B params (14.8B active) · Dense-GQA
4-bit VRAM: 11.4 GB
Qwen2.5-7B-Instruct
7.62B params (7.62B active) · Dense-GQA
4-bit VRAM: 5.85 GB
QwQ-32B Reasoning
32.8B params (32.8B active) · Dense-GQA
4-bit VRAM: 23.14 GB
Llama-4-Maverick-17B-128E (400B MoE)
400B params (17B active) · MoE-GQA
4-bit VRAM: 251.93 GB
Llama-4-Scout-17B-16E (109B MoE)
109B params (17B active) · MoE-GQA
4-bit VRAM: 70.22 GB
Llama-3.3-70B-Instruct
70.6B params (70.6B active) · Dense-GQA
4-bit VRAM: 47.25 GB
Llama-3.1-405B-Instruct
405B params (405B active) · Dense-GQA
4-bit VRAM: 257.51 GB
Llama-3.1-70B-Instruct
70.6B params (70.6B active) · Dense-GQA
4-bit VRAM: 47.25 GB
Llama-3.1-8B-Instruct
8.03B params (8.03B active) · Dense-GQA
4-bit VRAM: 6.67 GB
Llama-3.2-3B-Instruct
3.21B params (3.21B active) · Dense-GQA
4-bit VRAM: 3.54 GB
Mistral-Large-2-123B
123B params (123B active) · Dense-GQA
4-bit VRAM: 80.23 GB
Mistral-Small-3.1-24B-Instruct
24B params (24B active) · Dense-GQA
4-bit VRAM: 16.9 GB
Mixtral-8x22B-Instruct (MoE)
141B params (39B active) · MoE-GQA
4-bit VRAM: 90.46 GB
Mixtral-8x7B-Instruct (MoE)
46.7B params (12.9B active) · MoE-GQA
4-bit VRAM: 30.82 GB
Codestral-22B
22.2B params (22.2B active) · Dense-GQA
4-bit VRAM: 16.27 GB
Google Gemma-3-27B-IT
27.4B params (27.4B active) · Dense-GQA
4-bit VRAM: 21.64 GB
Google Gemma-3-12B-IT
12.2B params (12.2B active) · Dense-GQA
4-bit VRAM: 11.28 GB
Google Gemma-3-4B-IT
4.3B params (4.3B active) · Dense-GQA
4-bit VRAM: 4.4 GB
Microsoft Phi-4 14B
14.7B params (14.7B active) · Dense-GQA
4-bit VRAM: 11.4 GB
Cohere Command-A 111B
111B params (111B active) · Dense-GQA
4-bit VRAM: 71.99 GB
Yi-1.5-34B-Chat
34.4B params (34.4B active) · Dense-GQA
4-bit VRAM: 24.02 GB
GLM-4-32B-0414
32B params (32B active) · Dense-GQA
4-bit VRAM: 22.55 GB