llama.cpp

GGUF Quantization Picker

Pick the right GGUF quantization for your hardware. Compare Q4, Q5, Q6, Q8 and more on quality vs. VRAM so you fit the largest model your GPU can hold.