Imatrix

[Andre-Leubke-97]

Hi Qwen Team,

First of all, thank you for your incredible contributions to the open-source AI community. The reasoning and coding capabilities across the Qwen lineup are truly outstanding, and the new 3.x / 3.6 generation models are a massive leap forward.

To run these heavy Dense models efficiently on local consumer hardware (CPU/RAM) via llama.cpp, many users rely on quantizations. While IQ-quants preserve great logic, they introduce a heavy CPU speed penalty (10-20% inference slowdown) due to complex vector-codebook decoding.

The perfect "performance sweet spot" for local execution is using standard Q-quants (like Q3_K_M or Q4_K_M) combined with a high-quality Importance Matrix (imatrix). This keeps the CPU speed high while protecting the crucial attention layers and reasoning capabilities of the model.

Could you please consider uploading and providing the official .imatrix (or .dat) calibration files for all of your Dense models, especially for the newly released 3.x / 3.6 series?

Since these matrices are generated on your side using your original, high-quality training datasets, they would allow the community to build flawless, highly optimized local GGUF quants. This process protects the core intelligence of the models perfectly without compromising your actual training data privacy at all.

Having these official imatrix files available directly from the manufacturer would be an absolute game-changer for the local LLM and hardware-enthusiast community.

Thank you for your time and continued support!