Liquid AI Releases QAD Q4_0 Checkpoints for Faster, Higher-Quality Edge Inference

Liquid AI has released four QAD Q4_0 GGUF checkpoints for its LFM2.5 family: the 230M, 350M, 1.2B-Instruct, and 2.6B models. The files use quantization-aware distillation, training a lower-precision model against a higher-precision teacher rather than applying quantization only after training.

The goal is to make 4-bit deployment less costly in quality. Liquid AI reports that the checkpoints recover roughly 97% of the average BF16 performance otherwise lost through quantization, while retaining the memory footprint and throughput expected from native Q4_0 models. Its evaluations covered reasoning, instruction following, tool use, and agent-style tasks, alongside size-appropriate math benchmarks.

Why builders should care

For developers targeting phones, Raspberry Pi systems, and other constrained devices, the release offers a way to run small language models locally without automatically moving to a larger or less efficient quantization format. In Liquid AI’s hardware tests, the 230M and 350M versions reportedly reached Q5_K_M-level quality while decoding 4–33% faster. The 1.2B and 2.6B versions matched Q4_K_M quality with throughput gains of 3–14%.

The checkpoints are available through Hugging Face and can be used with llama.cpp or other runtimes that support GGUF Q4_0 files. Developers can download the model-specific artifacts and test them directly with llama-cli, making the release particularly relevant for offline assistants, embedded applications, and edge AI prototypes.

Source: Hugging Face Blog