mLateOn Sets New HAKARI-Bench High for Multilingual ColBERT Retrieval

LightOn’s mLateOn is emerging as a strong option for multilingual search and retrieval-augmented generation. Evaluated on HAKARI-Bench, the model achieved a 65.52 Macro score and 63.96 Micro score, placing it first among the 11 ColBERT-style models included in the benchmark.

The result is notable because mLateOn activates about 115 million parameters—with 312 million total—yet outperformed substantially larger late-interaction competitors. Its Overall Macro score was 8.33 points ahead of the second-place model, pplx-embed-v1-late-0.6b, which uses roughly 441 million active parameters.

mLateOn combines a multilingual mmBERT encoder with ColBERT-style token-level matching and supports sequences of up to 8,192 tokens. That longer configured input window helped it handle documents more fully in HAKARI-Bench’s mixed evaluation. On the narrower Multilingual NanoBEIR test, it scored 63.33, landing between Qwen3-Embedding-8B at 62.87 and Nemotron-3-Embed-8B at 64.28. It also remained competitive on English-only retrieval, scoring 68.59 on NanoBEIR-en.

For AI builders, the model points to a practical alternative to single-vector dense embeddings. Late interaction can preserve fine-grained token matches and reuse document representations for rescoring, potentially improving retrieval quality without requiring an 8-billion-parameter encoder. However, the benchmark’s creator notes that HAKARI-Bench scores reflect both matching quality and the input limits configured for each model. Teams should therefore test latency, storage, indexing, and hardware costs on their own workloads before switching architectures.

Source: Hugging Face Blog