A Production-Grade Replay Test for Swapping LLMs in Conversational Agents

Hugging Face Blog has published a production-focused approach to evaluating replacement language models in conversational agents. The method addresses a practical problem: public benchmarks rarely reflect an agent’s actual prompts, tools, memory, tone, or customer workflows.

The replay engine starts with approved synthetic conversations and reconstructs each interaction under the same operational conditions used in production. Skills, actions, memory, prompts, message history, transfers between agents, and termination rules remain unchanged. The only intended variable is the model making decisions at each turn.

Tool execution is simulated for safety and repeatability. Matching tool calls receive the historical production response; divergent calls receive a failure response instead of a fabricated success. The system also limits each turn to five tool calls, reducing the risk of runaway loops.

What the evaluation found

The team tested eight models, including GPT-4.1 and GPT-5 families, Gemini 2.5, Kimi-K2.5, and GPT-OSS-120B. Three models were approved: GPT-5.4 mini, GPT-5.4 nano, and Kimi-K2.5.

The most important result concerned the verdict system. Three rejected models scored above 79% on the weighted Final Score but failed an independently applied Hallucination Rate threshold. Rechecking results run by run showed that two of those rejections were statistical ties, while one reflected a consistent structural problem.

For teams building AI agents, the lesson is broader than model rankings. A single average can let strong performance offset a serious safety failure. Production evaluation should preserve realistic context, isolate the model swap, and treat critical requirements—such as hallucination limits—as separate pass/fail gates rather than merely weighted score components.

Source: Hugging Face Blog