Model Routing Fine-Tuned on Your Agent Harness Traces

Melvin Vivas · X post · 2026-08-27 · Open on X

Topics: Fine-tuning & Model Customization, LLMOps, Deployment & Monitoring, AI Agents, Tool Use & MCP · Level: advanced

Summary

An example of how model routing can work: fine-tune Liquid AI's LFM2.5 Encoder on traces from your agent harness, which record each prompt and the model used for it. The router then learns which model you would normally pick for a given prompt. This assumes you already choose different models for different tasks.

Key points

• Log your agent harness traces as (prompt, model chosen) pairs.
• Fine-tune the small LFM2.5 Encoder on those traces to predict the right model.
• The router then copies your own past model choices for each kind of prompt.
• This only works if you already use different models for different tasks.

Resources mentioned

• LFM2.5-Encoder-350M · tool · huggingface.co · free
  A 350M-parameter encoder model from Liquid AI that does zero-shot prompt classification and routing against categories you supply at runtime.
  Also in: Zero-Shot Prompt Routing with Liquid AI's LFM 2.5-Encoder-350M (Melvin Vivas on X · notes), AIBackends v0.7.0: Prompt Routing with Liquid AI's LFM2.5 Encoder (Melvin Vivas on X · notes), Zero-Shot Prompt Routing with Liquid AI's LFM 2.5-Encoder-350M (Melvin Vivas on X · notes), Zero-Shot Prompt Routing by Task Complexity with LFM2.5-Encoder (Melvin Vivas on X · notes)

Try this

• Collect prompt and model-choice traces from your agent harness.
• Fine-tune LFM2.5 Encoder on your own agent traces to build a personalized model router.