Fine-Tune ModernBERT-base as a Task-Routing Classifier

Melvin Vivas · X post · 2026-09-19 · Open on X

Topics: Fine-tuning & Model Customization, AI Agents, Tool Use & MCP · Level: intermediate

Summary

The creator is experimenting with fine-tuning ModernBERT-base to classify incoming tasks and send each one to one of two agents (Astra or Luna). The point: a small encoder classifier you train yourself can replace an LLM call for routing.

Key points

• Model: ModernBERT-base, an encoder model that suits classification.
• Goal: classify each task and route it to one of two agents/workers.
• A small fine-tuned classifier is cheaper and faster than asking an LLM to do the routing.
• Message: train your own classification model for routing decisions.

Resources mentioned

• ModernBERT-base · tool · huggingface.co · free
  An open-source modernized BERT encoder model. The quoted post uses it as the speed baseline for LFM2.5-Encoder.
  Also in: Train Your Own LLM Model Router by Fine-Tuning ModernBERT as a Classifier (Melvin Vivas on X · notes), Free Colab Notebooks: ModernBERT, Gemma QLoRA, GLiNER & Guardrails (Melvin Vivas on X · notes), Train ModernBERT as a Prompt Router Between Two Models (Colab Notebook) (Melvin Vivas on X · notes), Fine-tuning ModernBERT-base as a task router (quote post) (Melvin Vivas on X · notes) and 2 more

Try this

• Train your own classification model instead of using an LLM for routing.
• Fine-tune ModernBERT-base as a router that sends tasks to different agents.