Train ModernBERT as a Prompt Router Between Two Models (Colab Notebook)

Melvin Vivas · X post · 2026-09-18 · Open on X

Topics: Fine-tuning & Model Customization, LLMOps, Deployment & Monitoring, AI System Design & Architecture · Level: intermediate

Summary

The creator shares a Colab notebook that fine-tunes a ModernBERT classifier on prompts. The classifier decides whether a task should go to a cheaper model (Luna) or a stronger one (Astra). It shows that a small encoder model can do model routing without calling an LLM. He notes that real use needs a much larger and more diverse dataset.

Key points

• Model routing: a small classifier predicts which LLM should handle each prompt.
• Fine-tune ModernBERT for sequence classification with two labels: Luna and Astra.
• Training data is prompts labeled with the model that should handle them.
• The example is small. Production routing needs a much larger and more diverse dataset to give reliable predictions.
• The idea was inspired by Jev: you don't need an LLM for every task.
• The notebook opens directly in Google Colab.

Resources mentioned

• donvito/notebooks · repo · github.com · free
  The creator's notebooks for fine-tuning and running local models, which you can run in Google Colab, including a GLiNER2.5-Decide intent classification example.
  Also in: Run Local Models on a Free GPU with Google Colab (T4) (Melvin Vivas on X · notes), Free Colab Notebooks for Local Model Fine-Tuning and Inference (Melvin Vivas on X · notes), Intent classification for support using GLiNER2.5-Decide notebook (Melvin Vivas on X · notes), Getting started with GLiNER2.5-Decide in a Colab notebook (Melvin Vivas on X · notes) and 3 more
• ModernBERT-base · tool · huggingface.co · free
  An open-source modernized BERT encoder model. The quoted post uses it as the speed baseline for LFM2.5-Encoder.
  Also in: Train Your Own LLM Model Router by Fine-Tuning ModernBERT as a Classifier (Melvin Vivas on X · notes), Free Colab Notebooks: ModernBERT, Gemma QLoRA, GLiNER & Guardrails (Melvin Vivas on X · notes), Fine-Tune ModernBERT-base as a Task-Routing Classifier (Melvin Vivas on X · notes), Fine-tuning ModernBERT-base as a task router (quote post) (Melvin Vivas on X · notes) and 2 more
• Google Colab · tool · colab.research.google.com · free · recommended by both Bashiri Smith & Melvin Vivas
  Free GPU notebooks.
  Also in: Run Notebooks on a Free GPU with Google Colab (T4, 15GB VRAM) (Melvin Vivas on X · notes), Run Local Models on a Free GPU with Google Colab (T4) (Melvin Vivas on X · notes), Free Colab Notebooks for Local Model Fine-Tuning and Inference (Melvin Vivas on X · notes), Intent classification for support using GLiNER2.5-Decide notebook (Melvin Vivas on X · notes) and 12 more

Try this

• Open the notebook in Colab and run the ModernBERT router training.
• Build a larger and more diverse labeled prompt dataset before using a router like this for real.
• Build a cost-saving LLM router: fine-tune ModernBERT to send each prompt to a cheap or a strong model, then measure cost savings and quality.