Screening LLM Prompts and Responses on CPU with AIBackends + GliGuard

Melvin Vivas · X post · 2026-08-25 · Open on X

Topics: AI Safety, Security & Guardrails, LLMOps, Deployment & Monitoring · Level: intermediate

Summary

AIBackends v0.5.0, the creator's open-source Python library, adds support for Fastino's GliGuard 300M, a small guardrail model that runs on CPU. You can use it to screen user prompts for safety, toxicity and jailbreaks, and model responses for safety, toxicity and refusals. A Colab notebook shows how it works.

Key points

• AIBackends v0.5.0 adds guardrails through GliGuard 300M by Fastino Labs
• GliGuard is a 300M-parameter model that runs on CPU, so you don't need a GPU for moderation
• Prompt screening checks for safety, toxicity and jailbreak attempts
• Response screening checks for safety, toxicity and refusals
• Install with: pip install aibackends[guardrails]
• A ready-made Colab notebook (gliguard_moderation_colab.ipynb) demonstrates moderation

Resources mentioned

• AIBackends · repo · aibackends.com · free
  Open-source API server runtime for common AI use cases that supports many models and providers (Ollama, LM Studio, OpenRouter, OpenAI, Anthropic).
  Also in: Building a production website with Opus 5.5 and TanStack (Melvin Vivas on X · notes), CamelFlow: open-source visual viewer for Apache Camel routes (Melvin Vivas on X · notes), AI Backends: A Production AI Workflow Engineering Site (Link Share) (Melvin Vivas on X · notes), Demo: Claude Opus 5.5 Generating a Motion-Graphics Video for AIBackends (Melvin Vivas on X · notes) and 16 more
• aibackends 0.5.0 on PyPI · docs · pypi.org · free
  The PyPI package page for aibackends version 0.5.0.
• GliGuard Moderation Colab Notebook · tool · colab.research.google.com · free
  A Google Colab notebook showing GliGuard moderation of prompts and responses with AIBackends.
• GliGuard 300M · tool · huggingface.co · free
  Fastino Labs' small guardrail model that runs on CPU and classifies prompts and responses for safety, toxicity, jailbreaks and refusals.
  Also in: Free Colab Notebooks: ModernBERT, Gemma QLoRA, GLiNER & Guardrails (Melvin Vivas on X · notes)
• Fastino Labs (@fastinoAI) on X · person · x.com · free
  The X account of Fastino Labs, which builds GLiNER decision models and shares demos.
  Also in: GLiDE: Fastino's Decision Model with Adaptive Thinking (Melvin Vivas on X · notes), GLiDE by Fastino Labs: A Post-Trainable Reasoning Decision Model (Melvin Vivas on X · notes), GLiNER decision model demos from Fastino Labs (Melvin Vivas on X · notes), AIBackends v0.8.1 adds GLiNER2.5-Decide local classification (Melvin Vivas on X · notes) and 4 more

Try this

• Install with pip install aibackends[guardrails]
• Try the GliGuard moderation Colab notebook
• Add a CPU-based guardrail step that screens user prompts and model responses in your LLM app