Running AI Locally: Rebuilding AIBackends as a Python Library with Open Models
Melvin Vivas · X video post · 2026-04-26 · 0:07 · 485 views · Open on X
Topics: LLMOps, Deployment & Monitoring, AI Safety, Security & Guardrails, AI Dev Tools & Productivity · Level: intermediate
Summary
Melvin Vivas shows a short demo of AIBackends, his project for running AI tasks locally. He rebuilt it as a Python library over a weekend, using Cursor with GPT 5.4/5.5 as coding models. It runs open models such as Google DeepMind's and Alibaba's Qwen on your own machine. One of its features is redacting personal data (PII) with NVIDIA's gliner-PII model. The caption is cut off after "Image", so the rest of the feature list is unknown.
Key points
- Open models from Google DeepMind (likely Gemma) and Alibaba Qwen are now good enough to run AI locally, without cloud APIs.
- AIBackends was rebuilt as a Python library over a single weekend using an AI coding assistant (Cursor) with GPT 5.4/5.5.
- PII redaction is handled locally by gliner-PII, NVIDIA's model for finding personal data, so sensitive text never leaves your machine.
- Wrapping several local AI tasks (PII redaction, image tasks and others) in one Python library makes them reusable inside your own apps.
- The caption is cut off after 'Image', so the rest of the feature list is unknown. The 7-second video has no speech.
Resources mentioned
- AIBackends · repo · aibackends.com · free
Open-source API server runtime for common AI use cases that supports many models and providers (Ollama, LM Studio, OpenRouter, OpenAI, Anthropic).
Also in: Redacting PII Locally with GLiNER-PII Before AI Processing (Melvin Vivas on X · notes), Building a production website with Opus 5.5 and TanStack (Melvin Vivas on X · notes), CamelFlow: open-source visual viewer for Apache Camel routes (Melvin Vivas on X · notes), AI Backends: A Production AI Workflow Engineering Site (Link Share) (Melvin Vivas on X · notes) and 17 more - GLiNER-PII · tool · huggingface.co · free
NVIDIA's GLiNER-based model that finds personal data (PII) in text, so it can be redacted locally.
Also in: Redacting PII Locally with GLiNER-PII Before AI Processing (Melvin Vivas on X · notes), aibackends 0.3.0: Model Caching Speeds Up PII and OCR Inference (Melvin Vivas on X · notes), Detect PII and PHI with NVIDIA's GLiNER-PII Model (Melvin Vivas on X · notes) - Gemma 4 · tool · ai.google.dev · free
Google's family of open-weight models in several sizes, built to run on devices and offline, with multimodal and agentic abilities, and open to fine-tuning.
Also in: Meetily: Open-Source Local AI Meeting Notes App (Melvin Vivas on X · notes), New Gemma Developer Documentation Site (Melvin Vivas on X · notes), A100 80GB GPUs in Google Colab via Google AI Pro plans (Melvin Vivas on X · notes), A day of work done from a phone with Grok Bot agents (Melvin Vivas on X · notes) and 32 more - Qwen · tool · github.com · free
Alibaba's family of open-weight language models, including small models suited to fine-tuning and running locally.
Also in: Meetily: Open-Source Local AI Meeting Notes App (Melvin Vivas on X · notes), Local AI vs Cloud AI: When Running Models Locally Makes Sense (Melvin Vivas on X · notes), Local AI vs Cloud AI: Using the Right Model for the Right Job (Melvin Vivas on X · notes), OpenCode Go: Open Coding Models (DeepSeek, Qwen, Kimi, GLM) for $10/mo (Melvin Vivas on X · notes) and 2 more - Cursor · tool · x.com · free · recommended by both Bashiri Smith & Melvin Vivas
AI-native code editor (VS Code with AI built in).
Also in: Finding Cursor Cloud Agents After the Cursor–X Unification (Melvin Vivas on X · notes), Overnight notebook generation with Grok Bot, Cursor cloud and Grok 4.7 (Melvin Vivas on X · notes), Gemini 4 Argon in Cursor: CursorBench Score and Pricing (Melvin Vivas on X · notes), SpaceX AI / Cursor Ambassador Program Waitlist (Melvin Vivas on X · notes) and 135 more - GPT 5.5 · tool · openai.com · paid
OpenAI models the creator used as the coding model inside Cursor.
Also in: Models That Work With the Hermes Agent for Personal Productivity (Melvin Vivas on X · notes), Running GPT-5.5 via Codex as Hermes's Main Model (Melvin Vivas on X · notes), Conductor Walkthrough: Running Parallel Coding Agents in Isolated Git Worktrees (Melvin Vivas on X · notes), Composer 2.5 as the Default Coding Model in Cursor (Melvin Vivas on X · notes) and 2 more - Qwen (@Alibaba_Qwen) on X · person · x.com · free
Official X account of Alibaba's Qwen team, which posts model releases and demos.
Also in: Qwen-Audio-3.1: Alibaba's Five-Model Audio Stack (Melvin Vivas on X · notes), Demo: Qwen3.8-27B Running Locally with Pi and llama.cpp (Melvin Vivas on X · notes), Audio-Visual Vibe Coding with Qwen 3.5 Omni: Spoken Specs to Web App (Melvin Vivas on X · notes) - Google DeepMind (@GoogleDeepMind) on X · person · x.com · free
Google DeepMind's official X account, which posts research and model releases such as Gemma.
Also in: Gemma 4 and Google DeepMind's Open-Source Push (Melvin Vivas on X · notes), Gemma 4 E4B: A Local Agentic Edge Model in 6GB RAM (Melvin Vivas on X · notes) - NVIDIA AI (@NVIDIAAI) on X · person · x.com · free
NVIDIA AI's official X account, which posts model releases and endpoint availability.
Also in: Using Free OpenRouter Models (Nemotron 3.5) With Coworker (Melvin Vivas on X · notes), AI News Roundup: Claude Fable 5, Scientist AI, ZCode, NVIDIA RL (Melvin Vivas on X · notes), Jensen Huang on NVIDIA's Long-Term Commitment to Open Nemotron Models (Melvin Vivas on X · notes), Qwen3.5 Multimodal Model Now on NVIDIA AI Endpoints (Melvin Vivas on X · notes)
Try this
- Open the original X post to read the full AIBackends feature list, since the caption is cut off.
- Try running an open model (Gemma or Qwen) locally.
- Try gliner-PII to redact personal data locally before sending text to any cloud LLM.
- Build a Python library that wraps local AI tasks such as PII redaction and image processing behind one simple API, using open models like Gemma, Qwen and gliner-PII.
- Build a privacy filter that redacts PII locally with gliner-PII before forwarding prompts to a hosted LLM.
More in LLMOps, Deployment & Monitoring
- Self-Host a Coding Model on QuickPod with llama-swap and Use It in Claude Code
- Runpod Flash Reaches GA: Deploy AI Workloads from Python
- Z.ai's Lessons from Serving GLM-5 for Coding Agents at Scale
- Run Gemma 4 Locally with llama.cpp in Two Commands
- LM Studio's LM Link: Use a Local Model on Another Machine