Use a Local Model for Confidential Data with Your Agent

Melvin Vivas · X post · 2026-08-06 · Open on X

Topics: AI Safety, Security & Guardrails, AI Agents, Tool Use & MCP · Level: intermediate

Summary

If you give your Hermes agent confidential data such as bank statements, switch it to a local model so the data stays on your machine. The creator runs qwen3.6-35b-a3b-mtp in LM Studio for this.

Key points

• Don't send sensitive documents, such as bank statements, to cloud-hosted LLMs.
• Point your agent (Hermes) at a locally hosted model when it handles confidential tasks.
• Creator's setup: the qwen3.6-35b-a3b-mtp model served by LM Studio.
• A MoE model with about 3B active parameters (A3B) is practical to run locally.

Resources mentioned

• LM Studio · tool · x.com · free
  Desktop app for downloading and running LLMs locally, with a developer mode that serves models through an API.
  Also in: Running LLMs Locally Without an Expensive Rig (Melvin Vivas on X · notes), Adding Vercel AI Gateway as a provider in AIBackends with Devin (Melvin Vivas on X · notes), LoRA Fine-Tune Qwen3.5-2B on Your Tweets with Unsloth Studio (Melvin Vivas on X · notes), AIBackends: An API Layer Between Your App and AI Models (Now with Jev) (Melvin Vivas on X · notes) and 31 more
• Qwen 3.6 35B (MTP) · tool · huggingface.co · free
  Qwen 3.6 mixture-of-experts model (35B total, about 3B active parameters) with multi-token prediction for local inference.
  Also in: Running Qwen 3.6 35B locally on an RTX 3090 for agent tool calling (Melvin Vivas on X · notes), Models That Work With the Hermes Agent for Personal Productivity (Melvin Vivas on X · notes), Run Hermes Agent locally with Qwen 3.6 35B MTP in LM Studio (Melvin Vivas on X · notes), Local Personal AI Agent to Explain Your Investment Portfolio (Melvin Vivas on X · notes) and 4 more
• Hermes Agent · tool · github.com · free
  Nous Research's open-source AI agent with CLI, TUI and desktop interfaces. It now supports hands-free activation with a wake word.
  Also in: Inspecting Coding-Agent Traces Live with JSONL Viewer (Codex, Claude Code) (Melvin Vivas on X · notes), Hermes Agent: each bot is its own profile (Melvin Vivas on X · notes), An X research bot built with Hermes (Melvin Vivas on X · notes), Run Your Hermes Agent on Free LFM2.5-2.6B via OpenRouter (Melvin Vivas on X · notes) and 55 more

Try this

• Switch your agent to a local model before giving it bank statements or other confidential files.