Running Muse Glimmer 30B Locally with llama.cpp and the Hermes Agent
Melvin Vivas · X post · 2026-08-11 · Open on X
Topics: LLMOps, Deployment & Monitoring, AI Agents, Tool Use & MCP · Level: intermediate
Summary
Melvin Vivas tests Meta's Muse Glimmer 30B open model with the Hermes agent. He runs it locally through llama.cpp using Unsloth's 4-bit GGUF quant. The quoted guide lists his hardware and setup, so you can judge whether your own machine can run it.
Key points
- Model: unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL (Unsloth dynamic 4-bit quant).
- Served with llama.cpp and used as the backend for the Hermes agent.
- Setup: WSL Ubuntu 24.04.1 on Windows 11 Pro with an RTX 3090.
- VRAM use is about 23 GB at full context, so it fits on a single 24 GB card.
- He says it should also work with OpenClaw and Pi, but he hasn't tested those.
Resources mentioned
- unsloth/Muse-Glimmer-30B-GGUF · tool · huggingface.co · free
Unsloth's GGUF quantizations of Meta's Muse Glimmer 30B open-weights model, used here with the UD-Q4_K_XL quant.
Also in: Three Local GGUF Models That Fit on an RTX 3090 (24GB) (Melvin Vivas on X · notes), Run Muse Glimmer 30B Locally on an RTX 3090 with llama.cpp (Melvin Vivas on X · notes), Run Muse Glimmer 30B Locally with llama.cpp and Connect It to Hermes Agent (Melvin Vivas on X · notes) - llama.cpp · repo · github.com · free
An open-source C/C++ engine for running GGUF models locally. Its llama-server command provides an OpenAI-compatible HTTP server.
Also in: Run Pi with local models on an RTX Spark laptop via llama.cpp (Melvin Vivas on X · notes), llama.app: Run Local Models on a Mac Without the llama.cpp CLI (Melvin Vivas on X · notes), Deploy Cheap Quantized GGUF Models on HF Inference Endpoints with llama.cpp (Melvin Vivas on X · notes), Run Local LLMs on the Computer You Already Have: A Beginner's Path (Melvin Vivas on X · notes) and 33 more - Hermes Agent · tool · github.com · free
Nous Research's open-source AI agent with CLI, TUI and desktop interfaces. It now supports hands-free activation with a wake word.
Also in: jsonl-viewer: Viewing Live Agent Traces from Codex, Claude Code, Pi & Hermes (Melvin Vivas on X · notes), Inspecting Coding-Agent Traces Live with JSONL Viewer (Codex, Claude Code) (Melvin Vivas on X · notes), Hermes Agent: each bot is its own profile (Melvin Vivas on X · notes), An X research bot built with Hermes (Melvin Vivas on X · notes) and 56 more - OpenClaw · tool · github.com · free · recommended by both Bashiri Smith & Melvin Vivas
Popular open-source personal AI agent project that showed strong consumer demand for agents.
Also in: Grok Bot vs OpenAI in the consumer AI agent market (Melvin Vivas on X · notes), Why Simpler Agent Bots Appeal to Non-Technical Users (Melvin Vivas on X · notes), Coworker: A Desktop AI Assistant App, Not a Coding Agent (Melvin Vivas on X · notes), Grok Bot's Real Competitors: OpenClaw and Hermes (Melvin Vivas on X · notes) and 9 more - Pi · tool · x.com · free
A customizable coding agent that can be extended through its Extensions API and connected to several model providers.
Also in: 100-Iteration Autonomous Coding Loop with Local Qwen on an RTX 3090 (Melvin Vivas on X · notes), Pi Agent: A Highly Customizable Agent with Durable Sessions (Melvin Vivas on X · notes), Prompting an Autonomous Agent Loop: Be Specific, Show Visible Progress (Melvin Vivas on X · notes), Running an Overnight 100-Iteration Agent Loop with a Local Qwen Model (Melvin Vivas on X · notes) and 54 more
Try this
- Follow the guide to serve Muse Glimmer 30B (UD-Q4_K_XL) with llama.cpp and connect it to an agent such as Hermes.
- Build a fully local agent stack: llama.cpp serving Muse Glimmer 30B on a 24 GB GPU, driven by Hermes, OpenClaw or Pi.
More in LLMOps, Deployment & Monitoring
- Run Qwen3.8-27B locally with llama.cpp (llama-server)
- Unsloth releases Qwen3.8-27B GGUF quantizations
- Run Muse Glimmer 30B Locally on an RTX 3090 with llama.cpp
- Run Muse Glimmer 30B Locally with llama.cpp and Connect It to Hermes Agent
- Running Liquid AI LFM2.5-2.6B Locally with llama-server
- Adding LFM2.5-2.6B support to AIBackends with Cursor