Run Muse Glimmer 30B Locally on an RTX 3090 with llama.cpp
Melvin Vivas · X post · 2026-08-13 · Open on X
Topics: LLMOps, Deployment & Monitoring, AI Agents, Tool Use & MCP · Level: intermediate
Summary
Shares a guide to running Muse Glimmer 30B on one RTX 3090 with llama.cpp under WSL Ubuntu on Windows 11. It uses Unsloth's Q4_K_XL GGUF quantization and needs about 23GB of VRAM at full context. The guide's author tested it with the Hermes agent and expects it to work with OpenClaw and Pi.
Key points
- Setup: Windows 11 Pro, WSL Ubuntu 24.04.1, RTX 3090 (24GB).
- Model: unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL.
- VRAM usage is about 23GB at full context, so it just fits on a 24GB card.
- Serve the model with llama.cpp and connect it to an agent harness (tested with Hermes agent; OpenClaw and Pi untested).
Resources mentioned
- llama.cpp · repo · github.com · free
An open-source C/C++ engine for running GGUF models locally. Its llama-server command provides an OpenAI-compatible HTTP server.
Also in: Run Pi with local models on an RTX Spark laptop via llama.cpp (Melvin Vivas on X · notes), llama.app: Run Local Models on a Mac Without the llama.cpp CLI (Melvin Vivas on X · notes), Deploy Cheap Quantized GGUF Models on HF Inference Endpoints with llama.cpp (Melvin Vivas on X · notes), Run Local LLMs on the Computer You Already Have: A Beginner's Path (Melvin Vivas on X · notes) and 33 more - unsloth/Muse-Glimmer-30B-GGUF · tool · huggingface.co · free
Unsloth's GGUF quantizations of Meta's Muse Glimmer 30B open-weights model, used here with the UD-Q4_K_XL quant.
Also in: Three Local GGUF Models That Fit on an RTX 3090 (24GB) (Melvin Vivas on X · notes), Running Muse Glimmer 30B Locally with llama.cpp and the Hermes Agent (Melvin Vivas on X · notes), Run Muse Glimmer 30B Locally with llama.cpp and Connect It to Hermes Agent (Melvin Vivas on X · notes) - Hermes Agent · tool · github.com · free
Nous Research's open-source AI agent with CLI, TUI and desktop interfaces. It now supports hands-free activation with a wake word.
Also in: jsonl-viewer: Viewing Live Agent Traces from Codex, Claude Code, Pi & Hermes (Melvin Vivas on X · notes), Inspecting Coding-Agent Traces Live with JSONL Viewer (Codex, Claude Code) (Melvin Vivas on X · notes), Hermes Agent: each bot is its own profile (Melvin Vivas on X · notes), An X research bot built with Hermes (Melvin Vivas on X · notes) and 56 more - OpenClaw · tool · github.com · free · recommended by both Bashiri Smith & Melvin Vivas
Popular open-source personal AI agent project that showed strong consumer demand for agents.
Also in: Grok Bot vs OpenAI in the consumer AI agent market (Melvin Vivas on X · notes), Why Simpler Agent Bots Appeal to Non-Technical Users (Melvin Vivas on X · notes), Coworker: A Desktop AI Assistant App, Not a Coding Agent (Melvin Vivas on X · notes), Grok Bot's Real Competitors: OpenClaw and Hermes (Melvin Vivas on X · notes) and 9 more - Pi · tool · x.com · free
A customizable coding agent that can be extended through its Extensions API and connected to several model providers.
Also in: 100-Iteration Autonomous Coding Loop with Local Qwen on an RTX 3090 (Melvin Vivas on X · notes), Pi Agent: A Highly Customizable Agent with Durable Sessions (Melvin Vivas on X · notes), Prompting an Autonomous Agent Loop: Be Specific, Show Visible Progress (Melvin Vivas on X · notes), Running an Overnight 100-Iteration Agent Loop with a Local Qwen Model (Melvin Vivas on X · notes) and 54 more
Try this
- Install llama.cpp in WSL Ubuntu and load the UD-Q4_K_XL GGUF on a 24GB GPU.
- Point an agent harness (e.g., Hermes agent) at the local llama.cpp server.
- Run a fully local coding agent on Muse Glimmer 30B with a single consumer GPU.
More in LLMOps, Deployment & Monitoring
- Running LFM2.5-2.6B Q4_K_M Locally with llama.cpp and Pi
- Run Qwen3.8-27B locally with llama.cpp (llama-server)
- Unsloth releases Qwen3.8-27B GGUF quantizations
- Running Muse Glimmer 30B Locally with llama.cpp and the Hermes Agent
- Run Muse Glimmer 30B Locally with llama.cpp and Connect It to Hermes Agent
- Running Liquid AI LFM2.5-2.6B Locally with llama-server