Unsloth releases Qwen3.8-27B GGUF quantizations
Melvin Vivas · X post · 2026-08-14 · Open on X
Topics: LLMOps, Deployment & Monitoring, Industry Trends & Job Market · Level: beginner
Summary
Announces that Unsloth has published GGUF builds of Qwen3.8-27B, so you can run it locally with llama.cpp-compatible tools. The post thanks Unsloth AI.
Key points
- Unsloth publishes GGUF quantizations of new open models soon after they come out
- GGUF files work with llama.cpp-based local runtimes
Resources mentioned
- Unsloth · tool · unsloth.ai · free
Open-source library for fast, memory-efficient fine-tuning of open LLMs (LoRA/QLoRA) on a single GPU or in Colab.
Also in: 100-Iteration Autonomous Coding Loop with Local Qwen on an RTX 3090 (Melvin Vivas on X · notes), Prompting an Autonomous Agent Loop: Be Specific, Show Visible Progress (Melvin Vivas on X · notes), Running an Overnight 100-Iteration Agent Loop with a Local Qwen Model (Melvin Vivas on X · notes), Agent tasks: Unsloth fine-tuning, ComfyUI face swap, script writing (Melvin Vivas on X · notes) and 36 more - Qwen3.8-27B-Q4_K_M (Unsloth) · tool · huggingface.co · free
A 4-bit (Q4_K_M) quantized version of the 27B Qwen model from Unsloth, small enough to run on one consumer GPU.
Also in: 100-Iteration Autonomous Coding Loop with Local Qwen on an RTX 3090 (Melvin Vivas on X · notes), Prompting an Autonomous Agent Loop: Be Specific, Show Visible Progress (Melvin Vivas on X · notes), Running an Overnight 100-Iteration Agent Loop with a Local Qwen Model (Melvin Vivas on X · notes), Use a Hugging Face Inference Endpoint as a Custom Model Provider in Pi Agent (Melvin Vivas on X · notes) and 5 more
More in LLMOps, Deployment & Monitoring
- AIBackends Adds Support for LFM2.5-VL-3B
- Running LFM2.5-2.6B Q4_K_M Locally with llama.cpp and Pi
- Run Qwen3.8-27B locally with llama.cpp (llama-server)
- Run Muse Glimmer 30B Locally on an RTX 3090 with llama.cpp
- Running Muse Glimmer 30B Locally with llama.cpp and the Hermes Agent
- Run Muse Glimmer 30B Locally with llama.cpp and Connect It to Hermes Agent