Use a Hugging Face Inference Endpoint as a Custom Model Provider in Pi Agent
Melvin Vivas · X post · 2026-10-06 · Open on X
Topics: LLMOps, Deployment & Monitoring, AI Agents, Tool Use & MCP, AI Dev Tools & Productivity · Level: intermediate
Summary
Shows how to host an open model (a GGUF build of Qwen3.8-27B) on Hugging Face Inference Endpoints and add it to the Pi coding agent as a provider. The endpoint offers an OpenAI-compatible API. You point Pi's models.json at the endpoint's /v1 URL, then the hosted model works like any other provider.
Key points
- Deploy an open model on Hugging Face Inference Endpoints (endpoints.huggingface.co); the model used here is Qwen3.8-27B-UD-Q4_K_M.
- In Pi's models.json, add a 'huggingface' provider with baseUrl https://<instance-id>.endpoints.huggingface.cloud/v1 and api 'openai-completions'.
- Read the API key from an environment variable: apiKey '$HF_ENDPOINT_API_KEY'.
- Compatibility flags: supportsDeveloperRole false, supportsReasoningEffort false, maxTokensField 'max_tokens'.
- Model entry: id 'unsloth/Qwen3.8-27B-GGUF', reasoning false, text input, contextWindow 32768, maxTokens 16384.
- Replace <instance-id> with your endpoint ID and set HF_ENDPOINT_API_KEY before running Pi.
Resources mentioned
- Hugging Face Inference Endpoints · tool · endpoints.huggingface.co · paid
Paid managed service for deploying models from the Hugging Face Hub to dedicated, autoscaling infrastructure.
Also in: Qwen3.8-27B on HF Inference Endpoints: Context Size Matters for Agent Builds (Melvin Vivas on X · notes), Pi Agent on a Self-Deployed Qwen 27B Q4_K_M via Hugging Face Inference Endpoints (Melvin Vivas on X · notes), Deploy Cheap Quantized GGUF Models on HF Inference Endpoints with llama.cpp (Melvin Vivas on X · notes), Deploy Open-Source Models with Hugging Face Inference Endpoints (Melvin Vivas on X · notes) and 2 more - Hugging Face · website · huggingface.co · free · recommended by both Bashiri Smith & Melvin Vivas
Named in the prompt as a beginner-friendly but powerful AI platform for models used in AI projects.
Also in: llama.app: Run Local Models on a Mac Without the llama.cpp CLI (Melvin Vivas on X · notes), Grok Bot workflow: turn model-release posts into Colab notebooks (Melvin Vivas on X · notes), Generate Portfolio AI Project Ideas with Claude and Hugging Face Tasks (Bashiri Smith on Facebook · notes), EmbeddingGemma 2 released on Hugging Face (Melvin Vivas on X · notes) and 48 more - Pi · tool · x.com · free
A customizable coding agent that can be extended through its Extensions API and connected to several model providers.
Also in: 100-Iteration Autonomous Coding Loop with Local Qwen on an RTX 3090 (Melvin Vivas on X · notes), Pi Agent: A Highly Customizable Agent with Durable Sessions (Melvin Vivas on X · notes), Prompting an Autonomous Agent Loop: Be Specific, Show Visible Progress (Melvin Vivas on X · notes), Running an Overnight 100-Iteration Agent Loop with a Local Qwen Model (Melvin Vivas on X · notes) and 54 more - Qwen3.8-27B-Q4_K_M (Unsloth) · tool · huggingface.co · free
A 4-bit (Q4_K_M) quantized version of the 27B Qwen model from Unsloth, small enough to run on one consumer GPU.
Also in: 100-Iteration Autonomous Coding Loop with Local Qwen on an RTX 3090 (Melvin Vivas on X · notes), Prompting an Autonomous Agent Loop: Be Specific, Show Visible Progress (Melvin Vivas on X · notes), Running an Overnight 100-Iteration Agent Loop with a Local Qwen Model (Melvin Vivas on X · notes), Pi Agent on a Self-Deployed Qwen 27B Q4_K_M via Hugging Face Inference Endpoints (Melvin Vivas on X · notes) and 5 more
Try this
- Deploy an open model on Hugging Face Inference Endpoints.
- Add the provider block to Pi's models.json, replacing <instance-id> with your endpoint ID.
- Set the HF_ENDPOINT_API_KEY environment variable.
- Run a self-hosted open model as the backend for a coding agent instead of a proprietary API.
More in LLMOps, Deployment & Monitoring
- Local AI vs Cloud AI: When Running Models Locally Makes Sense
- Local AI vs Cloud AI: Using the Right Model for the Right Job
- Monthly Claude Platform API credits for Max and Team plans
- Qwen3.8-27B on HF Inference Endpoints: Context Size Matters for Agent Builds
- Pi Agent on a Self-Deployed Qwen 27B Q4_K_M via Hugging Face Inference Endpoints
- Deploy Cheap Quantized GGUF Models on HF Inference Endpoints with llama.cpp