Local Evals v0.1.0: Run LLM Evals Locally
Melvin Vivas · X post · 2026-10-07 · Open on X
Topics: Evaluation (Evals) & Testing · Level: beginner
Summary
Melvin released v0.1.0 of Local Evals, his open-source eval app. It runs locally and works with local models or any OpenAI-compatible API. This release makes eval runs easier to set up for beginners.
Key points
- Local Evals runs on your own machine.
- It works with local models or any OpenAI-compatible API.
- v0.1.0 makes eval run setup simpler for beginners.
- The repo is public for anyone to try.
Resources mentioned
- donvito/local-evals · repo · github.com · free
Eval app that runs locally and tests local models or any OpenAI-compatible API on JSON extraction and tool calling.
Also in: local-evals: A Local Eval App with a Setup Wizard (Melvin Vivas on X · notes), Melvin Vivas's open-source AI tools, built with Codex (Melvin Vivas on X · notes), local-evals: A Local LLM Eval App Built by Codex Subagents (Melvin Vivas on X · notes), Measure Your Own Codex Token Burn: Orchestrator+Subagents vs Single Agent (Melvin Vivas on X · notes) and 1 more
Try this
- Clone the local-evals repo and try an eval run against a local model or an OpenAI-compatible API.
- Build a small eval suite for your own prompts with Local Evals and compare several local models.
More in Evaluation (Evals) & Testing
- Why Evals Matter: Making AI Apps Survive More Than a Demo
- MLflow 3.17.0: Jev models as LLM judges in GenAI evaluate
- Decision Model Rankings by Spend and Token Share
- local-evals: A Local Eval App with a Setup Wizard
- OpenAI Retires Its Evals Platform and Recommends promptfoo
- Benchmarks matter less than how a model does on your own task