local-evals: A Local Eval App with a Setup Wizard
Melvin Vivas · X post · 2026-10-07 · Open on X
Topics: Evaluation (Evals) & Testing, AI Dev Tools & Productivity · Level: intermediate
Summary
Melvin shares UX updates to local-evals, his open-source eval app. It runs locally and works with local models or any OpenAI-compatible API. A new wizard walks you through setting up eval runs.
Key points
- local-evals runs entirely on your machine.
- It works with local models or any OpenAI-compatible API.
- A new wizard makes setting up eval runs easier.
Resources mentioned
- donvito/local-evals · repo · github.com · free
Eval app that runs locally and tests local models or any OpenAI-compatible API on JSON extraction and tool calling.
Also in: Local Evals v0.1.0: Run LLM Evals Locally (Melvin Vivas on X · notes), Melvin Vivas's open-source AI tools, built with Codex (Melvin Vivas on X · notes), local-evals: A Local LLM Eval App Built by Codex Subagents (Melvin Vivas on X · notes), Measure Your Own Codex Token Burn: Orchestrator+Subagents vs Single Agent (Melvin Vivas on X · notes) and 1 more
Try this
- Try local-evals and use the wizard to set up an eval run against a local or OpenAI-compatible model.
More in Evaluation (Evals) & Testing
- MLflow 3.17.0: Jev models as LLM judges in GenAI evaluate
- Decision Model Rankings by Spend and Token Share
- Local Evals v0.1.0: Run LLM Evals Locally
- OpenAI Retires Its Evals Platform and Recommends promptfoo
- Benchmarks matter less than how a model does on your own task
- 5 Things AI Engineers Must Evaluate: RAG, Agents, Models, Data, Guardrails