Evaluation for AI Engineering: Free Full Guide (Resource Share)

Bashiri Smith · Facebook reel · 2026-08-28 · 0:08 · 5,720 views · Open on Facebook

Topics: Evaluation (Evals) & Testing · Level: beginner

Summary

This 8-second reel has no speech. It exists to share Bashiri Smith's free full guide on evaluation (evals) for AI engineering, a Google Drive document called "Evaluation field guide". The caption also promotes the creator's BASWE.Ai Engineer community on Skool. The video itself doesn't teach anything about evals, so all the learning is in the linked guide.

Key points

• The video has no spoken content. Everything useful is in the caption links.
• The main resource is the creator's free "full guide on Evaluation for AI engineering", a Google Drive document titled "Evaluation field guide".
• Evaluation (evals) is presented as a core AI engineering topic that deserves a full guide of its own.
• The caption also advertises the creator's Skool community, BASWE.Ai Engineer. It promises a personalized learning roadmap and daily calls with working AI engineers and recruiters.

Resources mentioned

• BASWE.Ai Engineer (Skool community) · community · skool.com · paid
  The creator's paid community and program, with an AI learning roadmap (including the full ops and evaluation track), daily calls with engineers and recruiters, resume and portfolio help, and a job-search pipeline.
  Also in: Basic RAG Pipeline in 60 Seconds: From Documents to Grounded Answers (Bashiri Smith on Facebook · notes), Pointer to Bashiri Smith's Complete AI Engineer Roadmap for 2026 (Bashiri Smith on Facebook · notes), Step-by-Step Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes), How to Evaluate a RAG Pipeline: Retrieval vs. Generation (Interview Answer) (Bashiri Smith on Facebook · notes) and 76 more

From the PDF shared here: Evaluation Field Guide (baswe.ai engineer accelerator – Ops and Evaluation module)

Open the original · 41 pages

Bashiri Smith's 41-page practitioner's guide to evaluating AI systems. It covers eval fundamentals (eval-driven development, golden datasets, metric types, basic statistics), deterministic and overlap metrics for LLM outputs, LLM-as-a-judge (rubric design, judge biases, checking judges against human labels), RAG evaluation (retrieval metrics and the RAG triad), agent trajectory evaluation, model benchmarking and fine-tune evaluation, and production evals (tracing, online signals, A/B tests, drift, guardrails, CI eval gates). It ends with a map of the eval tool stack, four portfolio projects and interview signals. Read it once end to end, then use it as a reference, and build one of the portfolio projects.

• Langfuse · tool · langfuse.com · free
  Listed under platforms that combine tracing and evaluation.
  Also in: Step-by-Step Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes), 7 Habits to Become an AI Engineer: Books, Tooling, Research & Shipping (Bashiri Smith on Facebook · notes), Taking a RAG App to Production: Evals, Guardrails, Cost and Tracing (Bashiri Smith on Facebook · notes)
• LangSmith · tool · docs.smith.langchain.com · free
  Tracing, logging and evals; also used to track cost and latency.
  Also in: Step-by-Step Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes), 7 Habits to Become an AI Engineer: Books, Tooling, Research & Shipping (Bashiri Smith on Facebook · notes)
• RAGAS · tool · github.com · free
  Listed under RAG evaluation tools in the guide's evaluation stack.
  Also in: SWE-to-AI Engineer Plan for 2027: LLMs, RAG, Agents, Evals, Job Search (Bashiri Smith on Facebook · notes), Taking a RAG App to Production: Evals, Guardrails, Cost and Tracing (Bashiri Smith on Facebook · notes)
• OpenAI Evals · repo · github.com · free · recommended by both Bashiri Smith & Melvin Vivas
  Listed under eval frameworks for defining and running evaluations.
  Also in: Is OpenAI Evals Still Maintained? (Melvin Vivas on X · notes)
• promptfoo · repo · github.com · free · recommended by both Bashiri Smith & Melvin Vivas
  Listed under eval frameworks for defining and running evaluations, offline and in CI.
  Also in: Running LLM evals with promptfoo on local llama.cpp models (Melvin Vivas on X · notes)
• Arize Phoenix · tool · github.com · free
  Listed under platforms that combine tracing and evaluation.
• Braintrust · tool · braintrust.dev · free
  Listed under platforms that combine tracing and evaluation.
• DeepEval · tool · github.com · free
  Listed under eval frameworks for defining and running evaluations, offline and in CI.
• Guardrails AI · tool · github.com · free
  Listed under guardrails tools for enforcing quality and safety on every request at runtime.
• NeMo Guardrails · tool · github.com · free
  Listed under guardrails tools for enforcing quality and safety on every request at runtime.
• TruLens · tool · trulens.org · free
  Listed under RAG evaluation tools in the guide's evaluation stack.
• Weights & Biases Weave · tool · github.com · free
  Listed under platforms that combine tracing and evaluation.

Try this

• Open and read the free Evaluation field guide from the Google Drive link.
• Optionally, look at the BASWE.Ai Engineer Skool community if you want a guided roadmap and calls with working engineers and recruiters.