5 RAG Metrics: Top-k, Recall@k, Precision@k, Top-p and Cost per Request
Bashiri Smith · Facebook reel · 2026-10-07 · 0:53 · 3,694 views · Open on Facebook
Topics: Retrieval-Augmented Generation (RAG), Evaluation (Evals) & Testing, LLMOps, Deployment & Monitoring · Level: beginner
Summary
A short explainer of five numbers every AI engineer working on RAG should understand. It covers retrieval settings and metrics (top-k, recall@k, precision@k) with worked percentage examples. It also covers the generation sampling setting top-p and the cost of answering one question. The creator offers a RAG cheat sheet and a link to his AI engineer community.
Key points
- Top-k sets how many chunks the retriever returns. With k=5 you get the 5 highest-scoring chunks.
- Recall@k = relevant chunks retrieved in the top k / all relevant chunks. Example: 2 of 4 relevant chunks found = 50% recall.
- Precision@k = relevant chunks in the top k / k. Example: 2 relevant out of 5 returned = 40% precision.
- Raising k usually helps recall but can hurt precision. Tune k with both metrics in view.
- Top-p (nucleus sampling) controls generation, not retrieval. At 0.9 the model keeps the most likely next tokens until their combined probability reaches 90%, then samples from that set.
- Cost per request = retrieval + any reranking + all model calls needed to answer one question. Track it as a production metric.
Resources mentioned
- BASWE.AI: RAG Guide (RAG Cheat Sheet) · pdf · drive.google.com · free
The creator's Google Drive folder with a RAG cheat sheet and guide covering RAG concepts and metrics. - BASWE.Ai Engineer (Skool community) · community · skool.com · paid
The creator's paid community and program, with an AI learning roadmap (including the full ops and evaluation track), daily calls with engineers and recruiters, resume and portfolio help, and a job-search pipeline.
Also in: Basic vs Production AI Engineering: RAG, Agents, and Context over Prompts (Bashiri Smith on Facebook · notes), ChatGPT Prompt to Generate Standout Portfolio Project Ideas (Bashiri Smith on Facebook · notes), Vector Database vs. Graph Database: When to Use Each (and Both for RAG) (Bashiri Smith on Facebook · notes), The 5 Levels of RAG: From Naive Retrieval to Agentic RAG (Bashiri Smith on Facebook · notes) and 93 more
Try this
- Comment 'metrics' on the reel to get the full RAG cheat sheet.
- Measure recall@k and precision@k on your RAG system with a small labeled set of relevant chunks.
- Track cost per request (retrieval + reranking + model calls) for your RAG pipeline.
More in Retrieval-Augmented Generation (RAG)
- The 5 Levels of RAG: From Naive Retrieval to Agentic RAG
- Production RAG: Check Retrieval, Answer Support, and Unseen Questions
- Entity Linking: Connecting Text Mentions to Structured Records
- AI Engineer Interview: RAG Evals, Hybrid Search & Cost/Latency Optimization
- AI Engineer Mistakes: RAG Evaluation, Hybrid Search and Cutting Cost
- Production RAG Interview Answers: Design, Debugging, and Scaling