17-Step AI Engineer Roadmap: From Basic RAG to Agents, Evals, LLMOps & Governance
Bashiri Smith · Facebook reel · 2026-08-22 · 0:11 · 90,085 views · Open on Facebook
Topics: Start Here: Roadmaps & Strategy, Retrieval-Augmented Generation (RAG), AI Agents, Tool Use & MCP · Level: intermediate
Summary
Bashiri Smith says building a basic RAG chatbot isn't enough to become a serious AI engineer, and lays out a 17-step learning path. It starts with LLM fundamentals and RAG, then covers advanced retrieval, agents and design patterns, MCP and multi-agent systems. It continues with evaluation (including LLM-as-a-judge and agent evals), guardrails, observability, LLMOps, CI/CD, data versioning, classic ML metrics, and AI governance. The reel has no speech; all the content is in the caption.
Key points
- Foundations first: tokens, transformers, context windows and inference. Then basic RAG: chunking, embeddings, vector databases, retrieval and generation.
- Advanced retrieval goes beyond basic RAG with hybrid search, query rewriting, metadata filtering and reranking.
- Agents run a loop of reason, act, observe, repeat. Key design patterns are ReAct, reflection, planning, routing and tool use.
- Model Context Protocol (MCP) connects agents to tools, APIs and data. Multi-agent systems have agents that specialize, delegate and collaborate.
- Evals: define what 'good' means, use LLM-as-a-judge with structured criteria at scale, and score agent trajectories, tool calls, task completion and outcomes.
- Production: guardrails (validation, moderation, PII protection, prompt-injection defenses), observability (latency, cost, failures), LLMOps lifecycle and CI/CD to test and ship changes safely.
- Reproducibility and ML basics: version datasets with DVC, and know bias/variance, overfitting, precision, recall, F1 and ROC-AUC.
- Governance: human oversight, risk management, the NIST AI RMF and the EU AI Act.
Resources mentioned
- ReAct: Synergizing Reasoning and Acting in Language Models · paper · arxiv.org · free
The original ReAct paper behind the tool-use reason-act-observe loop. - Model Context Protocol (MCP) · docs · modelcontextprotocol.io · free · recommended by both Bashiri Smith & Melvin Vivas
Open standard for connecting models to tools and data sources through one interface. It turns an N×M integration problem into N+M.
Also in: Extend Grok Bot with Your Own MCP Servers and Connectors (Melvin Vivas on X · notes), AI Engineer Vocab Quiz: 11 Acronyms to Know for Interviews (Bashiri Smith on Facebook · notes), The 3 AI Engineering Skills: Generative AI, RAG, and Agentic AI (Bashiri Smith on Facebook · notes), xAI's Experimental TypeScript SDK: Grok Text, Voice, Image and Video in One SDK (Melvin Vivas on X · notes) and 11 more - DVC (Data Version Control) · tool · dvc.org · free · open in a browser to verify
Version your data like code.
Also in: Five Core AI Engineering Topic Areas: A Study Roadmap Checklist (Bashiri Smith on Facebook · notes) - NIST AI Risk Management Framework (AI RMF) · pdf · nist.gov · free
US NIST framework for identifying and managing risks of AI systems. - EU AI Act · pdf · eur-lex.europa.eu · free
The European Union's law regulating AI systems by risk level. - The Complete AI Engineer Roadmap for 2026 (Exact Courses + Step-by-Step) · video · youtube.com · free
Bashiri Smith's free YouTube training on upgrading your skills and becoming competitive for AI engineering roles, with specific courses listed step by step.
Also in: AI Career Paths for Software Engineers: AI Engineer, FDE, Platform & More (Bashiri Smith on Facebook · notes), SWE-to-AI Engineer Plan for 2027: LLMs, RAG, Agents, Evals, Job Search (Bashiri Smith on Facebook · notes), Pointer to Bashiri Smith's Complete AI Engineer Roadmap for 2026 (Bashiri Smith on Facebook · notes), Step-by-Step Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes) and 20 more - BASWE.Ai Engineer (Skool community) · community · skool.com · paid
The creator's paid community and program, with an AI learning roadmap (including the full ops and evaluation track), daily calls with engineers and recruiters, resume and portfolio help, and a job-search pipeline.
Also in: Basic vs Production AI Engineering: RAG, Agents, and Context over Prompts (Bashiri Smith on Facebook · notes), ChatGPT Prompt to Generate Standout Portfolio Project Ideas (Bashiri Smith on Facebook · notes), Vector Database vs. Graph Database: When to Use Each (and Both for RAG) (Bashiri Smith on Facebook · notes), The 5 Levels of RAG: From Naive Retrieval to Agentic RAG (Bashiri Smith on Facebook · notes) and 93 more
Try this
- Go beyond a basic RAG chatbot and work through the 17 steps in order.
- Learn advanced retrieval: hybrid search, query rewriting, metadata filtering and reranking.
- Build agents with the ReAct, reflection, planning, routing and tool-use patterns, and connect them to tools through MCP.
- Set up evals: define success criteria, use LLM-as-a-judge, and score agent trajectories and tool calls.
- Add guardrails, observability, CI/CD and data versioning with DVC to your LLM apps.
- Study the NIST AI RMF and the EU AI Act for governance.
- Comment 'AI' on the post to receive the full roadmap and community link.