17-Step AI Engineer Roadmap: From Basic RAG to Agents, Evals, LLMOps & Governance

Bashiri Smith · Facebook reel · 2026-08-22 · 0:11 · 90,085 views · Open on Facebook

Topics: Start Here: Roadmaps & Strategy, Retrieval-Augmented Generation (RAG), AI Agents, Tool Use & MCP · Level: intermediate

Summary

Bashiri Smith says building a basic RAG chatbot isn't enough to become a serious AI engineer, and lays out a 17-step learning path. It starts with LLM fundamentals and RAG, then covers advanced retrieval, agents and design patterns, MCP and multi-agent systems. It continues with evaluation (including LLM-as-a-judge and agent evals), guardrails, observability, LLMOps, CI/CD, data versioning, classic ML metrics, and AI governance. The reel has no speech; all the content is in the caption.

Key points

• Foundations first: tokens, transformers, context windows and inference. Then basic RAG: chunking, embeddings, vector databases, retrieval and generation.
• Advanced retrieval goes beyond basic RAG with hybrid search, query rewriting, metadata filtering and reranking.
• Agents run a loop of reason, act, observe, repeat. Key design patterns are ReAct, reflection, planning, routing and tool use.
• Model Context Protocol (MCP) connects agents to tools, APIs and data. Multi-agent systems have agents that specialize, delegate and collaborate.
• Evals: define what 'good' means, use LLM-as-a-judge with structured criteria at scale, and score agent trajectories, tool calls, task completion and outcomes.
• Production: guardrails (validation, moderation, PII protection, prompt-injection defenses), observability (latency, cost, failures), LLMOps lifecycle and CI/CD to test and ship changes safely.
• Reproducibility and ML basics: version datasets with DVC, and know bias/variance, overfitting, precision, recall, F1 and ROC-AUC.
• Governance: human oversight, risk management, the NIST AI RMF and the EU AI Act.

Resources mentioned

• ReAct: Synergizing Reasoning and Acting in Language Models · paper · arxiv.org · free
  The paper behind the ReAct agent pattern, which interleaves reasoning steps with tool actions.
• Model Context Protocol (MCP) · docs · modelcontextprotocol.io · free · recommended by both Bashiri Smith & Melvin Vivas
  An open standard for connecting LLM apps and agents to tools and data sources. The official docs explain how it works and how to build servers and clients.
  Also in: Grok Bot Templates: Sharing, Publishing and Safely Installing Bots (Melvin Vivas on X · notes), 8-Week Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes), Paste Documentation URLs into Codex Instead of Using MCP (Melvin Vivas on X · notes), OpenAI Agents API: Hosted Codex Harness for Long-Running Cloud Agents (Melvin Vivas on X · notes) and 7 more
• DVC (Data Version Control) · tool · dvc.org · free · open in a browser to verify
  Version your data like code.
  Also in: Five Core AI Engineering Topic Areas: A Study Roadmap Checklist (Bashiri Smith on Facebook · notes)
• NIST AI Risk Management Framework (AI RMF) · pdf · nist.gov · free
  US NIST framework for identifying and managing risks of AI systems.
• EU AI Act · pdf · eur-lex.europa.eu · free
  The European Union's law regulating AI systems by risk level.
• The Complete AI Engineer Roadmap for 2026 (Exact Courses + Step-by-Step) · video · youtube.com · free
  Bashiri Smith's free YouTube training on upgrading your skills and becoming competitive for AI engineering roles, with specific courses listed step by step.
  Also in: SWE-to-AI Engineer Plan for 2027: LLMs, RAG, Agents, Evals, Job Search (Bashiri Smith on Facebook · notes), Pointer to Bashiri Smith's Complete AI Engineer Roadmap for 2026 (Bashiri Smith on Facebook · notes), Step-by-Step Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes), Software Engineer to AI Engineer Before 2027: A 5-Step Career Plan (Bashiri Smith on Facebook · notes) and 19 more
• BASWE.Ai Engineer (Skool community) · community · skool.com · paid
  The creator's paid community and program, with an AI learning roadmap (including the full ops and evaluation track), daily calls with engineers and recruiters, resume and portfolio help, and a job-search pipeline.
  Also in: Basic RAG Pipeline in 60 Seconds: From Documents to Grounded Answers (Bashiri Smith on Facebook · notes), Pointer to Bashiri Smith's Complete AI Engineer Roadmap for 2026 (Bashiri Smith on Facebook · notes), Step-by-Step Roadmap to a $200K+ AI Engineering Role (Bashiri Smith on Facebook · notes), How to Evaluate a RAG Pipeline: Retrieval vs. Generation (Interview Answer) (Bashiri Smith on Facebook · notes) and 76 more

Try this

• Go beyond a basic RAG chatbot and work through the 17 steps in order.
• Learn advanced retrieval: hybrid search, query rewriting, metadata filtering and reranking.
• Build agents with the ReAct, reflection, planning, routing and tool-use patterns, and connect them to tools through MCP.
• Set up evals: define success criteria, use LLM-as-a-judge, and score agent trajectories and tool calls.
• Add guardrails, observability, CI/CD and data versioning with DVC to your LLM apps.
• Study the NIST AI RMF and the EU AI Act for governance.
• Comment 'AI' on the post to receive the full roadmap and community link.