Production Mastery

Weeks 15–17 · 4 Modules + Capstone. Building is only half the job. This phase teaches you to ship, monitor, and maintain AI systems at scale. This is what separates engineers from hobbyists.

Module 22: Testing AI Systems — AI systems need different testing strategies. Covers pytest (fixtures, async testing), FastAPI TestClient, streaming endpoint tests, K6 load testing (virtual users, scenarios, thresholds), and LLM output evaluation (factuality, relevance, automated pipelines).

Module 23: LLMOps & Observability — You can't improve what you can't measure. Covers LangSmith (tracing, evaluation datasets, prompt versioning, cost tracking), structured logging, Prometheus metrics, AI-specific metrics (latency, tokens, errors), cost management, and budget alerts.

Module 24: Production Deployment — Getting AI to production is different from traditional software. Covers Docker (Dockerfile best practices, multi-stage builds), Google Cloud Run (config, secrets, scaling, Cloud SQL), GitHub Actions for Python, CI/CD automation, rollbacks, and the production checklist.

Module 25: Ethics, Safety & Governance — AI systems can cause harm. Build responsibly. Covers bias and fairness, prompt injection defense, data poisoning, model extraction, PII leakage, explainability, GDPR for AI, ISO 42001, model cards, content safety filters, and incident response.

Capstone Project: AI-Powered Document Intelligence Platform — Build a complete system: Next.js 14+ frontend with streaming chat, FastAPI backend with Pydantic, Google Gemini with RAG and function calling, Postgres with pgvector, Docker + Cloud Run + Vercel, LangSmith observability, pytest + Playwright + K6 testing. Required: auth, document upload, intelligent chunking, streaming chat with citations, conversation history, admin dashboard, and safety measures.

Phase Outcome: Ship, monitor, and scale real AI applications. Complete a production capstone that proves mastery of the entire Hashflag Stack.