AI/LLM Fundamentals for Engineers: Transformers, Fine-tuning vs RAG vs LoRA, and Agent Protocols

The fundamentals underneath every LLM system design: tokens and BPE (and why Hindi or Tamil text costs more), embeddings and cosine similarity, the Transformer and self-attention worked through with real numbers, decoder-only generation and sampling, why long context is expensive, training vs fine-tuning vs inference, prompt engineering vs RAG vs fine-tuning, full fine-tuning vs LoRA vs QLoRA with parameter and memory arithmetic, evaluation basics, MCP vs A2A, and when an agent framework beats a plain loop.

About 140 minutes · 18 exam questions · 12 interview questions · 2 code labs

What you will learn

  1. Tokens and tokenization: BPE, why token counts drive cost and latency, and the Indian-language tax
  2. Embeddings and vector similarity: cosine, dimensions, and what "semantic" really means
  3. The Transformer in engineering terms: from token ids to next-token probabilities
  4. Self-attention step by step: Q, K, V and a worked 3-token example with real numbers
  5. Decoder-only generation and sampling: temperature, top-p, the context window and why attention cost grows with length
  6. Training vs fine-tuning vs inference: how a model comes to exist and which part your team actually touches
  7. Prompt engineering vs RAG vs fine-tuning: a decision table for knowledge, behaviour, freshness and cost
  8. Full fine-tuning vs LoRA vs QLoRA: low-rank adapters, parameter counts and GPU memory
  9. Evaluation basics: hallucination, groundedness, correctness, and why vibes are not a release gate
  10. Agent protocols: MCP for tools and context, A2A for agent-to-agent delegation, and how they compose
  11. Agent types and frameworks: LangGraph, CrewAI, n8n-style workflows, and when a plain loop is better
  12. LLM glossary for system-design interviews: the 30 terms you should define in one breath

Previous: Key Technology Deep Dives: DynamoDB, Temporal & ZooKeeper