LearnAI

The papers that built modern AI

Plain-English walkthroughs of landmark research: what each paper actually says, why it mattered, and where its ideas show up in the roadmaps you are learning.

Paper stories5
  1. Transformer · 2017Attention is All You NeedAshish Vaswani, Noam Shazeer et al.Replace recurrence with self-attention. Train in parallel, get better quality, unlock every modern LLM, ViT, and multimodal model that followed.
  2. Scale · 2020Language Models are Few-Shot Learners (GPT-3)Tom B. Brown, Benjamin Mann et al.Scale a Transformer to 175B parameters, train on 300B tokens. Suddenly it solves new tasks from a few prompt examples alone. No fine-tuning needed. Birth of prompt engineering and the direct ancestor of ChatGPT.
  3. MoE · 2024DeepSeek-V3 Technical ReportDeepSeek-AIA 671B-parameter Mixture of Experts model (only 37B active per token) trained for ~$5.5M to match GPT-4 quality, via Multi-head Latent Attention, FP8 training, and Multi-Token Prediction. Open weights.
  4. Fine-Tune · 2021LoRA: Low-Rank Adaptation of Large Language ModelsEdward Hu, Yelong Shen et al.Freeze the big model. Add tiny low-rank matrices to each weight. Train only those. Get ~10,000× fewer trainable params, same quality as full fine-tuning, and the ability to swap thousands of fine-tunes from one base model.
  5. RAG · 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez et al.Combine a parametric language model (BART) with a non-parametric retriever (DPR) over Wikipedia. The model reads relevant passages before answering, so it gets facts right without storing them all in weights, and you can cite sources.