Series

LLM Architecture: GPT-2

The fundamentals every model since has inherited.

Chapters

5 of 8
  1. 01LLM Tokenization2 min
  2. 02From Raw Text to LLM Training Data6 min
  3. 03Layer Normalization2 min
  4. 04Attention, Step by Step9 min
  5. 05Dropout4 min
  6. 06Shortcut connections—
  7. 07The feed-forward network—
  8. 08From vectors to logits—

Search

Search pages, articles, and resources