LLM from scratch
A GPT built to understand, not to ship
training replay · representative log
step 4800Press play to watch a GPT learn to read — from random noise to sentences.
sample ▸
“deep sleep lowers resting heart rate and helps the body recover.”
A GPT built from first principles, tokenizer, attention, training loop, all of it, trained on a corpus of health facts, then supervised-fine-tuned into a small chat model. No frameworks doing the thinking.
Written as an explain-it-simply series: every mechanism gets a working implementation and a plain-language account of why it exists. The point wasn’t the model; it was earning the right to reason about the tools I use every day.
Stack
- Python
- PyTorch
- Tokenization → attention → SFT