powstik
Prove
Growth Paths / AI Research 2: Small GPT and loss curves
ExpertSkills

AI Research 2: Small GPT and loss curves

Train two model sizes, read the curves, and say what changed and why.

Train two model sizes, read the curves, and say what changed and why. Free resources, your code in a public repo, and real work confirmed by people. Part of the Varsity Curriculum: AI Research.

Part of: AI Research · Proves for: Engineers, PG

1 required outcomes6 weeksCredential on completion

Path outcomes

How this is verified: 4 are confirmed by people who saw the work.

SkillsOptional

Learn LLMs from Scratch (AI Engineering Phase 10)

Prerequisite (optional here): the AI Engineering phase this builds on.

Take this Challenge →

2 milestones

  1. 1Do the phase's lessons in public
  2. 2Train a tiny GPT end to end
Skills

Train a small GPT and explain its loss curves

Research step 2.

Take this Challenge →

2 milestones

  1. 1Train two sizes of a small GPT
  2. 2Write up what the curves show

Resources for this path

No affiliate links. No sponsored placements.

Free, small GPT training code (MIT License) to start from or compare against.

Free video course and notebooks (MIT License): backprop and language models built by hand.