Milestone map
Milestone map
2 milestones
Do the phase's lessons in public
1–1 weeks (about 10 hrs/week, est)
Work through the 12 lessons of Phase 9: Reinforcement Learning from AI Engineering from Scratch (free) and keep the reinforcement learning lessons' notebooks (MDPs, Q-learning, DQN, policy gradients, actor-critic, PPO) in one public repo as you go. About 7 hours (est).
Proof required
Link your public repo with the reinforcement learning lessons' notebooks (MDPs, Q-learning, DQN, policy gradients, actor-critic, PPO), and show your self-check (quizzes passed, notebooks run, tests passing where they exist).
What gets checked
- Each notebook runs.
- Learning curves are averaged over several seeds.
- Each algorithm beats a random policy (shown).
Common mistakes
- Copying the repo's solutions: verifiers read your history and ask about your code.
- Skipping the self-check, then asking for review too early.
Powstik Guide
A public repo with the reinforcement learning lessons' notebooks (MDPs, Q-learning, DQN, policy gradients, actor-critic, PPO).
Steps
- Make a public repo for this phase.
- Do one lesson at a time and commit after each.
- Self-check: Run each lesson's code; it must work on your machine as the lesson shows. Run each lesson's notebook top to bottom; its outputs must match the lesson.
- Write a short README: what is here and how to run it.
- Ask three people who know the topic to check it.
Template
README Phase 9: Reinforcement Learning (AI Engineering from Scratch) What is here: How to run it: Self-check result: What was hardest:
What gets sent back
- One big upload with no history.
- Self-check not shown.
- Code that does not run from the README.
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Each notebook runs.
- Learning curves are averaged over several seeds.
- Each algorithm beats a random policy (shown).
- Ask about one choice in the code; the answer should be clear.
You'll sign in first, then come straight back here.
Teach an agent a task
2–4 weeks (6–10 hrs/week)
Use what you learned. Make an agent that learns a game or control task (a gridworld you design, or CartPole), with one value-based and one policy-gradient method compared over at least 5 seeds. It must run, be evaluated and be documented.
Proof required
Link an agent that learns a game or control task (a gridworld you design, or CartPole), with one value-based and one policy-gradient method compared over at least 5 seeds, with a README that says how to run it and what the evaluation found.
What gets checked
- It runs from the README.
- Both methods are compared fairly, with spread shown.
- The reward design is explained.
Common mistakes
- A demo with no evaluation.
- Numbers without the data or code that produced them.
Powstik Guide
An agent that learns a game or control task (a gridworld you design, or CartPole), with one value-based and one policy-gradient method compared over at least 5 seeds.
Steps
- Write one line: what it does and for whom.
- Build the smallest version that works.
- Write the evaluation first, then improve against it.
- Write the README with the results.
- Ask three people to check it.
Template
README What it does: How to run it: How it was evaluated: Results: Limits:
What gets sent back
- Does not run from the README.
- No evaluation.
- Claims more than the evaluation shows.
Resources
Foundationstart here
What a verifier looks for
- It runs from the README.
- Both methods are compared fairly, with spread shown.
- The reward design is explained.
- Run it, or watch it run: it must work, be evaluated and be documented.
- Check that the work is the person's own.
You'll sign in first, then come straight back here.