Milestone map
Milestone map
2 milestones
Do the phase's lessons in public
2–3 weeks (about 10 hrs/week, est)
Work through the 54 lessons of Phase 14: Agent Engineering from AI Engineering from Scratch (free) and keep the agent engineering lessons' code and notebooks (the agent loop, planning, memory, tool use, frameworks, observability, evaluation) in one public repo as you go. About 30 hours (est).
Proof required
Link your public repo with the agent engineering lessons' code and notebooks (the agent loop, planning, memory, tool use, frameworks, observability, evaluation), and show your self-check (quizzes passed, notebooks run, tests passing where they exist).
What gets checked
- Each lesson's code runs and its quiz is passed.
- Agents are traced (logs or OpenTelemetry).
- Evaluation runs are kept in the repo.
Common mistakes
- Copying the repo's solutions: verifiers read your history and ask about your code.
- Skipping the self-check, then asking for review too early.
Powstik Guide
A public repo with the agent engineering lessons' code and notebooks (the agent loop, planning, memory, tool use, frameworks, observability, evaluation).
Steps
- Make a public repo for this phase.
- Do one lesson at a time and commit after each.
- Self-check: Run each lesson's code; it must work on your machine as the lesson shows. Run each lesson's notebook top to bottom; its outputs must match the lesson. Each lesson has a quiz (quiz.json, also on the course site): pass it.
- Write a short README: what is here and how to run it.
- Ask three people who know the topic to check it.
Template
README Phase 14: Agent Engineering (AI Engineering from Scratch) What is here: How to run it: Self-check result: What was hardest:
What gets sent back
- One big upload with no history.
- Self-check not shown.
- Code that does not run from the README.
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Each lesson's code runs and its quiz is passed.
- Agents are traced (logs or OpenTelemetry).
- Evaluation runs are kept in the repo.
- Ask about one choice in the code; the answer should be clear.
You'll sign in first, then come straight back here.
Build an agent and write up its failures
2–4 weeks (6–10 hrs/week)
Use what you learned. Make an agent with tool use that does a real multi-step task, with an evaluation set of at least 20 tasks and a written failure analysis. It must run, be evaluated and be documented.
Proof required
Link an agent with tool use that does a real multi-step task, with an evaluation set of at least 20 tasks and a written failure analysis, with a README that says how to run it and what the evaluation found.
What gets checked
- It runs from the README.
- The success rate on the task set is shown.
- The failure analysis names causes and one fix that was shipped.
Common mistakes
- A demo with no evaluation.
- Numbers without the data or code that produced them.
Powstik Guide
An agent with tool use that does a real multi-step task, with an evaluation set of at least 20 tasks and a written failure analysis.
Steps
- Write one line: what it does and for whom.
- Build the smallest version that works.
- Write the evaluation first, then improve against it.
- Write the README with the results.
- Ask three people to check it.
Template
README What it does: How to run it: How it was evaluated: Results: Limits:
What gets sent back
- Does not run from the README.
- No evaluation.
- Claims more than the evaluation shows.
Resources
Foundationstart here
What a verifier looks for
- It runs from the README.
- The success rate on the task set is shown.
- The failure analysis names causes and one fix that was shipped.
- Run it, or watch it run: it must work, be evaluated and be documented.
- Check that the work is the person's own.
You'll sign in first, then come straight back here.