Milestone map
Milestone map
2 milestones
Do the phase's lessons in public
1–1 weeks (about 10 hrs/week, est)
Work through the 25 lessons of Phase 12: Multimodal AI from AI Engineering from Scratch (free) and keep the multimodal lessons' notebooks (vision transformers, CLIP, vision-language models, multimodal RAG) in one public repo as you go. About 14 hours (est).
Proof required
Link your public repo with the multimodal lessons' notebooks (vision transformers, CLIP, vision-language models, multimodal RAG), and show your self-check (quizzes passed, notebooks run, tests passing where they exist).
What gets checked
- Each notebook runs.
- Each model is tried on your own images or documents.
- Results are described honestly, including failures.
Common mistakes
- Copying the repo's solutions: verifiers read your history and ask about your code.
- Skipping the self-check, then asking for review too early.
Powstik Guide
A public repo with the multimodal lessons' notebooks (vision transformers, CLIP, vision-language models, multimodal RAG).
Steps
- Make a public repo for this phase.
- Do one lesson at a time and commit after each.
- Self-check: Run each lesson's code; it must work on your machine as the lesson shows. Run each lesson's notebook top to bottom; its outputs must match the lesson.
- Write a short README: what is here and how to run it.
- Ask three people who know the topic to check it.
Template
README Phase 12: Multimodal AI (AI Engineering from Scratch) What is here: How to run it: Self-check result: What was hardest:
What gets sent back
- One big upload with no history.
- Self-check not shown.
- Code that does not run from the README.
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Each notebook runs.
- Each model is tried on your own images or documents.
- Results are described honestly, including failures.
- Ask about one choice in the code; the answer should be clear.
You'll sign in first, then come straight back here.
Ask questions about documents or images
2–4 weeks (6–10 hrs/week)
Use what you learned. Make a document or image question-answering tool using a vision-language model, with an evaluation set of at least 30 questions and the failure cases written up. It must run, be evaluated and be documented.
Proof required
Link a document or image question-answering tool using a vision-language model, with an evaluation set of at least 30 questions and the failure cases written up, with a README that says how to run it and what the evaluation found.
What gets checked
- It runs from the README.
- The score on the 30+ questions is shown.
- Failure cases are grouped by cause.
Common mistakes
- A demo with no evaluation.
- Numbers without the data or code that produced them.
Powstik Guide
A document or image question-answering tool using a vision-language model, with an evaluation set of at least 30 questions and the failure cases written up.
Steps
- Write one line: what it does and for whom.
- Build the smallest version that works.
- Write the evaluation first, then improve against it.
- Write the README with the results.
- Ask three people to check it.
Template
README What it does: How to run it: How it was evaluated: Results: Limits:
What gets sent back
- Does not run from the README.
- No evaluation.
- Claims more than the evaluation shows.
Resources
Foundationstart here
What a verifier looks for
- It runs from the README.
- The score on the 30+ questions is shown.
- Failure cases are grouped by cause.
- Run it, or watch it run: it must work, be evaluated and be documented.
- Check that the work is the person's own.
You'll sign in first, then come straight back here.