Milestone map
Milestone map
2 milestones
Do the course work in public
3–5 weeks (4–5 hrs/week)
Take Data Warehouse Concepts, Design, and Data Integration for free and do the module 1 and 2 homework: load a real dataset into Postgres with Docker, then run the load on a schedule. Keep everything in one public repo or notebook as you go. OSSU lists a Coursera course, where graded work is paid. We use the free Data Engineering Zoomcamp.
Proof required
Link your public repo or notebook with the module 1 and 2 homework: load a real dataset into Postgres with Docker, then run the load on a schedule, and a screenshot or log showing the self-check passed.
What gets checked
- The pipeline runs from the README with one command.
- Homework answers match what the pipeline loaded.
- Re-running it does not load the same rows twice.
Common mistakes
- Copying posted answers: verifiers read the history.
- Skipping the self-check, then asking for review too early.
Powstik Guide
A public repo or notebook with the module 1 and 2 homework: load a real dataset into Postgres with Docker, then run the load on a schedule.
Steps
- Make a public repo or notebook first.
- Do one unit at a time and commit after each.
- Run the self-check: Answer each homework question from your own running pipeline, then compare with the posted answers.
- Write a short README: what is here and how to run it.
- Ask three people who know the topic to check it.
Template
README Course: Data Warehouse Concepts, Design, and Data Integration (Data Engineering Zoomcamp (DataTalks.Club), modules 1–2) What is here: How to run it: Self-check result: What was hardest:
What gets sent back
- No history, just one big upload.
- Self-check not shown.
- Code that does not run from the README.
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- The pipeline runs from the README with one command.
- Homework answers match what the pipeline loaded.
- Re-running it does not load the same rows twice.
- Ask one question about a choice in the code; the answer should be clear.
You'll sign in first, then come straight back here.
Pipe a public dataset into a database
1–3 weeks (5–8 hrs/week)
Use what you learned. Make a pipeline that pulls a public dataset you care about into a database every day or week, with a log of each run. Keep it small enough to finish in about two weeks.
Proof required
Link a pipeline that pulls a public dataset you care about into a database every day or week, with a log of each run, with a README that says how to run or read it.
What gets checked
- It has run at least three times on its own (log shown).
- Bad rows are caught, not loaded.
- The README says where the data comes from.
Common mistakes
- A project too big to finish.
- A demo that only works on your machine.
Powstik Guide
A pipeline that pulls a public dataset you care about into a database every day or week, with a log of each run.
Steps
- Write one line: what it does and for whom.
- Build the smallest version that works.
- Test it on someone else's machine or with someone else reading it.
- Write the README.
- Ask three people to check it.
Template
README What it does: Who it is for: How to run it: What I would do next:
What gets sent back
- Does not run from the README.
- Not clearly the person's own work.
- Claims more than it does.
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- It has run at least three times on its own (log shown).
- Bad rows are caught, not loaded.
- The README says where the data comes from.
- Check that the work is the person's own.
You'll sign in first, then come straight back here.
Part of