Milestone map
Milestone map
2 milestones
Do the course work in public
4–7 weeks (5–10 hrs/week)
Take Mining Massive Datasets for free and do homework for MapReduce, similar items (min-hashing), link analysis (PageRank) and frequent itemsets, with your code. Keep everything in one public repo or notebook as you go. OSSU links the edX page; we link the authors' free book, lectures and self-study homework.
Proof required
Link your public repo or notebook with homework for MapReduce, similar items (min-hashing), link analysis (PageRank) and frequent itemsets, with your code, and a screenshot or log showing the self-check passed.
What gets checked
- Gradiance scores are shown for each chapter.
- Your code runs on a dataset too big for one spreadsheet.
- Results are checked against a small case done by hand.
Common mistakes
- Copying posted answers: verifiers read the history.
- Skipping the self-check, then asking for review too early.
Powstik Guide
A public repo or notebook with homework for MapReduce, similar items (min-hashing), link analysis (PageRank) and frequent itemsets, with your code.
Steps
- Make a public repo or notebook first.
- Do one unit at a time and commit after each.
- Run the self-check: Register for Gradiance self-study with the class token on mmds.org and pass the homework for each chapter.
- Write a short README: what is here and how to run it.
- Ask three people who know the topic to check it.
Template
README Course: Mining Massive Datasets (Mining of Massive Datasets (free book and lectures, mmds.org)) What is here: How to run it: Self-check result: What was hardest:
What gets sent back
- No history, just one big upload.
- Self-check not shown.
- Code that does not run from the README.
Resources
Foundationstart here
Depthgo deeper
Masteryfor the dedicated
What a verifier looks for
- Gradiance scores are shown for each chapter.
- Your code runs on a dataset too big for one spreadsheet.
- Results are checked against a small case done by hand.
- Ask one question about a choice in the code; the answer should be clear.
You'll sign in first, then come straight back here.
Find similar items at scale
1–3 weeks (5–8 hrs/week)
Use what you learned. Make a program that finds near-duplicates or similar items in a large public dataset (news articles, products, songs) with min-hashing, with timings. Keep it small enough to finish in about two weeks.
Proof required
Link a program that finds near-duplicates or similar items in a large public dataset (news articles, products, songs) with min-hashing, with timings, with a README that says how to run or read it.
What gets checked
- It runs on the full dataset, not a sample only.
- A sample of matches is checked by hand.
- Timings show it scales better than comparing every pair.
Common mistakes
- A project too big to finish.
- A demo that only works on your machine.
Powstik Guide
A program that finds near-duplicates or similar items in a large public dataset (news articles, products, songs) with min-hashing, with timings.
Steps
- Write one line: what it does and for whom.
- Build the smallest version that works.
- Test it on someone else's machine or with someone else reading it.
- Write the README.
- Ask three people to check it.
Template
README What it does: Who it is for: How to run it: What I would do next:
What gets sent back
- Does not run from the README.
- Not clearly the person's own work.
- Claims more than it does.
Resources
Foundationstart here
Depthgo deeper
Masteryfor the dedicated
What a verifier looks for
- It runs on the full dataset, not a sample only.
- A sample of matches is checked by hand.
- Timings show it scales better than comparing every pair.
- Check that the work is the person's own.
You'll sign in first, then come straight back here.