Apply Statistical Learning Theory to Your Own Dataset
10 weeks · 0 milestones
Document the bias-variance trade-off, model selection using cross-validation, and hypothesis testing applied to a dataset you collected yourself — not a pre-built Kaggle dataset. Your write-up must explain your data collection methodology, the specific modelling decisions you made and why, and the statistical tests you ran to validate each decision. Using your own data is the proof standard: it means the analysis cannot be pre-generated from a known dataset. Proof: the write-up and dataset reviewed by a statistician or ML practitioner who asks 'what would your cross-validation results look like if you doubled the number of folds?' — you must answer using your specific data and analysis, not the general principle.
Milestone map
Milestone map
3 milestones
Study the theoretical foundations of statistical learning: probably approximately correct (PAC) learning framework, VC dimension and its relationship to sample complexity, bias-variance tradeoff, and the bias-variance decomposition. Complete at least fifteen exercises from MIT OCW 18.657 or equivalent. Prove at least one VC dimension bound for a specific hypothesis class from scratch.
Proof required
Submit: a typed solution set for at least fifteen exercises on PAC learning, VC dimension, or sample complexity (with all derivation steps — not just final answers); a typed proof of the VC dimension for a specific hypothesis class (e.g. half-spaces in R^n, axis-aligned rectangles in R^2) with all proof steps; and a 200-word explanation of why VC dimension predicts generalisation error. An ML researcher or mathematician must confirm the solutions and proof are correct.
What gets checked
- At least fifteen exercise solutions with all derivation steps — not just final answers
- VC dimension proof is complete — not a sketch or reference to the standard result
- An ML researcher or mathematician has confirmed the solutions and proof are correct