Milestone map
Milestone map
3 milestones
Master Vectors, Matrices, and Linear Transformations
10–16 weeks
Study the foundations of linear algebra for machine learning: vector spaces, matrix operations, linear transformations, eigenvectors and eigenvalues, and matrix decompositions (LU, QR, SVD). Complete at least twenty exercises from MIT OCW 18.06 or equivalent, working through the derivations — not just applying formulas. Implement SVD from scratch using the power iteration method.
Proof required
Submit: a typed solution set for at least twenty problems from MIT OCW 18.06 problem sets (or equivalent) with all steps shown; a public GitHub repository (or Colab notebook) containing your SVD implementation using power iteration with a test comparing your output to NumPy's SVD; and a 200-word explanation of why SVD is used in dimensionality reduction. A mathematician, CS lecturer, or ML researcher must confirm the solutions and implementation are correct.
What gets checked
- At least twenty problem set solutions with all derivation steps shown — not just final answers
- SVD implementation produces output matching NumPy's SVD within numerical tolerance on the same input
- A mathematician, CS lecturer, or ML researcher has confirmed the solutions and implementation are correct
Common mistakes
- Using NumPy for the SVD computation — the proof requires implementing power iteration, not calling numpy.linalg.svd
- Problem set solutions with only final answers and no derivation steps — the proof requires showing the work
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Are at least twenty problem solutions shown with all derivation steps — not just final answers?
- Does the SVD implementation produce output matching NumPy's SVD within numerical tolerance?
- Ask: 'why does PCA use SVD rather than eigendecomposition directly on the data matrix?' — tests deeper understanding
Apply Linear Algebra to ML Algorithm Derivations
8–12 weeks (after milestone 1)
Derive the closed-form solution to linear regression using the normal equations (showing the matrix derivation step by step). Derive the gradient of the cross-entropy loss for logistic regression using matrix calculus. Implement PCA from scratch using SVD and verify it against scikit-learn's PCA on a real dataset.
Proof required
Submit: a typed derivation of the linear regression normal equations (matrix calculus steps, not just the final formula); a typed derivation of the logistic regression cross-entropy gradient; and a public GitHub repository (or Colab) containing your PCA implementation with a comparison to scikit-learn's PCA showing the explained variance ratios match. A mathematician or ML researcher must confirm the derivations are correct.
What gets checked
- Normal equations derivation shows all matrix calculus steps — not just the final (XᵀX)⁻¹Xᵀy formula
- PCA implementation produces explained variance ratios matching scikit-learn's PCA within numerical tolerance
- A mathematician or ML researcher has confirmed the derivations are correct
Common mistakes
- Normal equations derivation that starts from the formula without showing the matrix calculus derivation
- PCA implementation that uses scikit-learn for the core computation — must use SVD directly
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Does the normal equations derivation show all matrix calculus steps — not just the formula?
- Does the PCA implementation match scikit-learn's explained variance ratios?
- Ask: 'what happens to the normal equations when XᵀX is not invertible?' — tests understanding of rank deficiency
Present Linear Algebra for ML to an Expert Reviewer
2–4 weeks (after milestone 2)
Present your derivations and implementations to a mathematician or ML researcher in a live technical review. The reviewer will pose at least two questions about the mathematical properties of the algorithms you implemented — questions you must answer using the linear algebra concepts, not just code recall.
Proof required
Submit: a recording or transcript of a live review with a mathematician or ML researcher; your derivation documents and implementation repository as the basis for the review; and documentation of at least two questions posed during the review and your responses.
What gets checked
- Review was live — not an async written exchange
- At least two mathematical questions were posed about the algorithms' properties — not just the implementations
- Your responses used linear algebra concepts specifically — not just code-level descriptions
Common mistakes
- Live review that becomes a code walkthrough without mathematical questions
- Responses that describe what the code does rather than using linear algebra concepts to explain the algorithm
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Was the review live — not async?
- Were at least two mathematical questions about algorithm properties posed?
- Did the responses use linear algebra concepts — not just code-level descriptions?