Milestone map
Milestone map
3 milestones
Master Vectors, Matrices, and Linear Transformations
10–16 weeks
Study the foundations of linear algebra for machine learning: vector spaces, matrix operations, linear transformations, eigenvectors and eigenvalues, and matrix decompositions (LU, QR, SVD). Complete at least twenty exercises from MIT OCW 18.06 or equivalent, working through the derivations — not just applying formulas. Implement SVD from scratch using the power iteration method.
Proof required
Submit: a typed solution set for at least twenty problems from MIT OCW 18.06 problem sets (or equivalent) with all steps shown; a public GitHub repository (or Colab notebook) containing your SVD implementation using power iteration with a test comparing your output to NumPy's SVD; and a 200-word explanation of why SVD is used in dimensionality reduction. A mathematician, CS lecturer, or ML researcher must confirm the solutions and implementation are correct.
What gets checked
- At least twenty problem set solutions with all derivation steps shown — not just final answers
- SVD implementation produces output matching NumPy's SVD within numerical tolerance on the same input
- A mathematician, CS lecturer, or ML researcher has confirmed the solutions and implementation are correct
Common mistakes
- Using NumPy for the SVD computation — the proof requires implementing power iteration, not calling numpy.linalg.svd
- Problem set solutions with only final answers and no derivation steps — the proof requires showing the work
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Are at least twenty problem solutions shown with all derivation steps — not just final answers?
- Does the SVD implementation produce output matching NumPy's SVD within numerical tolerance?
- Ask: 'why does PCA use SVD rather than eigendecomposition directly on the data matrix?' — tests deeper understanding
You'll sign in first, then come straight back here.
Apply Linear Algebra to ML Algorithm Derivations
8–12 weeks (after milestone 1)
Derive the closed-form solution to linear regression using the normal equations (showing the matrix derivation step by step). Derive the gradient of the cross-entropy loss for logistic regression using matrix calculus. Implement PCA from scratch using SVD and verify it against scikit-learn's PCA on a real dataset.
Proof required
Submit: a typed derivation of the linear regression normal equations (matrix calculus steps, not just the final formula); a typed derivation of the logistic regression cross-entropy gradient; and a public GitHub repository (or Colab) containing your PCA implementation with a comparison to scikit-learn's PCA showing the explained variance ratios match. A mathematician or ML researcher must confirm the derivations are correct.
What gets checked
- Normal equations derivation shows all matrix calculus steps — not just the final (XᵀX)⁻¹Xᵀy formula
- PCA implementation produces explained variance ratios matching scikit-learn's PCA within numerical tolerance
- A mathematician or ML researcher has confirmed the derivations are correct
Common mistakes
- Normal equations derivation that starts from the formula without showing the matrix calculus derivation
- PCA implementation that uses scikit-learn for the core computation — must use SVD directly
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Does the normal equations derivation show all matrix calculus steps — not just the formula?
- Does the PCA implementation match scikit-learn's explained variance ratios?
- Ask: 'what happens to the normal equations when XᵀX is not invertible?' — tests understanding of rank deficiency
You'll sign in first, then come straight back here.
Present Linear Algebra for ML to an Expert Reviewer
2–4 weeks (after milestone 2)
Present your derivations and implementations to a mathematician or ML researcher in a live technical review. The reviewer will pose at least two questions about the mathematical properties of the algorithms you implemented — questions you must answer using the linear algebra concepts, not just code recall.
Proof required
Submit: a recording or transcript of a live review with a mathematician or ML researcher; your derivation documents and implementation repository as the basis for the review; and documentation of at least two questions posed during the review and your responses.
What gets checked
- Review was live — not an async written exchange
- At least two mathematical questions were posed about the algorithms' properties — not just the implementations
- Your responses used linear algebra concepts specifically — not just code-level descriptions
Common mistakes
- Live review that becomes a code walkthrough without mathematical questions
- Responses that describe what the code does rather than using linear algebra concepts to explain the algorithm
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Was the review live — not async?
- Were at least two mathematical questions about algorithm properties posed?
- Did the responses use linear algebra concepts — not just code-level descriptions?
You'll sign in first, then come straight back here.