Derive the Linear Algebra Behind Machine Learning
8 weeks · 0 milestones
Produce hand-derived solutions to 20 exercises spanning matrix operations (multiplication, inversion, transposition), eigendecomposition, singular value decomposition, and PCA derivation — all derivations must be symbolic or hand-worked (no NumPy for the derivation steps, only for verification afterwards). The work must show the derivation reasoning, not just the result. Proof: the solutions reviewed by a maths lecturer or ML researcher who presents 2–3 unseen problems during the review session — you must work through them live and explain your reasoning at each step, not just produce an answer.
Milestone map
Milestone map
3 milestones
Study the foundations of linear algebra for machine learning: vector spaces, matrix operations, linear transformations, eigenvectors and eigenvalues, and matrix decompositions (LU, QR, SVD). Complete at least twenty exercises from MIT OCW 18.06 or equivalent, working through the derivations — not just applying formulas. Implement SVD from scratch using the power iteration method.
Proof required
Submit: a typed solution set for at least twenty problems from MIT OCW 18.06 problem sets (or equivalent) with all steps shown; a public GitHub repository (or Colab notebook) containing your SVD implementation using power iteration with a test comparing your output to NumPy's SVD; and a 200-word explanation of why SVD is used in dimensionality reduction. A mathematician, CS lecturer, or ML researcher must confirm the solutions and implementation are correct.
What gets checked
- At least twenty problem set solutions with all derivation steps shown — not just final answers
- SVD implementation produces output matching NumPy's SVD within numerical tolerance on the same input
- A mathematician, CS lecturer, or ML researcher has confirmed the solutions and implementation are correct