Derive the Linear Algebra Behind Machine Learning

Proof required

Submit: a typed solution set for at least twenty problems from MIT OCW 18.06 problem sets (or equivalent) with all steps shown; a public GitHub repository (or Colab notebook) containing your SVD implementation using power iteration with a test comparing your output to NumPy's SVD; and a 200-word explanation of why SVD is used in dimensionality reduction. A mathematician, CS lecturer, or ML researcher must confirm the solutions and implementation are correct.

What gets checked

• At least twenty problem set solutions with all derivation steps shown — not just final answers
• SVD implementation produces output matching NumPy's SVD within numerical tolerance on the same input
• A mathematician, CS lecturer, or ML researcher has confirmed the solutions and implementation are correct

Common mistakes

• Using NumPy for the SVD computation — the proof requires implementing power iteration, not calling numpy.linalg.svd
• Problem set solutions with only final answers and no derivation steps — the proof requires showing the work

Resources

What a verifier looks for

• Are at least twenty problem solutions shown with all derivation steps — not just final answers?
• Does the SVD implementation produce output matching NumPy's SVD within numerical tolerance?
• Ask: 'why does PCA use SVD rather than eigendecomposition directly on the data matrix?' — tests deeper understanding

You'll sign in first, then come straight back here.

Proof required

Submit: a typed derivation of the linear regression normal equations (matrix calculus steps, not just the final formula); a typed derivation of the logistic regression cross-entropy gradient; and a public GitHub repository (or Colab) containing your PCA implementation with a comparison to scikit-learn's PCA showing the explained variance ratios match. A mathematician or ML researcher must confirm the derivations are correct.

What gets checked

• Normal equations derivation shows all matrix calculus steps — not just the final (XᵀX)⁻¹Xᵀy formula
• PCA implementation produces explained variance ratios matching scikit-learn's PCA within numerical tolerance
• A mathematician or ML researcher has confirmed the derivations are correct

Common mistakes

• Normal equations derivation that starts from the formula without showing the matrix calculus derivation
• PCA implementation that uses scikit-learn for the core computation — must use SVD directly

Resources

What a verifier looks for

• Does the normal equations derivation show all matrix calculus steps — not just the formula?
• Does the PCA implementation match scikit-learn's explained variance ratios?
• Ask: 'what happens to the normal equations when XᵀX is not invertible?' — tests understanding of rank deficiency

You'll sign in first, then come straight back here.

Proof required

Submit: a recording or transcript of a live review with a mathematician or ML researcher; your derivation documents and implementation repository as the basis for the review; and documentation of at least two questions posed during the review and your responses.

What gets checked

• Review was live — not an async written exchange
• At least two mathematical questions were posed about the algorithms' properties — not just the implementations
• Your responses used linear algebra concepts specifically — not just code-level descriptions

Common mistakes

• Live review that becomes a code walkthrough without mathematical questions
• Responses that describe what the code does rather than using linear algebra concepts to explain the algorithm

Resources

What a verifier looks for

• Was the review live — not async?
• Were at least two mathematical questions about algorithm properties posed?
• Did the responses use linear algebra concepts — not just code-level descriptions?

You'll sign in first, then come straight back here.