Implement a Transformer Component From a Published Paper Specification

Proof required

Submit: a public GitHub repository (or Colab) containing your transformer implementation with at least self-attention, multi-head attention, positional encoding, layer norm, and feed-forward sublayers coded from scratch; a training loss plot showing the model learns on a small task; and a written explanation (300 words) of why multi-head attention produces richer representations than single-head attention. An ML researcher or senior ML engineer must confirm the implementation is architecturally correct.

What gets checked

• Implementation builds the transformer from attention matrices — no HuggingFace Transformers or pre-built attention modules used
• Training loss plot shows learning on a small sequence task — not a flat loss curve
• An ML researcher or senior ML engineer has confirmed the architectural implementation is correct

Common mistakes

• Using HuggingFace Transformers' attention implementation directly — the proof requires implementing attention from matrix operations
• Training on a task where a flat loss curve is expected — must show the model is actually learning

Resources

What a verifier looks for

• Is the attention mechanism implemented from matrix operations — no HuggingFace attention module?
• Does the training loss plot show the model is actually learning on the small task?
• Ask: 'why does positional encoding need to encode position at all?' — tests understanding of the permutation-invariance issue in attention

You'll sign in first, then come straight back here.

Proof required

Submit: a technical analysis (at least 600 words) of the full training pipeline for a specific published LLM (GPT-3, LLaMA 2, or equivalent), citing the original papers; and a worked mathematical example (written, not code) showing how the PPO or DPO objective is computed for a specific token sequence. An ML researcher or senior ML engineer must review and confirm the analysis is technically accurate.

What gets checked

• Analysis cites primary papers — the original GPT-3 or LLaMA 2 paper, not blog summaries
• Worked mathematical example for PPO or DPO is correct — not a high-level description of what the objective does
• An ML researcher has confirmed the analysis is technically accurate

Common mistakes

• Analysis that summarises blog posts without citing primary papers — must engage with the actual methodology sections
• Worked mathematical example that describes the objective without computing it for a specific example

Resources

What a verifier looks for

• Does the analysis cite primary papers — not blog summaries?
• Is the worked mathematical example for PPO or DPO a genuine computation — not a description?
• Ask: 'what problem does RLHF solve that supervised fine-tuning alone does not?' — tests understanding of the alignment stage

You'll sign in first, then come straight back here.

Proof required

Submit: a technical analysis (at least 400 words) of three architectural or training limitations of current LLMs, with proposed solutions and primary citations; and a recording or transcript of a live review session with an ML researcher where they challenged at least two specific claims.

What gets checked

• Analysis addresses architectural or training limitations specifically — not general capability gaps
• At least two proposed solutions cite primary research — not blog descriptions of research
• An ML researcher challenged at least two specific claims in a documented live review

Common mistakes

• Analysis that describes capability limitations ('LLMs can't reason') without connecting to architectural constraints
• Proposed solutions drawn from blog posts rather than primary papers

Resources

What a verifier looks for

• Does the analysis address architectural or training constraints — not general capability gaps?
• Do the proposed solutions cite primary papers?
• Were at least two specific claims challenged in the documented live review?

You'll sign in first, then come straight back here.