powstik
Prove
LiveProof-of-Work · Oct–Nov: every verified proof counts. 28 days leftAll outcomes
Skills

Train a small GPT and explain its loss curves

6 weeks · 2 milestones

Train two model sizes, read the curves, and say what changed and why. Research step 2 of the Varsity Curriculum: AI Research. Free resources; your code in a public repo; one person who knows the topic confirms it.

Part of: AI Research · Proves for: Engineers, PG

Milestone map

Milestone map

2 milestones

Train two sizes of a small GPT

2–4 weeks (about 8 hrs/week, est)

Build a small GPT trained from scratch at two sizes (for example about 1M and 10M parameters) on the same corpus, logging training and validation loss, tokens seen and compute. Keep it in one public repo and commit as you go.

Proof required

Link the public repo with the training code, the config for each size, the logged train and validation loss curves, and samples from each model.

What gets checked

  • Both sizes train from the README with the stated configs.
  • Train and validation loss are both logged and plotted.
  • Compute (tokens or FLOPs) is recorded for each run.

Common mistakes

  • One big upload with no commit history.
  • Results that the repo cannot reproduce.

Powstik Guide

A public repo with a small GPT trained from scratch at two sizes (for example about 1M and 10M parameters) on the same corpus, logging training and validation loss, tokens seen and compute.

Steps

  1. Make a public repo.
  2. Write the tests or the evaluation first.
  3. Build the smallest version that passes them.
  4. Write a README: what it is and how to reproduce the results.
  5. Ask someone who knows the topic to check it.

Template

README
Train a small GPT and explain its loss curves
What is here:
How to reproduce:
Results:
Limits:

What gets sent back

  • No tests or evaluation.
  • Code that does not work from the README.
  • Copied code with no history.

Resources

Foundationstart here

What a verifier looks for

  • Both sizes train from the README with the stated configs.
  • Train and validation loss are both logged and plotted.
  • Compute (tokens or FLOPs) is recorded for each run.
  • Open the repo: does the code match the results shown?

You'll sign in first, then come straight back here.

Write up what the curves show

1–2 weeks (about 6 hrs/week, est)

Write up a short report comparing the two sizes: loss against compute, where overfitting starts, and one change you tried (data, context length or learning rate) and its effect. Put the report in the repo next to the code that produced it.

Proof required

Link the report in your public repo, with the plots and the code or commands that made them.

What gets checked

  • Loss is plotted against compute, not only against steps.
  • Overfitting is identified from the validation curve.
  • One controlled change is reported with its effect.

Common mistakes

  • Claims the plots do not support.
  • A single seed reported as a result.

Powstik Guide

A short report in the repo with plots and the commands that made them.

Steps

  1. State the question in one line.
  2. Say what you held fixed and what you changed.
  3. Show the plot with the spread over seeds.
  4. Write what it shows, and what it does not.

Template

Report
Question:
Setup (fixed / changed):
Results (plot, seeds):
What it shows:
What it does not show:
Next:

What gets sent back

  • No plot.
  • No statement of what was held fixed.

Resources

Depthgo deeper

What a verifier looks for

  • Loss is plotted against compute, not only against steps.
  • Overfitting is identified from the validation curve.
  • One controlled change is reported with its effect.
  • Is the conclusion honest about what was not tested?

You'll sign in first, then come straight back here.

Part of