powstik
Prove
LiveProof-of-Work · Oct–Nov: every verified proof counts. 28 days leftAll outcomes
Skills

Fine-tune a model with LoRA and publish the evaluation

6 weeks · 2 milestones

Fine-tune an open model cheaply, then publish an evaluation anyone can repeat. Research step 3 of the Varsity Curriculum: AI Research. Free resources; your code in a public repo; one person who knows the topic confirms it.

Part of: AI Research · Proves for: Engineers, PG

Milestone map

Milestone map

2 milestones

Fine-tune with LoRA or QLoRA

2–4 weeks (about 8 hrs/week, est)

Build a LoRA or QLoRA adapter for an open-weights model on a task and dataset you choose, with a held-out test set that the model never saw in training. Keep it in one public repo and commit as you go.

Proof required

Link the public repo with the training code and data card, and the published adapter (for example on the Hugging Face Hub) with its licence and the base model it needs.

What gets checked

  • The held-out test set is separate from training data, and how was checked is written down.
  • The adapter is published with its licence and the base model named.
  • Training code and settings let someone else repeat it.

Common mistakes

  • One big upload with no commit history.
  • Results that the repo cannot reproduce.

Powstik Guide

A public repo with a LoRA or QLoRA adapter for an open-weights model on a task and dataset you choose, with a held-out test set that the model never saw in training.

Steps

  1. Make a public repo.
  2. Write the tests or the evaluation first.
  3. Build the smallest version that passes them.
  4. Write a README: what it is and how to reproduce the results.
  5. Ask someone who knows the topic to check it.

Template

README
Fine-tune a model with LoRA and publish the evaluation
What is here:
How to reproduce:
Results:
Limits:

What gets sent back

  • No tests or evaluation.
  • Code that does not work from the README.
  • Copied code with no history.

Resources

Foundationstart here

What a verifier looks for

  • The held-out test set is separate from training data, and how was checked is written down.
  • The adapter is published with its licence and the base model named.
  • Training code and settings let someone else repeat it.
  • Open the repo: does the code match the results shown?

You'll sign in first, then come straight back here.

Publish an evaluation anyone can repeat

1–2 weeks (about 6 hrs/week, est)

Write up an evaluation of the base model and your adapter on the held-out set (and one standard benchmark through an evaluation harness), with the exact commands, scores and failure examples. Put the report in the repo next to the code that produced it.

Proof required

Link the report in your public repo, with the plots and the code or commands that made them.

What gets checked

  • Base and fine-tuned scores are reported on the same set.
  • The exact commands to reproduce the scores are given.
  • Failure cases are shown, not only wins.

Common mistakes

  • Claims the plots do not support.
  • A single seed reported as a result.

Powstik Guide

A short report in the repo with plots and the commands that made them.

Steps

  1. State the question in one line.
  2. Say what you held fixed and what you changed.
  3. Show the plot with the spread over seeds.
  4. Write what it shows, and what it does not.

Template

Report
Question:
Setup (fixed / changed):
Results (plot, seeds):
What it shows:
What it does not show:
Next:

What gets sent back

  • No plot.
  • No statement of what was held fixed.

Resources

Depthgo deeper

What a verifier looks for

  • Base and fine-tuned scores are reported on the same set.
  • The exact commands to reproduce the scores are given.
  • Failure cases are shown, not only wins.
  • Is the conclusion honest about what was not tested?

You'll sign in first, then come straight back here.

Part of