Skills

Psychology 12: Capstone — a preregistered replication

10 weeks · 3 milestones

Replicate a classic finding: preregister it, run it on open data or an anonymous survey of consenting adults, publish the code, data and write-up, and defend it. The last step of the Varsity Curriculum: Psychology, read by a blind review for Honors. Foundations only. Not a clinical or counselling qualification.

Part of: Psychology · Proves for: UG

Milestone map

3 milestones

Preregister the replication

3 weeks (about 8 hrs/week)

Pick a classic finding with open materials. Plan a close replication using open data (for example Many Labs) or an anonymous online survey of consenting adults. Preregister the hypothesis, sample size and exact analysis on OSF or AsPredicted before you collect or open the data. If you collect data from people: adults only (18+), consent first, answers anonymous, and no health or clinical questions. Under 18? Use an open dataset; never collect data yourself.

Proof required

Link the time-stamped preregistration and the public repo with your materials (experiment code or the open dataset's link) and the consent text, if you collect data.

What gets checked

The preregistration is dated before the data.
The sample size has a power analysis.
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

Common mistakes

Preregistering after looking at the data.
A plan too vague to check.

Powstik Guide

A dated preregistration and a public repo with the materials.

Steps

Pick the finding and read the original paper.
Choose open data or an anonymous adult survey.
Write the plan and power analysis.
Preregister it, then start.

Template

Preregistration
Finding and original paper:
Hypothesis:
Design:
Sample size and power analysis:
Exact analysis:
Data rules (exclusions):

What gets sent back

No date before the data.
Data from minors or with health questions.

Resources

Foundationstart here

Depthgo deeper

What a verifier looks for

Is the preregistration dated before the data?
Could someone else run the analysis from it?
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

You'll sign in first, then come straight back here.

Run it and analyse it as planned

4 weeks (about 8 hrs/week)

Collect the data or download it, then run exactly the analysis you preregistered. Anything extra is labelled exploratory. If you collect data from people: adults only (18+), consent first, answers anonymous, and no health or clinical questions. Under 18? Use an open dataset; never collect data yourself.

Proof required

Link the public repo with the anonymous data, the analysis code, the output, and a note listing every change from the plan.

What gets checked

The planned analysis is run first and reported in full.
Every deviation from the plan is listed.
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

Common mistakes

Dropping the planned test because it failed.
Exploratory results presented as planned.

Powstik Guide

Anonymous data, analysis code and output in a public repo.

Steps

Collect or download the data.
Run the preregistered analysis.
List every change from the plan.
Label anything extra as exploratory.

Template

Results
Planned test and result:
Effect size and interval:
Deviations from the plan:
Exploratory (labelled):

What gets sent back

Analysis that differs from the plan without saying so.
Identifiable data.

Resources

Foundationstart here

Depthgo deeper

What a verifier looks for

Does the code reproduce the reported numbers?
Are deviations listed?
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

You'll sign in first, then come straight back here.

Write it up in public and defend it

3 weeks (about 8 hrs/week)

Write the replication up (introduction, method, results, discussion) and publish it with the code and data. Then defend it for 15 minutes to at least three people who know psychology. This is what the Honors blind review reads.

Proof required

Link the public write-up with the repo and the recording of the defence, and name the people who questioned you.

What gets checked

The write-up says plainly whether the finding replicated.
Code, data and preregistration are linked from it.
The defence answers questions about the method.

Common mistakes

Spinning a failed replication as a success.
A write-up with no data or code.

Powstik Guide

A public write-up with code, data and preregistration linked, and a recorded 15-minute defence.

Steps

Write it up in the standard order.
Link the preregistration, code and data.
Invite three people who know psychology.
Record the defence and its questions.

Template

Write-up
Title:
Did it replicate (yes, no, partly):
Method:
Results:
What this means and what it doesn't:

What gets sent back

No preregistration linked.
No defence recording.

Resources

Foundationstart here

What a verifier looks for

Does the conclusion follow from the planned test?
Is everything needed to repeat it public?
Were hard questions answered honestly?

You'll sign in first, then come straight back here.

Part of

Psychology 12: Capstone →

Related outcomes

Milestone map

3 milestones

Preregister the replication

3 weeks (about 8 hrs/week)

Proof required

Link the time-stamped preregistration and the public repo with your materials (experiment code or the open dataset's link) and the consent text, if you collect data.

What gets checked

The preregistration is dated before the data.
The sample size has a power analysis.
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

Common mistakes

Preregistering after looking at the data.
A plan too vague to check.

Powstik Guide

A dated preregistration and a public repo with the materials.

Steps

Pick the finding and read the original paper.
Choose open data or an anonymous adult survey.
Write the plan and power analysis.
Preregister it, then start.

Template

Preregistration
Finding and original paper:
Hypothesis:
Design:
Sample size and power analysis:
Exact analysis:
Data rules (exclusions):

What gets sent back

No date before the data.
Data from minors or with health questions.

Resources

Foundationstart here

Depthgo deeper

What a verifier looks for

Is the preregistration dated before the data?
Could someone else run the analysis from it?
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

You'll sign in first, then come straight back here.

Run it and analyse it as planned

4 weeks (about 8 hrs/week)

Proof required

Link the public repo with the anonymous data, the analysis code, the output, and a note listing every change from the plan.

What gets checked

The planned analysis is run first and reported in full.
Every deviation from the plan is listed.
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

Common mistakes

Dropping the planned test because it failed.
Exploratory results presented as planned.

Powstik Guide

Anonymous data, analysis code and output in a public repo.

Steps

Collect or download the data.
Run the preregistered analysis.
List every change from the plan.
Label anything extra as exploratory.

Template

Results
Planned test and result:
Effect size and interval:
Deviations from the plan:
Exploratory (labelled):

What gets sent back

Analysis that differs from the plan without saying so.
Identifiable data.

Resources

Foundationstart here

Depthgo deeper

What a verifier looks for

Does the code reproduce the reported numbers?
Are deviations listed?
Data from people follows the rules: adults, consent, anonymous, no health or clinical data.

You'll sign in first, then come straight back here.

Write it up in public and defend it

3 weeks (about 8 hrs/week)

Proof required

Link the public write-up with the repo and the recording of the defence, and name the people who questioned you.

What gets checked

The write-up says plainly whether the finding replicated.
Code, data and preregistration are linked from it.
The defence answers questions about the method.

Common mistakes

Spinning a failed replication as a success.
A write-up with no data or code.

Powstik Guide

A public write-up with code, data and preregistration linked, and a recorded 15-minute defence.

Steps

Write it up in the standard order.
Link the preregistration, code and data.
Invite three people who know psychology.
Record the defence and its questions.

Template

Write-up
Title:
Did it replicate (yes, no, partly):
Method:
Results:
What this means and what it doesn't:

What gets sent back

No preregistration linked.
No defence recording.

Resources

Foundationstart here

What a verifier looks for

Does the conclusion follow from the planned test?
Is everything needed to repeat it public?
Were hard questions answered honestly?

You'll sign in first, then come straight back here.