Milestone map
Milestone map
3 milestones
Design a behavioural research study and pre-register it on OSF
4 weeks
Design a behavioural data collection study on a non-sensitive topic with a non-vulnerable population — options include observational studies of publicly visible behaviour, survey research with anonymous adult participants on topics posing minimal risk, or secondary analysis of a publicly available dataset. Complete a full research design document: research question, theoretical rationale, sampling strategy, data collection method, variables and operationalisation, planned analysis, and ethical considerations. Pre-register the design on OSF (Open Science Framework) before collecting any data.
Proof required
OSF pre-registration link (public or pending) for the study design, plus a research design document (600+ words) covering research question, theoretical rationale, sampling strategy, variables and operationalisation, planned analysis, and ethical considerations.
What gets checked
- OSF pre-registration is submitted before data collection begins — not written retrospectively
- Sampling strategy specifies the target population and why it is appropriate for the research question
- Ethical considerations address data anonymisation, informed consent process, and any potential participant risks
Common mistakes
- Starting data collection before pre-registering — pre-registration is only meaningful before data collection; retroactive pre-registration is not legitimate research design documentation
- Choosing a research question without a theoretical rationale — behavioural research is framed by theory about why the pattern should exist, not just curiosity about whether it does
- Underspecifying the analysis plan — a pre-registration that says 'I will run some statistical tests' does not constrain analytical flexibility
Resources
Foundationstart here
What a verifier looks for
- Access the OSF pre-registration link — confirm it is public or pending, and pre-dates data collection
- Review the sampling strategy — confirm it specifies the target population with justification
- Check the ethical considerations — confirm anonymisation, consent process, and participant risk assessment are addressed
Collect real behavioural data and conduct the pre-registered analysis
5 weeks
Execute the study designed in M1: collect real data from real participants or a publicly available dataset following the pre-registered design. Complete the pre-registered analysis using free statistical tools (Python with pandas and scipy, R, or JASP). Produce a results section documenting what was found: descriptive statistics, inferential test results with effect sizes, and any deviations from the pre-registered analysis plan (with justification). Data and analysis code must be available for inspection.
Proof required
Results document (400+ words) covering descriptive statistics, inferential test results with effect sizes, and any deviations from the pre-registered plan with justification — plus a public OSF link or GitHub repository containing the anonymised data and analysis code.
What gets checked
- Results report effect sizes alongside p-values — statistical significance alone is insufficient
- All deviations from the pre-registered analysis are documented with justification — not silently omitted
- Anonymised data and analysis code are publicly accessible at the submitted link
Common mistakes
- Reporting only statistically significant results and omitting null findings — pre-registration exists precisely to prevent this; the pre-registered analysis must be reported in full
- Not documenting deviations from the pre-registered plan — any analysis not pre-registered must be clearly labelled as exploratory
- Using effect size language without computing effect sizes — Cohen's d, r, or eta-squared must be calculated, not just mentioned
Resources
Foundationstart here
What a verifier looks for
- Access the OSF or GitHub link — confirm anonymised data and analysis code are present and runnable
- Check results report — confirm effect sizes are reported alongside p-values
- Review deviations section — confirm any unplanned analyses are labelled as exploratory
Present findings and methodology to a psychology researcher
1 week
Present the study design, results, and interpretation to a psychology researcher or social scientist who challenges both the methodology and the conclusions. The reviewer must probe specific decisions — 'why this operationalisation of your variable?', 'what alternative explanation does your design not rule out?', 'your effect size is small — is this finding practically meaningful?'. Document the challenges and your responses. This methodology Q&A satisfies the ADVERSARIAL VERIFICATION RULE for social-* outcomes.
Proof required
Q&A notes (250+ words) documenting the reviewer's research background, at least three specific methodological or interpretive challenges, your responses, and any design or analysis changes you would make based on the session.
What gets checked
- Reviewer has research methodology experience in psychology or social science
- At least three specific methodological challenges are documented — not general approval
- At least one design or analysis change is documented based on the Q&A
Common mistakes
- Presenting to someone who validates rather than challenges — the reviewer must engage with the research design specifically
- Not addressing alternative explanations when challenged — a researcher who cannot identify confounds in their own design has not understood the methodology deeply enough
- Ignoring practical significance when challenged — a small but statistically significant effect in a well-powered study is a real result worth interrogating
Resources
What a verifier looks for
- Confirm the reviewer has research methodology experience in psychology or social science
- Review Q&A notes — confirm at least three specific methodological challenges are documented
- Check that at least one design or analysis change is documented based on the session