Milestone map
Milestone map
3 milestones
Conduct the pre-registered analysis and interpret results
4 weeks
Conduct the pre-registered analysis on the collected data following the OSF pre-registration. Report results in APA format: descriptive statistics, primary analysis results (with effect sizes and confidence intervals, not just p-values), and any pre-registered secondary analyses. If the results differ from hypotheses, address this directly — null results and unexpected findings are valid research outcomes. Include a discussion section interpreting the findings in relation to the theoretical rationale stated in the pre-registration.
Proof required
Results and discussion document (700+ words) in APA format covering: descriptive statistics, primary analysis results with effect sizes and confidence intervals, discussion of findings in relation to pre-registered hypotheses, and direct address of null or unexpected results if present.
What gets checked
- Results reported with effect sizes and confidence intervals — not just p-values
- Null or unexpected results are addressed directly — not silently omitted
- Discussion interprets findings in relation to the pre-registered theoretical rationale
Common mistakes
- Reporting only statistically significant results and treating null results as failures — null results are valid scientific findings that must be reported
- Reporting p-values without effect sizes — effect sizes are required for meaningful interpretation
- Discussion that reaches beyond the data — conclusions must be grounded in what the analysis actually shows
Resources
Foundationstart here
What a verifier looks for
- Check results reporting — confirm effect sizes and confidence intervals are present alongside p-values
- Review null/unexpected results — confirm they are addressed directly if present
- Check discussion — confirm it is grounded in the actual analysis results
Write a full research report and identify limitations and future directions
4 weeks
Write a complete APA-format research report: abstract, introduction (theoretical rationale + literature review), methods, results, and discussion — including limitations section and future directions. The limitations section must address specific limitations of this study's design and sample — not generic psychology research limitations. The future directions must be grounded in the specific gaps this study created or highlighted.
Proof required
Full research report in APA format (2000+ words) covering all sections including a limitations section addressing at least three specific limitations of this study's design and sample, and a future directions section grounded in the specific gaps this study created or highlighted.
What gets checked
- Report is in APA format with all standard sections present
- Limitations section addresses at least three specific limitations — not generic research limitations
- Future directions are grounded in this study's specific gaps — not generic calls for more research
Common mistakes
- Generic limitations section — 'sample was not representative' is only a limitation if explained specifically for this study's sample and design
- Future directions that are generic — 'further research is needed' is not a future direction
- Discussion that does not engage with the pre-registered hypotheses — the pre-registration is the framework for interpreting results
Resources
What a verifier looks for
- Check APA format — confirm all standard sections are present
- Review limitations — confirm at least three are specific to this study's design and sample
- Check future directions — confirm they are grounded in this study's specific gaps
Defend research report under psychologist or researcher peer review
1 week
Submit the research report for peer review by a registered psychologist, psychology academic, or quantitative methods researcher. The reviewer must challenge methods and interpretation — 'your effect size is small enough that the practical significance is questionable — how do you address that?', 'your limitations section identifies selection bias but your discussion doesn't address how it affects conclusions — why not?'. Peer-only verification is never sufficient for this outcome. This Q&A satisfies the ADVERSARIAL VERIFICATION RULE.
Proof required
Q&A notes (250+ words) documenting the reviewer's credentials (registered psychologist, psychology academic, or quantitative methods researcher), at least three specific challenges to methods or interpretation with responses, and any report revisions committed to.
What gets checked
- Reviewer is a registered psychologist, psychology academic, or quantitative methods researcher — peer verification is not acceptable
- At least three specific challenges to methods or interpretation are documented
- At least one report revision is committed to based on the review
Common mistakes
- Using a peer verifier instead of a registered psychologist or psychology researcher
- Not being able to explain why a specific limitation does not invalidate a key conclusion under challenge
- No report revisions after a genuine peer review challenge
Resources
What a verifier looks for
- Confirm reviewer is a registered psychologist, psychology academic, or quantitative methods researcher
- Review Q&A notes — confirm at least three specific challenges to methods or interpretation are documented
- Check that at least one report revision is committed to