Milestone map
Milestone map
3 milestones
Develop a research question and design a quantitative or mixed-methods study
4 weeks
Develop a specific psychology research question — not a broad topic but a testable hypothesis or research question. Design a study to answer it: specify the design (experimental, quasi-experimental, correlational, cross-sectional, longitudinal), the sampling strategy, the measures or instruments to be used (existing validated measures only — not bespoke), the procedure, and the planned analysis. Pre-register the design on the Open Science Framework (OSF) before any data collection begins. Pre-registration is required — not optional.
Proof required
Research design document (600+ words) with OSF pre-registration link, covering: research question with theoretical rationale, study design, sampling strategy, validated measures with citations, procedure, planned analysis, and ethical considerations — all documented before data collection.
What gets checked
- Research question is specific and testable — not a broad topic
- OSF pre-registration link is present and timestamped before any data collection
- Validated measures are cited — not bespoke measures developed for this study
Common mistakes
- Designing the study after data collection and treating the pre-registration as documentation — pre-registration must occur before data collection
- Using self-developed bespoke measures without validation — psychology research uses validated instruments for replicability
- Ethical considerations section that does not address the specific ethical issues in this study's design
Resources
Foundationstart here
What a verifier looks for
- Confirm the OSF pre-registration link is present and access it — verify it is timestamped before data collection
- Check validated measures — confirm each is cited to the original validation study
- Review ethical considerations — confirm they address the specific issues in this study's design
Collect data from real participants and document the procedure
5 weeks
Collect data from real participants following the pre-registered procedure. All participants must provide informed consent. Document the data collection: number of participants (with any deviations from pre-registered target and reasoning), any protocol deviations from the pre-registered procedure with reasoning, and a data quality check. Data and materials must be deposited on OSF or a public repository after collection for reproducibility. The CLINICAL CONTENT SAFETY RULE applies: do NOT collect data on clinical populations without appropriate ethical approval and supervised clinical oversight.
Proof required
Data collection document (400+ words) covering: consent procedure used, actual sample with any deviations from pre-registered target, any protocol deviations with reasoning, data quality check results, and OSF or repository link for deposited data and materials.
What gets checked
- Consent procedure is documented — not assumed
- Any deviations from the pre-registered protocol are documented with reasoning — not silently incorporated
- Data and materials are deposited in a public repository with an accessible link
Common mistakes
- Not documenting protocol deviations — any deviation from the pre-registered procedure must be recorded
- Collecting data on clinical populations without ethical approval and supervised oversight — this violates the CLINICAL CONTENT SAFETY RULE
- Not depositing data and materials — reproducibility requires public access to data and materials
Resources
What a verifier looks for
- Confirm consent procedure is documented
- Check for protocol deviations — confirm any are documented with reasoning
- Access the repository link — confirm data and materials are deposited
Defend research design and data collection under psychologist or researcher challenge
1 week
Present the pre-registered research design and data collection to a registered psychologist, psychology academic, or quantitative methods researcher for a structured Q&A. The reviewer must challenge design choices and data collection — 'your sampling strategy — how does selection bias affect generalizability?', 'your pre-registered analysis — you deviated here — how does that affect your conclusions?'. Peer-only verification is never sufficient for this outcome. This Q&A satisfies the ADVERSARIAL VERIFICATION RULE.
Proof required
Q&A notes (250+ words) documenting the reviewer's credentials (registered psychologist, psychology academic, or quantitative methods researcher), at least three specific challenges to design or data collection with responses, and any analysis updates committed to.
What gets checked
- Reviewer is a registered psychologist, psychology academic, or quantitative methods researcher — peer verification is not acceptable
- At least three specific design or data challenges are documented
- At least one analysis update is committed to based on the session
Common mistakes
- Using a peer verifier instead of a registered psychologist or psychology researcher
- Not being able to explain deviations from the pre-registered protocol under challenge
- No analysis updates after a genuine design or data quality challenge
Resources
What a verifier looks for
- Confirm reviewer is a registered psychologist, psychology academic, or quantitative methods researcher
- Review Q&A notes — confirm at least three specific design or data challenges are documented
- Check that at least one analysis update is committed to