Milestone map
Milestone map
3 milestones
Specify cognitive question and design experimental paradigm
2–3 weeks
Select a specific cognitive neuroscience research question and design an experimental paradigm to address it. Accessible alternative: if neuroimaging equipment is unavailable, design a behavioural paradigm (no specialist equipment needed — online tools such as jsPsych or Gorilla run in a browser) or design an analysis of an existing open neuroimaging dataset from OpenNeuro. Human participant ethics approval is not required for this design milestone, but must be addressed if M2 involves real data collection.
Proof required
Submit your research question (one specific sentence), the experimental paradigm design (task structure, stimuli, conditions, response measures, and trial structure), a power analysis justifying your target sample size, and a brief section addressing how the paradigm isolates the cognitive function of interest from confounds.
What gets checked
- Research question names a specific cognitive function or process — not 'how does the brain process attention' but 'how does spatial cueing of attention affect response time to lateralised targets'
- Paradigm design specifies the independent variable, dependent variable, and the key control conditions that isolate the cognitive function from confounds
- Power analysis references a prior effect size from the published literature for a similar paradigm — not a generic 'I need 30 participants'
Common mistakes
- Designing a paradigm that measures a cognitive construct too broadly — a good cognitive neuroscience paradigm isolates a specific process through strategic contrasts
- Skipping the control condition — the cognitive neuroscience inference depends on the contrast between conditions, not the response to a single condition
Resources
Foundationstart here
Depthgo deeper
Masteryfor the dedicated
What a verifier looks for
- Ask the submitter to explain what cognitive function the paradigm isolates and how the control condition achieves that isolation — the logic of the contrast is the core of a cognitive paradigm.
- Ask what confound is hardest to eliminate in this design and how the design minimises it — reveals whether they have thought through what else might explain the expected result.
- Ask how they chose their effect size for the power analysis — the prior effect size should come from a published study using a similar paradigm.
Collect or extract data and run preliminary analysis
2–6 weeks depending on data collection or dataset extraction
Execute the data collection phase following your designed paradigm (behavioural study: n≥20 participants) or extract and pre-process data from an open neuroimaging dataset on OpenNeuro. Document all deviations from the design and run the key statistical tests. Ethics: if collecting real participant data, ensure appropriate informed consent is in place — a brief information sheet and consent form are required. Exempt categories (no vulnerable populations, no deception, no sensitive data) can typically be approved via an institutional low-risk review.
Proof required
Submit your raw data (anonymised participant data file or extracted dataset), your data preprocessing notes including any exclusions and the justification for each, and the output of your key statistical test with the result stated (test statistic, degrees of freedom, p-value, effect size).
What gets checked
- Data file is anonymised — no participant names or identifiable information in the submitted data
- Exclusions are documented with a stated criterion applied before seeing the data (e.g., 'participants with accuracy below 60% excluded', not 'outliers removed')
- Statistical result is fully reported — test name, statistic, degrees of freedom, p-value, and effect size
Common mistakes
- Post-hoc exclusion criteria — exclusion decisions made after seeing results bias the analysis; criteria must be pre-specified or at minimum documented as exploratory
- Not checking for participant-level quality before running group-level statistics — individual response time distributions, accuracy rates, and attention checks must be reviewed before group tests
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Ask the submitter to describe their exclusion criteria and confirm they were established before looking at the results — post-hoc exclusions change the analysis outcome.
- Ask what the effect size is and what it means in practical terms — reporting p < .05 without an effect size does not communicate scientific importance.
- Ask whether the result would be different with a Bonferroni or FDR correction if multiple comparisons were run — multiple testing is a common issue in cognitive neuroscience.
Write report and present with Q&A
1–2 weeks to write and schedule review
Complete a research report connecting the findings to cognitive neuroscience theory and presenting the implications for understanding the cognitive function under study. Present to a cognitive neuroscientist or experimental psychologist for a Q&A that probes the design logic, the statistical interpretation, and the connection between the results and the theoretical conclusions.
Proof required
Submit your complete research report (3000–4500 words: introduction with cognitive theory background, methods, results with figures, discussion including theoretical implications, and conclusion) plus a Q&A record showing specific challenges from the reviewer and your responses. The reviewer must be named and their cognitive neuroscience or experimental psychology background stated.
What gets checked
- Discussion section connects the findings to a specific cognitive theory or model — not 'this is consistent with attention research' but which specific model or theory and how the result supports, challenges, or refines it
- Q&A record shows at least two specific challenges to the design logic or statistical interpretation and substantive responses
- Limitations section addresses the specific design constraints that limit the generalisability of the conclusion
Common mistakes
- A discussion that treats a null result as a failure rather than as a finding — null results are scientifically valid in a well-powered study and should be discussed as such
- A reviewer from a different branch of neuroscience who does not work with experimental cognitive paradigms — the Q&A requires familiarity with behavioural experiment design
Resources
Depthgo deeper
What a verifier looks for
- Ask the submitter to name the specific cognitive theory their result bears on and how exactly the result relates to it — vague 'consistent with attention research' answers indicate weak theoretical engagement.
- Ask what would have been concluded if the result had gone the other way — confirms the experimental design was theoretically motivated, not just exploratory.
- Verify the reviewer works with experimental cognitive paradigms — a computational or clinical neuroscientist may not be able to challenge the design logic at the appropriate level.
Part of