Cognitive Neuroscience Experimental Design

Proof required

Submit your research question (one specific sentence), the experimental paradigm design (task structure, stimuli, conditions, response measures, and trial structure), a power analysis justifying your target sample size, and a brief section addressing how the paradigm isolates the cognitive function of interest from confounds.

What gets checked

• Research question names a specific cognitive function or process — not 'how does the brain process attention' but 'how does spatial cueing of attention affect response time to lateralised targets'
• Paradigm design specifies the independent variable, dependent variable, and the key control conditions that isolate the cognitive function from confounds
• Power analysis references a prior effect size from the published literature for a similar paradigm — not a generic 'I need 30 participants'

Common mistakes

• Designing a paradigm that measures a cognitive construct too broadly — a good cognitive neuroscience paradigm isolates a specific process through strategic contrasts
• Skipping the control condition — the cognitive neuroscience inference depends on the contrast between conditions, not the response to a single condition

Resources

What a verifier looks for

• Ask the submitter to explain what cognitive function the paradigm isolates and how the control condition achieves that isolation — the logic of the contrast is the core of a cognitive paradigm.
• Ask what confound is hardest to eliminate in this design and how the design minimises it — reveals whether they have thought through what else might explain the expected result.
• Ask how they chose their effect size for the power analysis — the prior effect size should come from a published study using a similar paradigm.

You'll sign in first, then come straight back here.

Proof required

Submit your raw data (anonymised participant data file or extracted dataset), your data preprocessing notes including any exclusions and the justification for each, and the output of your key statistical test with the result stated (test statistic, degrees of freedom, p-value, effect size).

What gets checked

• Data file is anonymised — no participant names or identifiable information in the submitted data
• Exclusions are documented with a stated criterion applied before seeing the data (e.g., 'participants with accuracy below 60% excluded', not 'outliers removed')
• Statistical result is fully reported — test name, statistic, degrees of freedom, p-value, and effect size

Common mistakes

• Post-hoc exclusion criteria — exclusion decisions made after seeing results bias the analysis; criteria must be pre-specified or at minimum documented as exploratory
• Not checking for participant-level quality before running group-level statistics — individual response time distributions, accuracy rates, and attention checks must be reviewed before group tests

Resources

What a verifier looks for

• Ask the submitter to describe their exclusion criteria and confirm they were established before looking at the results — post-hoc exclusions change the analysis outcome.
• Ask what the effect size is and what it means in practical terms — reporting p < .05 without an effect size does not communicate scientific importance.
• Ask whether the result would be different with a Bonferroni or FDR correction if multiple comparisons were run — multiple testing is a common issue in cognitive neuroscience.

You'll sign in first, then come straight back here.

Proof required

Submit your complete research report (3000–4500 words: introduction with cognitive theory background, methods, results with figures, discussion including theoretical implications, and conclusion) plus a Q&A record showing specific challenges from the reviewer and your responses. The reviewer must be named and their cognitive neuroscience or experimental psychology background stated.

What gets checked

• Discussion section connects the findings to a specific cognitive theory or model — not 'this is consistent with attention research' but which specific model or theory and how the result supports, challenges, or refines it
• Q&A record shows at least two specific challenges to the design logic or statistical interpretation and substantive responses
• Limitations section addresses the specific design constraints that limit the generalisability of the conclusion

Common mistakes

• A discussion that treats a null result as a failure rather than as a finding — null results are scientifically valid in a well-powered study and should be discussed as such
• A reviewer from a different branch of neuroscience who does not work with experimental cognitive paradigms — the Q&A requires familiarity with behavioural experiment design

Resources

What a verifier looks for

• Ask the submitter to name the specific cognitive theory their result bears on and how exactly the result relates to it — vague 'consistent with attention research' answers indicate weak theoretical engagement.
• Ask what would have been concluded if the result had gone the other way — confirms the experimental design was theoretically motivated, not just exploratory.
• Verify the reviewer works with experimental cognitive paradigms — a computational or clinical neuroscientist may not be able to challenge the design logic at the appropriate level.

You'll sign in first, then come straight back here.