Physics Open Data Analysis

Proof required

Submit your physics research question (a specific, quantitative question that the dataset can address), your selected dataset with its source URL, download or access method, and a description of the dataset's physical content (what physical process generated the data, what quantities are measured, what the units are, and the relevant range of parameters).

What gets checked

• Research question is quantitative and specific — not 'analyse the CERN data' but 'test whether the Higgs boson decay products show the expected invariant mass peak in the CERN Dimuon run from the CMS Open Data portal'
• Dataset is from an authoritative open physics source — CERN Open Data, LIGO Open Science Center, NASA, SDSS, or equivalent; not secondary summaries but the primary experimental data
• Physical content is described at the level of the observable measured — what detector, what physical process, what measurement resolution, what systematic effects are documented

Common mistakes

• Selecting a dataset without confirming the data format and analysis tools are compatible — CERN data requires CMS software or ROOT; LIGO data requires GWpy; confirm the tool chain works before selecting the dataset
• Framing a discovery-type question from open data — most physics open data questions are about reproducing known results or applying known techniques to published data, not making new discoveries; frame accordingly

Resources

What a verifier looks for

• Ask the submitter to describe the physical measurement process that generated the data — what detector, what interaction, what the raw signal is before analysis.
• Ask what the known result is for the question they are investigating — most open data physics questions should produce results consistent with published values.
• Ask what systematic effects in the dataset could affect the analysis — each detector and experiment has documented systematic uncertainties.

Proof required

Submit your analysis code (Python, ROOT, or equivalent) with documented pipeline steps, at minimum two figures (a data quality or diagnostic plot and the main result figure), and a quantitative result summary reporting the extracted physical quantity with its statistical and systematic uncertainty.

What gets checked

• Code is submitted and runs without errors — not pseudocode or description but working analysis code
• Statistical and systematic uncertainties are both reported — distinguishing random statistical uncertainty from systematic detector or analysis-method uncertainty is fundamental to experimental physics
• Result is compared against the expected or published value — the comparison must be quantitative (agreement within uncertainty, or specific statement of the discrepancy)

Common mistakes

• Reporting statistical uncertainty only without systematic uncertainty — for any real particle physics or gravitational wave dataset, systematic uncertainties from the detector and analysis choices are typically larger than the statistical uncertainty and must be estimated
• Not documenting the background estimation or selection cuts — physics analysis without documented event selection or background treatment is not reproducible

Resources

What a verifier looks for

• Ask the submitter to explain the background estimation method — how did they distinguish signal from background in the data?
• Ask what would happen to the result if a specific analysis cut was tightened or loosened — tests understanding of how event selection affects signal purity and statistical power.
• Ask what the largest systematic uncertainty is and how it was estimated — quantitative systematic uncertainty estimation is a core experimental physics skill.

Proof required

Submit your complete analysis report (2500–4000 words: introduction with physical context and the known result being reproduced or tested, methods including the full analysis pipeline, results with figures and uncertainty budget, discussion comparing results to published values and interpreting physical significance) plus a Q&A record showing specific challenges to the analysis methodology or physical interpretation and your responses. The reviewer must be named and their experimental physics background stated.

What gets checked

• Results are presented with a complete uncertainty budget — listing all significant statistical and systematic contributions with their estimated magnitude
• Discussion explicitly addresses any deviation from the expected result — not dismissed but analysed in terms of which systematic effect most plausibly explains the discrepancy
• Q&A record shows at least two challenges to the analysis methodology or physical interpretation and substantive responses

Common mistakes

• A report that presents results without comparing to published values — open data physics analysis is almost always replication of known results; the comparison against the known value is the key scientific contribution of the analysis
• A reviewer without experimental data analysis experience in the relevant subfield — someone without knowledge of the detector or analysis methodology cannot probe the systematic uncertainty treatment

Resources

What a verifier looks for

• Ask the submitter to explain the physical significance of their result in one sentence — tests whether they understand the physics, not only the analysis pipeline.
• Ask what the main systematic uncertainty is and what it represents physically — probes understanding of the connection between analysis choices and physics.
• Ask how the analysis would change if a different dataset from the same experiment were used — tests understanding of statistical versus systematic effects.