Milestone map
Milestone map
3 milestones
Identify a real research question and select a suitable econometric model
4 weeks
Select a real research question that can be addressed with econometric methods using a publicly available dataset — questions about the effect of a policy on an outcome, the relationship between economic variables, or the determinants of a social or economic outcome. Choose a dataset (World Bank Open Data, FRED, OECD.Stat, or similar), formulate a testable hypothesis, and specify the econometric model: OLS, IV, difference-in-differences, panel data, or logit/probit. Document why this model is appropriate for the question and what identification assumptions it relies on.
Proof required
Research design document (500+ words) covering the research question, dataset source and key variables, econometric model specification, justification for the model choice, and identification assumptions with a discussion of whether they are likely to hold.
What gets checked
- Research question is specific enough to produce a falsifiable empirical result
- Model choice is justified by the structure of the question and the data — not chosen because it is familiar
- Identification assumptions are stated specifically — not just 'OLS assumptions hold'
Common mistakes
- Choosing a model and then finding a question to fit it — the question drives the model, not the other way round
- Claiming OLS is appropriate without discussing endogeneity — most social-science research questions have endogeneity concerns that require IV, DiD, or panel methods
- Not discussing the data source's limitations — every publicly available dataset has measurement and coverage limitations relevant to the research question
Resources
Foundationstart here
Depthgo deeper
What a verifier looks for
- Confirm the dataset is publicly available — request the source link
- Review the model justification — confirm it addresses why this model is appropriate for the question structure
- Check identification assumptions — confirm they are stated specifically for this model and data
Estimate the model and interpret results with diagnostics
5 weeks
Estimate the specified econometric model using Python (statsmodels, linearmodels), R, or STATA on the dataset from M1. Run appropriate diagnostic tests (heteroscedasticity, autocorrelation for time series, multicollinearity for OLS, instrument strength for IV). Interpret the results: report coefficient estimates with standard errors and confidence intervals, discuss practical significance alongside statistical significance, and state what the results imply for the research question.
Proof required
Results document (500+ words) covering coefficient estimates with standard errors and confidence intervals, diagnostic test results with interpretation, practical significance discussion, and the data and code available in a public repository (GitHub, OSF, or similar).
What gets checked
- Coefficient estimates are reported with standard errors and confidence intervals — not just p-values
- Diagnostic tests are run and interpreted — not just mentioned as passed
- Practical significance is discussed: is the effect size economically meaningful, not just statistically significant?
Common mistakes
- Reporting p-values without effect sizes — econometric results are often statistically significant but economically trivial; both must be addressed
- Running diagnostics without interpreting failures — finding heteroscedasticity and doing nothing about it undermines the inference
- Not making code and data publicly available — reproducibility is a basic standard for quantitative social science
Resources
Foundationstart here
What a verifier looks for
- Access the code repository — confirm the estimation code is present and runs on the dataset
- Review diagnostic test results — confirm they are interpreted, not just listed
- Check the practical significance discussion — confirm it addresses economic meaningfulness alongside statistical significance
Defend methodology and results under economist challenge
1 week
Present the econometric analysis to an economist or applied researcher for a methodology challenge. The reviewer must probe identification — 'your instrument — why is it truly exogenous?', 'you're claiming a causal effect but your model doesn't rule out reverse causality here', 'your sample period — do these results hold in different periods?'. Document the challenges and your responses. This Q&A satisfies the ADVERSARIAL VERIFICATION RULE for social-* outcomes.
Proof required
Q&A notes (250+ words) documenting the reviewer's credentials (economist or applied researcher), at least three specific identification or methodology challenges, your responses, and any analysis updates committed to based on the session.
What gets checked
- Reviewer is an economist or applied researcher with econometrics experience
- At least three specific identification or methodology challenges are documented
- At least one analysis update is committed to based on the session
Common mistakes
- Choosing a reviewer who approves the approach without probing identification assumptions — identification is where most applied econometrics breaks down
- Not being able to defend the instrument choice or causal claim under challenge — if the identification assumption cannot be defended in real time, the causal claim was unjustified
- Not updating any analysis based on the challenge — a legitimate identification critique requires a response
Resources
What a verifier looks for
- Confirm the reviewer is an economist or applied researcher with econometrics experience
- Review Q&A notes — confirm at least three identification or methodology challenges are documented
- Check that at least one analysis update is documented based on the session
Part of