The overall aim of this hackathon is for participants to learn the conceptual and practical aspects of using counterfactual analysis to identify ‘active ingredients’ in order to address wicked problems using observational data. ‘Counterfactual analysis’ refers to a family of methods that adjust for the potential impact of confounding variables in the association between an active ingredient and an outcome of interest. It does this by matching (or weighting) individuals based on confounding variables who have been exposed to different levels of a candidate active ingredient. When successfully matched (or weighted), groups with different levels of ‘exposure’ can be compared on their outcomes and any differences in those outcomes can be more confidently attributed to the active ingredient itself.
Many of the big societal challenges do not lend themselves to experimental manipulation or randomised controlled trials, making it difficult to establish causal effects. Large-scale data collection efforts (Generation Scotland, UK Household Longitudinal Study, UK Biobank, Millennium Cohort Study, etc.) and social media (reddit, Twitter/X, etc.) provide unprecedented volumes of observational data, but these observational data are vulnerable to reverse causation (the presumed outcome might actually be the cause), misidentification or confounding (the presumed cause or both cause and outcome might be correlated with a third variable that is responsible for their association),and social selection (individuals choose and shape their exposures in ways that align with their dispositions). Counterfactual analysis provides a way to mitigate these vulnerabilities and use observational data to identify’ active ingredients’ — the levers that may have the power to make a positive change within the interconnected machinery of society and culture.
The principles of counterfactual analysis are not new, but the approach is not widely known and can be difficult to implement. Our team has recently developed DigiCAT: a free, easy-to-use tool for implementing counterfactual analysis following best practices. This hackathon will provide participants with both a conceptual foundation for using counterfactual analysis and practical experience in implementing it using DigiCAT to address a research question. It will be, an in-person (on the University of Edinburgh central campus) highly interactive workshop combining direct instruction, semi-structured discussion, time for students to work on their projects, and coffee/tea breaks to reset and socialise with the other hackathon participants.
Advance preparation: See instructions for suggested background readings and preparatory work
Tentative schedule:
• 9:30-10:00: coffee/tea, introductions
• 10:00-12:00: overview of counterfactual analysis
• 12:00-13:30: lunch (provided) and informal discussion/brainstorming about research topics and data sets; form groups for hackathon
• 13:30-15:30: hacking! (students work in groups on a counterfactual analysis project)
• 15:30-16:00: coffee/tea break
• 16:00-17:00: wrap-up — share findings, reflect on experience, plan next steps
Approximately 4 months after the hackathon, we will hold an online follow-up session where project teams will be invited to present outputs from the research projects that began at the hackathon and receive additional support in continuing to use counterfactual analysis.