A principal advantage of using more than two treatment conditions in an experiment is that this

A Framework for Causal Inference – Basic Building Blocks

The framework for causal inference that is discussed here is now commonly referred to as the Rubin Causal Model (RCM; Holland, 1986), for a series of articles written in the 1970s (Rubin, 1974, 1976, 1977, 1978, 1980). Other approaches to causal inference, such as graphical ones (e.g., Pearl, 2000), are conceptually less satisfying, for reasons discussed, for instance, in Rubin (2004b, 2005). The presentation here is essentially a brief and relatively nontechnical version of that given in Rubin (2006).

For causal inference, there are several basic building blocks. A unit is a physical object, for example, a person, at a particular point in time. A treatment is an action that can be applied or withheld from that unit. We focus on the case of two treatments, although the extension to more than two treatments is simple in principle but not necessarily so with real data. Associated with each unit are two potential outcomes: the value of an outcome variable Y (e.g., test score) at a point in time t when the active treatment (e.g., new educational program) is used at an earlier time t0, and the value of Y at time t when the control educational program is used at t0. The objective is to learn about the causal effect of the application of the active treatment relative to the control (treatment) on Y. Formal notation for this meaning of a causal effect first appeared in Neyman (1923) in the context of randomization-based inference in randomized experiments. Let W indicate which treatment the unit received: W = 1 the active treatment, W = 0 the control treatment. Moreover, let Y(1) be the value of Y if the unit received the active version, and Y(0) the value if the unit received the control version. The causal effect of the active treatment relative to its control version is the comparison of Y(1) and Y(0) – typically the difference, Y(1) – Y(0), or perhaps the difference in logs, log[Y(1)] – log[Y(0)], or some other comparison, possibly the ratio. The fundamental problem for causal inference is that, for any individual unit, we can observe only one of Y(1) or Y(0), as indicated by W; that is, we observe the value of the potential outcome under only one of the possible treatments, namely the treatment actually assigned, and the potential outcome under the other treatment is missing. Thus, inference for causal effects is a missing-data problem – the “other” value is missing. Of importance in educational research, the gain score for a unit, posttest minus pretest, measures a change in time, and so is not a causal effect.

We learn about causal effects using replication, which involves the use of more than one unit. The way we personally learn from our own experience is replication involving the same physical object (me or you) with more units in time, thereby having some observations of Y(0) and some of Y(1). When we want to generalize to units other than ourselves, we typically use more objects; that is what is done in social science experiments, for example, involving students and possible educational interventions, such as value-added assessment (e.g., Rubin et al., 2004). Replication does not help without additional assumptions. The most straightforward assumption to make is the stable unit treatment value assumption (SUTVA; Rubin, 1980, 1990) under which the potential outcomes for the ith unit are determined by the treatment the ith unit received. That is, there is no interference between units (Cox, 1958) and there are no versions of treatments (Rubin, 1980). Then, all potential outcomes for N units with two possible treatments can be represented by an array with N rows and two columns, the ith unit having a row with two potential outcomes, Yi(0) and Yi(1), where each could, in principal, be a vector with many components. Obviously, SUTVA is a major assumption. Good researchers attempt to make such assumptions plausible by the design of their studies. For example, SUTVA becomes more plausible when units are isolated from each other, as when using, for the units, intact schools rather than students in the schools when studying an educational intervention, such as a smoking prevention program (e.g., see Peterson et al., 2000).

In addition to (1) the vector indicator of treatments for each unit in the study, W = {Wi}, (2) the array of potential outcomes when exposed to the active treatment, Y(1) = {Yi(1)}, and (3) the array of potential outcomes when not exposed, Y(0) = {Yi(0)}, we have (4) an array of covariates X = {Xi}, which are, by definition, unaffected by treatment, such as age, race sex, or pretest scores, where the ‘pre’ means prior to the intervention, that is, before t0. Covariates can be used to help define causal estimands. All causal estimands involve comparisons of Yi(0) and Yi(1) on either all N units, or a common subset of units; for example, the average causal effect across all units that are female as indicated by their Xi, or the median causal effect for units with Xi indicating male and Yi(0) indicating failure on the posttest under the control treatment.

Under SUTVA, all causal estimands can be calculated from the matrix of scientific values with ith row: (Xi, Yi(1), Yi(0)). By definition, all relevant information is encoded in Xi, Yi(0), Yi(1) and so the labeling of the N rows is a random permutation of 1,…, N, and the matrix is row exchangeable. Covariates play a particularly important role in the analysis of observational studies for causal effects where they are also known as possible confounders or risk factors. In some studies, the units exposed to the active treatment differ in their distribution of covariates in important ways from the units not exposed. To see how this issue influences our formal framework, we must define the assignment mechanism, the probabilistic mechanism that determines which units receive the active version of the treatment and which units receive the control version.

Introduction: Causal Inference as a Comparison of Potential Outcomes

Causal inference refers to an intellectual discipline that considers the assumptions, study designs, and estimation strategies that allow researchers to draw causal conclusions based on data. As detailed below, the term ‘causal conclusion’ used here refers to a conclusion regarding the effect of a causal variable (often referred to as the ‘treatment’ under a broad conception of the word) on some outcome(s) of interest. The dominant perspective on causal inference in statistics has philosophical underpinnings that rely on consideration of counterfactual states. In particular, it considers the outcomes that could manifest given exposure to each of a set of treatment conditions. Causal effects are defined as comparisons between these ‘potential outcomes.’ For instance, the causal effect of a drug on systolic blood pressure 1 month after the drug regime has begun (vs no exposure to the drug) would be defined as a comparison of systolic blood pressure that would be measured at this time given exposure to the drug with the systolic blood pressure that would be measured at the same point in time in the absence of exposure to the drug. The challenge for causal inference is that we are not generally able to observe both of these states: at the point in time when we are measuring the outcomes, each individual either has had drug exposure or has not.

Sequential Exchangeability and Identifiability Conditions

Making causal inferences about fixed treatments requires measuring and adjusting for a set of covariates L – informally, the confounders – required to achieve conditional exchangeability Ya∐A|L . This condition holds in conditionally randomized experiments in which individuals are assigned treatment with a probability that depends only on the values of their covariates L. The condition also holds in observational studies in which the probability of receiving treatment depends on the measured covariates L and, conditional on L, does not depend on any unmeasured, independent predictors U of the outcome. When exchangeability Ya∐A|L holds, we can obtain consistent estimates of the average causal effect of treatment via any methods that appropriately adjust for the variables in L, such as standardization and inverse probability (IP) weighting.

Similarly, making causal inferences about treatment strategies requires measuring and adjusting for the time-varying covariates L¯k – the time-varying confounders – required to achieve a sequential version of conditional exchangeability that encompasses all time points in the study. This condition holds in (ideal) sequentially randomized experiments in which individuals are assigned treatment at each time k with a probability that depends only on the values of their prior covariate history L¯k and treatment history A¯k−1. The condition also holds in observational studies in which the probability of receiving treatment at each time depends on their treatment and covariate history (A ¯k−1,L¯k) and, conditional on this history, does not depend on any unmeasured predictors U of the outcome.

In the HIV example summarized in Figure 1, this sequential exchangeability holds because there are no arrows into treatment A1 from any unmeasured cause U of the outcome: all arrows into treatment A1 come from prior treatment A0 and covariates L1. That is, Figure 1 represents a sequentially randomized trial in which treatment A0 is marginally randomized and treatment A1 is conditionally randomized given A0 and L1. The treated and the untreated at time 1 are exchangeable conditional on A0 and L1. Sequential exchangeability – the combination of Yg∐A0 and Yg∐A1|L1,A0 – holds for all treatment strategies g that involve the time-varying treatment (A0, A1).

Now imagine the very long causal diagram with many time points k = 0, 1…K that represents a sequentially randomized trial in which treatment Ak at each time k is randomly assigned conditional on prior treatment and covariate history ( A¯k−1,L¯k). The causal diagram would include no direct arrows from any unmeasured risk factors U into treatment Ak at any time k. In this sequential randomized trial, the treated and the untreated at each time k are exchangeable conditional on prior covariate history L¯k and treatment history A¯k−1. That is, sequential exchangeability Yg∐Ak|L¯k,A¯k−1 holds for all times k = 0, 1…K, and we say that there are no unmeasured time-varying confounders. Under sequential exchangeability, we can obtain consistent estimates of the average causal effect of a treatment strategy via any methods that appropriately adjust for history (A¯k−1,L¯k), such as standardization and IP weighting. The condition of sequential exchangeability Yg∐Ak|L¯k,A¯k−1 can be slightly weakened. We can use single world intervention graphs (Richardson and Robins, 2013) to graphically derive these weaker sequential exchangeability conditions for static and dynamic treatment regimes.

Causal inference for time-varying treatments also requires a generalization of the positivity condition from the fixed version ‘if fL(l) ≠ 0, fA|L(a|l) > 0 for all a’ to the sequential version ‘if fA¯k−1,L¯k[a¯k−1,l¯k]≠0,thenfAk|A¯k−1, L¯k[ak|a¯k−1,l¯k]>0forallak.’ The definition of consistency of counterfactuals can also be generalized in a sequential fashion from the fixed version ‘if A¯=a¯ for a given subject, then Ya¯=Y for that subject’ to the sequential version ‘Ya¯=Ya ¯∗ if a¯∗=a¯,Ya¯=YifA¯=a¯,L¯ka¯=L¯ka¯∗ifa¯k−1∗=a ¯k−1,L¯ka¯=L ¯kifA¯k−1=a¯k−1,’ where L¯ka¯ is the counterfactual L-history through time k under regime a¯. As always, causal inference requires well-defined interventions. Otherwise, a meaningful comparison of the treatment strategies is not possible.

Unfortunately, sequential exchangeability and positivity are not guaranteed to hold in observational studies. Achieving approximate exchangeability requires expert knowledge, which will guide investigators in the design of their studies to measure as many variables in L¯k as possible. For example, in an HIV study, experts would agree that time-varying variables like CD4 cell count, viral load, and other joint predictors of treatment and outcome need to be measured and appropriately adjusted for. However, in observational studies the condition of sequential exchangeability cannot be empirically tested. Like for fixed treatments, causal inference for time-varying treatments requires the untestable assumption of conditional exchangeability – only now sequentially during the follow-up rather than at baseline only.

Advantages over Nonexperimental Research

In nonexperimental research, causal inference is fraught with uncertainty. Lacking a procedure such as random assignment to ensure comparability of treatment and control groups, the researcher is forced to fall back on theoretical stipulations. For example, researchers who argue that active participation in civic organizations encourages the belief that most other people are trustworthy stipulate that beliefs follow from actions when interpreting the correlation between the two in survey data. This correlation, however, is also consistent with the hypothesis that trust encourages participation or that trust and participation bear no causal relationship to one another but happen to share similar causes. Unless the researchers manipulate political participation through some randomized mechanism, they cannot rule out the possibility that the putative relationship between participation and trust is spurious.

A variety of statistical analyses can be used to buttress causal claims based on observational data. Each technique aims to simulate the conditions of an experiment and does so by introducing certain theoretical assumptions. One approach involves the use of multivariate analysis in order to eliminate pretreatment differences between treatment and control groups. Multivariate regression, for example, eliminates the covariance between the treatment variable and other observable factors that are thought to influence the dependent variable. This approach works as long as no unobserved or imperfectly measured factors remain correlated with both the treatment and the outcome variable. Unfortunately, this assumption cannot be tested directly except by means of an experiment.

Another approach involves the use of instrumental variables regression. Here the researcher posits an instrumental variable that predicts the treatment, yet is uncorrelated with unmodeled causes of the outcome variable. Note that the randomization procedures used in experimentation are designed to generate just such an instrumental variable—a sequence of random numbers that predict nothing other than who gets the treatment. In nonexperimental work, the selection of an appropriate instrumental variable is a matter of speculation. Sometimes the proposed instruments are quite compelling. Whether a child's birthday falls immediately before or after the cut-off date for acceptance into kindergarten predicts educational attainment but seems unlikely to affect wage rates, and whether municipal government faces an election year predicts how much money is budgeted for police but seems unlikely to affect crime rates. More often, however, instrumental variables regression languishes for lack of plausible instruments.

The advantages of field experimentation go beyond random assignment. Because the experimenter is often in direct control of how a treatment is administered, there is less measurement uncertainty than is often the case with nonexperimental research, much of which relies on surveys. Asking a sample of survey respondents to report whether they have participated in a job-training program, read the newspaper, or attended the meeting of a civic organization often invites misreports and fails to garner detailed information about the specific content of these stimuli.

Finally, field experimentation can be used to create circumstances that are rare yet informative. Consider the case in which a researcher seeks to study the effects of marginal tax rates on a region's labor supply but tax rates historically vary little around a long-term average. With limited variance in the independent variable, the researcher cannot estimate the effects of tax rates with precision (putting aside the usual uncertainties associated with drawing causal inferences from nonexperimental correlations). An income tax field experiment has the advantage of creating new observations so that widely varying tax rates may be compared, permitting more precise estimates of the effects of tax rates. This experimental approach has the further advantage of enabling researchers to examine whether their modeling assumptions, hold outside the confines of historical data.

3.13.6 Summary

The possibility of making causal inferences about latent variables has great appeal for the social and behavioural scientist, simply because many of the concepts in which they are most interested are not directly measurable. Many of the statistical and technical problems in applying the appropriate models to empirical data have largely been solved, and sophisticated software such as EQS means that researchers can investigate and fit extremely complex models routinely. Unfortunately, in their rush not to be left behind in the causal modeling stakes, many investigators appear to have abandoned completely their proper scientific skeptism, and accepted models as reasonable, simply because it has been possible to fit them to data. This would not be so important if it were not the case that much of the research involved is in areas where action, perhaps far-reaching action, taken on the basis of the findings of the research can have enormous implications, for example, in resources for education and legislation on racial inequality. Consequently, both producers of such research and audiences or consumers of it need to be particularly concerned that the conclusions reached are valid ones. With this in mind I would like to end with the caveat issued by Cliff (1983, p.125):

beautiful computer programs do not really change anything fundamental. Correlational data are still correlational, and no computer program can take account of variables that are not in the analysis.

Causal relations can only be established through patient, painstaking attention to all the relevant variables, and should involve active manipulation as a final confirmation

6.1 Theory and Models

In thinking about describing children's causal inferences as rational, particularly in relation to social information, we had reason to be influenced by sociological ideas proposed by Weber (1962/1925). Weber made two points about the definition of rational behavior that we believe apply to a psychological description of rationality, and particularly to how children's causal inference can be thought of as rational.

The first is that social behavior comes in many forms and that much of it can be defined as rational, particularly given the fact that individuals have diverse desires and knowledge states. For Weber, the most important kind of social behavior was reciprocal (or “goal-directed”), in which individuals base their actions on the expected behavior of other people. Basing actions on the expectations of others' behavior is certainly part of contemporary theories of rational behavior (e.g. Csibra & Gergeley's, 2009 “Natural Pedagogy,” as mentioned previously, and to be discussed further below). We take as a starting point that arguing that children's causal inferences are rational implies that they have a set of expectations about how people (but also objects and events) behave and makes inferences from those beliefs. This is hopefully uncontroversial.

The second point that Weber emphasized is that there are compelling reasons to compare individuals' behaviors (and the behaviors of societies, but we will not consider this issue here) to some kind of ideal model. Weber's emphasis is reflected in the set of computational models describing the process by which information gathered from others can affect children's inferences about causal, statistical, or linguistic information (e.g. Butterfield et al., 2009; Eaves & Shafto, in this volume, Shafto et al., 2012; Sobel et al., 2010). These models have particularly relied on Bayesian methods.

Some have argued that such models of rational inference are implausible as descriptions of human cognition (see e.g. Jones & Love, 2011; Holyoak & Cheng, 2011). Weber himself did emphasize that ideal models should be descriptive (as opposed to normative) and plausible given the data (as opposed to requiring implausible calculations). Bayesian models have been incredibly important to advancing our understanding of causal inference, in both children and adults, but they are also (usually) intended as computational-level (cf. Marr, 1982) descriptions of reasoning processes. What this means is that when we argue that children's reasoning at certain points is consistent with Bayesian models, this is not to imply that they are engaging in Bayesian calculations (either explicitly or implicitly); rather, that such models describe group behavior well given a particular set of assumptions. Our hope is that other researchers share this belief, even if they do not explicitly state it.

Furthermore, there are promising computational models that potentially reveal the ways children might be engaging in such inferences that approximate Bayesian calculations (e.g. Bonawitz, Denison, Chen, Gopnik & Griffiths, 2011). These models, however, are still very much under development. Whether plausible computational descriptions of the process by which children make inferences from others' information ultimately emerge from this kind of investigation, from algorithms that use completely different kinds of architecture (e.g. McClelland & Thompson, 2007), or from algorithms yet undescribed, is an open question.

Publisher Summary

This chapter discusses the role of causal inference within the context of medical experiments. Several accounts of causation from the current philosophy of science literature are elaborated. One of the most appealing features of manipulation theories of causation is that, unlike the other approaches described so far, they provide a straightforward explanation of why the notion of causation would be so important to human beings. An approach to casual inference that includes an intuitive format for representing claims about cause and effect that facilitates general formulations of some commonly assumed principles linking causation and probability is presented. A different approach to noncompliance is based on the thought that although treatment assignment may not be an ideal intervention, it may nevertheless be an instrumental variable. The role of external validity and extrapolation in context of medical experiments is also elaborated. External validity has to do with whether the causal relationships learned about in the experimental context can be generalized in this manner.

Counterfactual arguments arise whenever analysts draw causal inferences about the impact either of policies that were adopted (what would have happened if they were not) or of policies that were rejected (what would have happened if they were adopted). Do such counterfactual arguments, however, advance our understanding of public policy? Although political partisans often use counterfactuals to justify predetermined conclusions, not all such arguments are thinly veiled tautologies. This article documents: (a) a pressing need to be explicit about standards for evaluating competing claims; (b) an unfortunate tendency in the scholarly literature to oscillate between dismissing dissonant counterfactuals as hopelessly speculative and proclaiming favorite counterfactuals as self-evidently true. If debates are not to reduce to expressions of ideological taste, it is necessary to articulate standards of evidence and proof that transcend rival schools of thought. To this end, the article advances a provisional taxonomy of standards for judging counterfactual claims that includes logical tests (clarity and cotenability), historical tests (the minimal-rewrite rule), theoretical and statistical tests (consistency with known laws and regularities), and projectability (capacity to stimulate accurate predictions in other policy contexts).

5.1 Internal Validity

Internal validity refers to the degree to which causal inference is reasonable because all alternative explanations for an apparent treatment effect can be ruled out. The RD design is very strong with respect to internal validity because the process of selection into treatments is fully known and only factors that would serendipitously induce a discontinuity in the pre–post relationship at the cutoff can be considered threats. However, validity also depends on how well the analyst can model the true pre–post relationship so that underlying nonlinear relationships do not masquerade as discontinuities at the cutoff point.

VII Conclusions

Experiments do offer unparalleled ability to ascertain causal inferences from a complex, confusing, and often chaotic world. The ability to isolate factors of interest to determine their influence on outcomes of concern provides one of the most compelling reasons to undertake experimental study. However, in many scholars’ views, the messy environment of real-world politics—rife with unintended consequences, blowback effects, and self-conscious actors—renders the applicability and generalizability of such sterile experimental environments limited.

However, the past use of experimental methodology has led to a number of important insights on topics of widespread interest to political scientists, including the impact of certain tactics such as direct mail on voter turnout, the influence of inter-ethnic identity on trust and cooperation, and the effect of negative advertising on candidate evaluation. No doubt further experimental work remains warranted as new technologies offer unprecedented opportunities to access previously inaccessible aspects of human decision-making processes and render the previously invisible transparent for all to witness in wonder.