Theory and interview focus.

The capacity to generate good description draws heavily on being able to identify the general, abstract problem or problems at the core of what someone is saying and to quickly ask about the conditions that likely set up this challenge or the options likely tried. At the outset, the interviewer has presumably already thought hard about the general focus of the conversation in the kinds of terms Robert Weiss outlines in his helpful book, Learning from StrangersFootnote ³ – for example, describing a process, learning how someone interpreted an event, presenting a point of view, framing hypotheses and identifying variables, etc. In policy-focused case studies, we begin by identifying the broad outcomes decision-makers sought and then consider hypotheses about the underlying strategic challenges or impediments decision-makers were likely to encounter. Collective action? Coordination across institutions? Alignment of interests or incentives (principal–agent problems)? Critical mass? Coordination of social expectations? Capacity? Risk mitigation? Spoilers? All of the above? Others?

Locking down an initial general understanding – “This is a case of what?” – helps launch the conversation: “As I understand it, you faced ___ challenge in order to achieve the outcomes the program was supposed to generate. How would you characterize that challenge? … This problem often has a couple of dimensions [fill in]. In what ways were these important here, or were they not so important?” Asking follow-up questions in order to assess each possible impediment helps overcome problems of omission, deliberate or inadvertent. In this sense, accuracy is partly a function of the richness of the dialogue between the interviewer’s theoretical imagination, the questions posed, and the answers received.

The interview then proceeds to document the steps taken to address each component problem, and here, again, theory is helpful. The characterization of the core strategic challenges spawns a set of hypotheses. For example, if the key delivery challenge is the need for collective action, then we know we will have to ask questions to help assess whether the outcome sought was really a public good as well as how the decision-maker helped devise a solution, including (most likely) a way to reduce the costs of contributing to the provision of that public good, a system for monitoring contributions, or whether there was one person or organization with an exceptional stake in the outcome and therefore a willingness to bear the costs. In short, the interview script flows in large part from a sense of curiosity informed by theory.

Theory also helps us think about the outcomes we seek to explain in a case study and discuss in an interview. In policy-relevant research we are constantly thinking in terms of measures or indicators, each of which has an imperfect relationship with the overarching development outcome we seek to explain. A skilled interviewer comes to a conversation having thought deeply about possible measures to evaluate the success or failure of an action and asks not only how the speaker, the interviewee, thought about this matter, but also whether any of these plausible measures were mooted, understanding that the public servants or citizens involved in a program may have had entirely different metrics in mind.

Often the outcomes are not that easy to measure. Nonetheless, we want something more concrete than a personal opinion, a thumbs up or thumbs down. To take one example, suppose the aim of a case study is to trace the impact of cabinet management or cabinet design on “ability of factions to work together.” This outcome is not easy to assess. Certainly, we want to know how people themselves define “ability to work together,” and open-ended questions are initially helpful. Instead of nudging the interviewee’s mind down a particular path, the interviewer allows people to organize their own thoughts.Footnote ⁴ But a skilled interviewer then engages the speaker to try to get a better sense of what it was about people’s perceptions or behavior that really changed, if anything. In the abstract it is possible to think of several possible measures: how long it took to arrange meetings to coordinate joint initiatives, how often requested meetings actually took place, how often the person interviewed was included in subcabinet deliberations that involved the other political parties, whether the person interviewed felt part of the decision process, whether the deputy minister (always from the other party in this instance) followed through on action items within the prescribed timeframe, whether there was name-calling in the meeting room or fistfights, etc.

An interviewer also wants to figure out whether the theory of change that motivated a policy intervention actually generated the outcomes observed. Again, theory plays a role. It helps to come to an interview with plausible alternative explanations in mind. In the example above, maybe power-sharing had little to do with reducing tension among faction leaders. Instead, war weariness, a shift in public opinion, a sharp expansion of economic opportunity outside government that reduced the desire to remain in government, the collapse of factional differences in the wake of demographic change, the personality of the head of government – any of these things might also have accounted for the results. A skilled interviewer devises questions to identify whether any of these causal dynamics were in play and which facts would help us understand the relative importance of one explanation versus the others.

Let me add a caveat at this point. Theoretically informed interviewing leads us to the kinds of descriptive detail important for analysis and understanding. However, it is also crucial to remember that our initial frameworks, if too narrowly defined, can cause us to lose the added value that interviews can generate. As Albert Reference HirschmanHirschman (1985) noted years ago, paradigms and hypotheses can become becoming a straightjacket (‘confirmation bias’), and the unique contribution of interview-based research is that it can foster a dialogue that corrects misimpressions. Openness to ideas outside the interview script is important for this reason

For example, understanding the source of political will is important in a lot of policy research, but sometimes the most important outcome the lead decision-maker wants to achieve is not the one that most people associated with a policy know about or share. Say we want to use interview-based cases to help identify the conditions that prompt municipal public works programs and other city services to invest in changes that would improve access to early childhood development services. It soon becomes clear that the mayors who had made the most progress in promoting this kind of investment and collaboration sought outcomes that went well beyond boosting children’s preparedness for preschool, the initial supposition, and, moreover, each wanted to achieve something quite distinctive. For some, the larger and longer-term aim was to reduce neighborhood violence, while for others the ambition was to diminish inequality or boost social capital and build trust. The open-ended question “Why was this program important to you?” helps leverage this insight.

Theory and the interview process

. Interviewing employs theory in a second sense as well. To reveal what really happened, we have to weed out the details people have remembered incorrectly while filling in the details some never knew and others didn’t consider important, didn’t want to highlight, or simply forgot. Therefore, in the context of an interview, it is the researcher’s job not only to seek relevant detail about processes, but also to perceive the gaps and silences and use additional follow-up questions or return interviews to secure explanations or elaboration.

In this instance, the researcher navigates through a series of hypotheses about the speaker’s relationship to the issue at hand and knowledge of events. A few of the questions that leverage information for assessing or weighting answers include:

• “You had a peripheral role in the early stages of these deliberations/this implementation process, as I understand it. How did you learn about the rationale for these decisions? [Co-workers on the committee? Friends? Briefed by the person who led the committee or kept the minutes? Gleaned this information as you began to participate?] How would you say that joining the deliberation/process/negotiation late colored your view of the issues and shaped your actions, or did it not make much difference?”

• “You were involved in a lot of difficult decisions at the time. How closely involved were you in this matter? Did you spend a lot of time on it? Was it an especially high priority for you, or was it just part of your daily work?” “Given all the difficult matters you had to deal with at the time, how greatly did this issue stand out, or is it hard to remember?” (Level of knowledge helps the interviewer weight the account when trying to integrate it with other information.)

• “The other people involved in this decision/process/negotiation had strong ties to political factions. At least some of them must have tried to influence you. At what stages and in what form did these kinds of pressures arise?” “Were some voices stronger than others?” “How would you say these lobbying efforts affected your decision/work/stance?”

• “As I understand it, you took a decision/action that was unusual/worked against your personal interest/was sure to be unpopular with some important people. How would you characterize your reasons for doing so?”

The information these questions leverage helps the case writer assess the likely accuracy of an account, in at least three ways: First, it helps us understand whether someone was in a position to know or heard about an action secondhand. Second, it helps us assess the integrity of a response – for example, does a statement run contrary to the speaker’s obvious personal interests and is it therefore more believable? Third, it can also help spot purely opportunistic spin: Is the view expressed consistent with the speaker’s other actions and attitudes, or is at odds with these? Can the person offer a clear story about this divergence, or did this perception or preference evolve in association with a promotion, an election, or some other event that may have influenced behavior?

Theory and ability to arbitrate among competing statements.

The third task of interview-based case research is to meld information garnered from different conversations and other types of sources in order to triangulate to the truth. This too is a theory-driven enterprise. Every time there is a clash between two assertions, we ask ourselves the familiar refrain “what could be going on here?” (hypothesis formation, drawn from theory), “how would I know?” (observable measures), and “by what method can I get this information?” (framing a follow-up question, checking a news report, consulting a register, etc.).

We may weigh the account of someone who joined a process late less heavily if it clashes with the information provided by those closer to a process, but it could be that the latecomer is less vulnerable to groupthink, has no reputation at stake, and offers a clearheaded story. Maybe we know the person was brought in as a troubleshooter and carried out a careful review of program data, or that the person is highly ambitious, eager to appear the hero who saved a failing initiative that, in fact, had not performed as badly as stated? Career paths, reputational information, and the written record – for example, longitudinal performance data – can all assist in making sense of disparate accounts.

This thought process may have to take place in the context of an interview as we listen and form follow-up questions, but it can also fuel exit interviews or second conversations designed both to provide another occasion to relate events remembered after the first encounter and to afford a chance to react to divergent information or ideas others may have voiced. This is the task of the exit interview.

To stress that skilled interviewing is theory-driven does not mean social scientists do a better job than journalists. Journalists might call the same kind of thought process “intuition” or “savvy,” but, when asked to step back, be self-conscious, and break down the mental exercise involved, the reality of what they do differs little from how social scientists or historians proceed in their work. The editor who tells a cub reporter, “I smell a rat” upon hearing the sketch of a story is positing an alternative hypothesis to test the adequacy of a description. Employing a general model built on experience, the editor pushes the reporter to use evidence to identify what the people described in the reporter’s draft are really doing.Footnote ⁵ A reporter’s intuition is equivalent to a social scientist’s skill in quickly framing plausible hypotheses and crafting a follow-up question that will yield the evidence to arbitrate among conflicting accounts – conducting social science inquiry “on the fly.”

Regardless of the interviewer’s own background, skilled interviewing places a premium on preparation. Even if the interviewer does not have a full-blown research design, it is crucial to have a preconversation sketch that frames hypotheses about the subject matter of the case and alternative plausible explanations; specifies the role the interviewee played in these events and the level of knowledge or type of gloss that relationship might produce (or at least does so with as much care as possible at this stage); and summarizes what archival sources say about the story line. This practice helps frame the questions that will tease out the evidence that disconfirms, modifies, or corroborates different versions of a story mid-conversation as new facts and observations emerge.

7.2.1 Case Selection Before the Processual Turn

Before “process tracing” entered the lexicon of social scientists, the dominant case selection strategy in case study research sought to maximize causal leverage via comparison, particularly via the “methods of agreement and difference” of John Stuart Reference MillMill (1843 [1974]: 388–391).

In Mill’s method of difference, the researcher purposively chooses two (or more) cases that experience different outcomes, despite otherwise being very similar on a number of relevant dimensions. Put differently, the researcher seeks to maximize variation in the outcome variable while minimizing variation amongst a set of plausible explanatory variables. It is for this reason that the approach also came to be referred to as the ‘most-similar systems’ or ‘most-similar cases’ design – while Mill’s nomenclature highlights variation in the outcome of interest, the alternative terminology highlights minimal variation amongst a set of possible explanatory factors. The underlying logic of this case selection strategy is that because the cases are so similar, the researcher can subsequently probe for the explanatory factor that actually does exhibit cross-case variation and isolate it as a likely cause.

Mill’s method of agreement is the mirror image of the method of difference. Here, the researcher chooses two (or more) cases that experience similar outcomes despite being very different on a number of relevant dimensions. That is, the researcher seeks to minimize variation in the outcome variable while maximizing variation amongst a set of plausible explanatory variables. An alternative, independent variable-focused terminology for this approach was developed – the ‘most-different systems’ or ‘most-different cases’ design – breeding some confusion. The underlying logic of this case selection strategy is that it helps the researcher isolate the explanatory factor that is similar across the otherwise different cases as a likely cause.Footnote ²

Figure 7.1 Case selection setup under Mill’s methods of difference and agreement

Mill himself did not believe that such methods could yield causal inferences outside of the physical sciences (Reference MillMill 1843 [1974]: 452). Nevertheless, in the 1970s a number of comparative social scientists endorsed Millian methods as the cornerstones of the comparative method. For example, Reference Przeworski and TeunePrzeworski and Teune (1970) advocated in favor of the most-different cases design, whereas Reference LijphartLijphart (1971) favored the most-similar cases approach. In so doing, scholars sought case selection techniques that would be as analogous as possible to regression analysis: focused on controlling for independent variables across cases, maximizing covariation between the outcome and a plausible explanatory variable, and treating cases as a qualitative equivalent to a row of dataset observations. It is not difficult to see why this contributed to the view that case study research serves as the “inherently flawed” version of econometrics (Reference Adams, Clemens, Orloff, Adams, Clemens and OrloffAdams, Clemens, and Orloff 2005: 25; Reference TarrowTarrow 2010). Indeed, despite his prominence as a case study researcher, Reference LijphartLijphart (1975: 165; Reference Lijphart1971: 685) concluded that “because the comparative method must be considered the weaker method,” then “if at all possible one should generally use the statistical (or perhaps even the experimental) method instead.” As Reference Hall, Mahoney and RueschemeyerHall (2003: 380; 396) brilliantly notes, case study research

In other words, the predominant case selection approach through the 1990s sought to do its best to reproduce a regression framework in a small-N setting – hence Lijphart’s concern with the “many variables, small number of cases” problem, which he argued could only be partially mitigated if, inter alia, the researcher increases the number of cases and decreases the number of variables across said cases (Reference Lijphart1971: 685–686). Later works embraced Lijphart’s formulation of the problem even as they sought to address it: for example, Reference Eckstein, Greenstein and PolsbyEckstein (1975: 85) argued that a “case” could actually be comprised of many “cases” if the unit of analysis shifted from being, say, the electoral system to, say, the voter. Predictably, such interventions invited retorts: Reference LiebersonLieberson (1994), for example, claimed that Millian methods’ inability to accommodate probabilistic causation,Footnote ³ interaction effects, and multivariate analysis would remain fatal flaws.

7.2.2 Enter Process Tracing

It is in this light that ‘process tracing’ – a term first used by Reference HobarthHobarth (1972) but popularized by Reference George and LaurenGeorge (1979) and particularly Reference George and BennettGeorge and Bennett (2005), Reference Brady and CollierBrady and Collier (2010), Reference Beach and PedersenBeach and Pedersen (2013), and Reference Bennett and CheckelBennett and Checkel (2015) – proved revolutionary for the ways in which social scientists conceive of case study research. Cases have gradually been reconceptualized not as dataset observations but as concatenations of concrete historical events that produce a specific outcome (Reference MahoneyGoertz and Mahoney 2012). That is, cases are increasingly treated as social processes, where a process is defined as “a particular type of sequence in which the temporally ordered events belong to a single coherent pattern of activity” (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 214). Although there exist multiple distinct conceptions of process tracing – from Bayesian approaches (Reference Bennett, Bennett and CheckelBennett 2015) to set-theoretic approaches (Reference Mahoney, Kimball and KoivuMahoney et al. 2009) to mechanistic approaches (Reference Beach and PedersenBeach and Pedersen 2013) to sequentialist approaches (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015) – their overall esprit is the same: reconstructing the sequence of events and interlinking causal logics that produce an outcome – isolating the ‘causes of effects’ – rather than probing a variable’s mean impact across cases via an ‘effects of causes’ approach.Footnote ⁴

For this intellectual shift to occur, processualist social scientists had to show how a number of assumptions underlying Millian comparative methods – as well as frequentist approaches more generally – are usually inappropriate for case study research. For example, the correlational approach endorsed by Reference Przeworski and TeunePrzeworski and Teune (1970), Reference LijphartLijphart (1971), and Reference Eckstein, Greenstein and PolsbyEckstein (1975) treats observational units as homogeneous and independent (Reference Hall, Mahoney and RueschemeyerHall 2003: 382; Reference MahoneyGoertz and Mahoney 2012). Unit homogeneity means that “different units are presumed to be fully identical to each other in all relevant respects except for the values of the main independent variable,” such that each observation contributes equally to the confidence we have in the accuracy and magnitude of our causal estimates (Reference Brady and CollierBrady and Collier 2010: 41–42). Given this assumption, more observations are better – hence, Reference LijphartLijphart (1971)’s dictum to “increase the number of cases” and, in its more recent variant, to “increase the number of observations” (Reference King, Keohane and VerbaKing, Keohane, and Verba 1994: 208–230). By independence, we mean that “for each observation, the value of a particular variable is not influenced by its value in other observations”; thus, each observation contributes “new information about the phenomenon in question” (Reference Brady and CollierBrady and Collier 2010: 43).

By contrast, practitioners of process tracing have shown that treating cases as social processes implies that case study observations are often interdependent and derived from heterogeneous units (Reference MahoneyGoertz and Mahoney 2012). Unit heterogeneity means that not all historical events, and the observable evidence they generate, are created equal. Hence, some observations may better enable the reconstruction of a causal process because they are more proximate to the central events under study. Correlatively, this is why historians accord greater ‘weight’ to primary than to secondary sources, and why primary sources concerning actors central to a key event are more important than those for peripheral figures (Reference TrachtenbergTrachtenberg 2009; Reference TanseyTansey 2007). In short, while process tracing may yield a bounty of observable evidence, we seek not to necessarily increase the number, but rather the quality, of observations. Finally, by interdependence we mean that because time is “fateful” (Reference SewellSewell 2005: 6), antecedent events in a sequence may influence subsequent events. This “fatefulness” has multiple sources. For instance, historical institutionalists have shown how social processes can exhibit path dependencies where the outcome of interest becomes a central driver of its own reproduction (Reference PiersonPierson 1996; Reference PiersonPierson 2000; Reference MahoneyMahoney 2000; Reference Hall, Mahoney and RueschemeyerHall 2003; Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015). At the individual level, processual sociologists have noted that causation in the social world is rarely a matter of one billiard ball hitting another, as in Reference HumeHume’s (1738 [2003]) frequentist concept of “constant conjunction.” Rather, it hinges upon actors endowed with memory, such that the micro-foundations of social causation rest on individuals aware of their own historicality (Reference SewellSewell 2005; Reference AbbottAbbott 2001; Reference Abbott2016).

At its core, eschewing the independence and unit homogeneity assumptions simply means situating case study evidence within its spatiotemporal context (Reference Hall, Mahoney and RueschemeyerHall 2003; Reference Falleti and LynchFalleti and Lynch 2009). This commitment is showcased by the language which process-sensitive case study researchers use when making causal inferences. First, rather than relating ‘independent variables’ to ‘dependent variables’, they often privilege the contextualizing language of relating ‘events’ to ‘outcomes’ (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015). Second, they prefer to speak not of ‘dataset observations’ evocative of cross-sectional analysis, but of ‘causal process observations’ evocative of sequential analysis (Reference Brady and CollierBrady and Collier 2010; Reference MahoneyGoertz and Mahoney 2012). Third, they may substitute the language of ‘causal inference via concatenation’ – a terminology implying that unobservable causal mechanisms are embedded within a sequence of observable events – for that of ‘causal inference via correlation’, evocative of the frequentist billiard-ball analogy (Reference Waldner and KincaidWaldner 2012: 68). The result is that case study research is increasingly hailed as a “distinctive approach that offers a much richer set of observations, especially about causal processes, than statistical analyses normally allow” (Reference Hall, Mahoney and RueschemeyerHall 2003: 397).

Risk 1: Ignoring Ordered Processes

Process-sensitive social scientists have long noted that “the temporal order of the events in a sequence [can be] causally consequential for the outcome of interest” (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 218; see also Reference PiersonPierson 2004: 54–78). For example, where individual acts of agency play a critical role – such as political elites’ response to a violent protest – “reordering can radically change [a] subject’s understanding of the meaning of particular events,” altering their response and the resulting outcomes (Reference AbbottAbbott 1995: 97).

An evocative illustration is provided by Reference SewellSewell’s (1996) analysis of how the storming of the Bastille in 1789 produced the modern concept of “revolution.” After overrunning the fortress, the crowd freed the few prisoners held within it; shot, stabbed, and beheaded the Bastille’s commander; and paraded his severed head through the streets of Paris (Reference SewellSewell 1996: 850). When the French National Assembly heard of the taking of the Bastille, it first interpreted the contentious event as “disastrous news” and an “excess of fury”; yet, when the king subsequently responded by retreating his troops to their provincial barracks, the Assembly recognized that the storming of the Bastille had strengthened its hand, and proceeded to reinterpret the event as a patriotic act of protest in support of political change (Reference SewellSewell 1996: 854–855). The king’s reaction to the Bastille thus bolstered the Assembly’s resolve to “invent” the modern concept of revolution as a “legitimate rising of the sovereign people that transformed the political system of a nation” (Reference SewellSewell 1996: 854–858). Proceeding counterfactually, had the ordering of events been reversed – had the king withdrawn his troops before the Bastille had been stormed – the National Assembly would have had little reason to interpret the popular uprising as a patriotic act legitimating reform rather than a violent act of barbarism.

Temporal ordering may also alter a social process’s political outcomes through macro-level mechanisms. For example, consider Reference FalletiFalleti’s (2005, Reference Falleti2010) analysis of the conditions under which state decentralization – the devolution of national powers to subnational administrative bodies – increases local political autonomy in Latin America. Through process tracing, Falleti demonstrates that when fiscal decentralization precedes electoral decentralization, local autonomy is increased, since this sequence endows local districts with the monetary resources necessary to subsequently administer an election effectively. However, when the reverse occurs, such that electoral decentralization precedes fiscal decentralization, local autonomy is compromised. For although the district is being offered the opportunity to hold local elections, it lacks the monetary resources to administer them effectively, endowing the national government with added leverage to impose conditions upon the devolution of fiscal resources.

For our purposes, what is crucial to note is not simply that temporal ordering matters, but that in ordered processes it is not the presence or absence of events that is most consequential for the outcome of interest. For instance, in Falleti’s analysis both fiscal and electoral decentralization occur. This means that a traditional Millian framework risks dismissing some explanatory events as causally irrelevant on the grounds that their presence is insufficient for explicating the outcome of interest (see Figure 7.2).

Figure 7.2 How ordered processes risk being ignored by a Millian setup

The way to deductively attenuate the foregoing risk is to develop an ordered theory and then modify the traditional Millian setup to assess the effect of ordering on an outcome of interest. That is, deductive theorizing aimed at probing the causal effect of ordering can guide us in constructing an appropriate Millan case selection design, such as that in Figure 7.3. In this example, we redefine the fourth independent variable to measure not the presence or absence of a fourth event, but rather to measure the ordering of two previously defined events (in this case, events 1 and 2). This case selection setup would be appropriate if deductive theorizing predicts that the outcome of interest is produced when event 1 is followed by event 2 (such that, unless this specific ordering occurs, the presence of events 1 and 2 is insufficient to generate the outcome). In other words, if Millian methods are to be deductively used to select cases for comparison, the way to guard against prematurely dismissing the causal role of temporal ordering is to explicitly theorize said ordering a priori. If this proves difficult, or if the researcher lacks sufficient knowledge to develop such a theory, it is advisable to switch to the more inductive method for case selection outlined in the next section.

Figure 7.3 Deductively incorporating ordered processes within a Millian setup

Risk 2: Ignoring Paced Processes

Processualist researchers have also emphasized that, beyond temporal order, “the speed or duration of events … is causally consequential” (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 219). For example, social scientists have long distinguished an “eventful temporality” (Reference SewellSewell 1996) from those “big, slow moving” incremental sequences devoid of rapid social change (Reference Pierson, Mahoney and RueschemeyerPierson 2003). For historical institutionalists, this distinction is illustrated by “critical junctures” – defined as “relatively short periods of time during which there is a substantially heightened probability that agents’ choices will affect the outcome of interest” (Reference Capoccia and KelemenCapoccia and Kelemen 2007: 348; Reference Capoccia, Mahoney and ThelenCapoccia 2015: 150–151) – on the one hand, and those “causal forces that develop over an extended period of time,” such as “cumulative” social processes, sequences involving “threshold effects,” and “extended causal chains” on the other hand (Reference PiersonPierson 2004: 82–90; Reference Mahoney, Thelen, Mahoney and ThelenMahoney and Thelen 2010).

An excellent illustration is provided by Reference BeissingerBeissinger (2002)’s analysis of the contentious events that led to the collapse of the Soviet State. Descriptively, the sequence of events has its origins in the increasing transparency of Soviet institutions and freedom of expression accompanying Gorbachev’s Glasnost (Reference BeissingerBeissinger 2002: 47). As internal fissures within the Politburo began to emerge in 1987, Glasnost facilitated media coverage of the split within the Soviet leadership (Reference Beissinger2002: 64). In response, “interactive attempts to contest the state grew regularized and began to influence one another” (Reference Beissinger2002: 74). These challenging acts mobilized around previously dormant national identities, and for the first time – often out of state incompetence – these early protests were not shut down (Reference Beissinger2002: 67). Protests reached a boiling point in early 1989 as the first semicompetitive electoral campaign spurred challengers to mobilize the electorate and cultivate grievances in response to regime efforts to “control nominations and electoral outcomes” (Reference Beissinger2002: 86). By 1990 the Soviet State was crumbling, and “in many parts of the USSR demonstration activity … had become a normal means for dealing with political conflict” (Reference Beissinger2002: 90).

Crucially, Beissinger stresses that to understand the causal dynamics of the Soviet State’s collapse, highlighting the chronology of events is insufficient. The 1987–1990 period comprised a moment of “thickened history” wherein “what takes place … has the potential to move history onto tracks otherwise unimaginable … all within an extremely compressed period of time” (Reference Beissinger2002: 27). Information overload, the density of interaction between diverse social actors, and the diffusion of contention engendered “enormous confusion and division within Soviet institutions,” allowing the hypertrophy of challenging acts to play “an increasingly significant role in their own causal structure” (Reference Beissinger2002: 97, 27). In this light, the temporal compression of a sequence of events can bolster the causal role of human agency and erode the constraints of social structure. Proceeding counterfactually, had the exact same sequence of contentious events unfolded more slowly, it is doubtful that the Soviet State would have suddenly collapsed.

Many examples of how the prolongation of a sequence of events can render them invisible, and thus produce different outcomes, could be referenced. Consider, for example, how global climate change – which is highlighted by Reference PiersonPierson (2004: 81) as a prototypical process with prolonged time horizons – conditions the psychological response of social actors. As a report from the American Psychological Association underscores, “climate change that is construed as rapid is more likely to be dreaded,” for “people often apply sharp discounts to costs or benefits that will occur in the future … relative to experiencing them immediately” (Reference SwimSwim et al. 2009: 24–25; Reference Loewenstein and ElsterLoewenstein and Elster 1992). This logic is captured by the metaphor of the “boiling frog”: “place a frog in a pot of cool water, and gradually raise the temperature to boiling, and the frog will remain in the water until it is cooked” (Reference BoyatzisBoyatzis 2006: 614).

What is important to note is that, once more, paced processes are not premised on the absence or presence of their constitutive events being causally determinative; rather, they are premised on the duration of events (or their temporal separation) bearing explanatory significance. Hence the traditional approach to case selection risks neglecting the causal impact of temporal duration on the outcome of interest (see Figure 7.4).

Figure 7.4 Paced processes risk being ignored by a Millian setup

Here, too, the way to deductively assess the causal role of pacing on an outcome of interest is to explicitly develop a paced theory before selecting cases for empirical analysis. On the one hand, we might theorize that it is the duration of a given event that is causally consequential; on the other hand, we might theorize that it is the temporal separation of said event from other events that is significant. Figure 7.5 suggests how a researcher can assess both theories through a revised Millian design. In the first example, we define a fourth independent variable measuring not the presence of a fourth event, but rather the temporal duration of a previously defined event (in this case, event 1). This would be an appropriate case selection design to assess a theory predicting that the outcome of interest occurs when event 1 unfolds over a prolonged period of time (such that if event 1 unfolds more rapidly, its mere occurrence is insufficient for the outcome). In the second example, we define a fourth independent variable measuring the temporal separation between two previously defined events (in this case, events 1 and 2). This would be an appropriate case selection design for a theory predicting that the outcome of interest only occurs when event 1 is temporally distant to event 2 (such that events 1 and 2 are insufficient for the outcome if they are proximate). Again, if the researcher lacks a priori knowledge to theorize how a paced process may be generating the outcome, it is advisable to adopt the inductive method of case selection described in Section 7.4.

Figure 7.5 Deductively incorporating paced processes within a Millian setup

Risk 3: Ignoring Equifinal Causal Processes

Finally, researchers have noted that causal processes may be mired by equifinality: the fact that “multiple combinations of values … produce the same outcome” (Reference MahoneyMahoney 2008: 424; see also Reference George and BennettGeorge and Bennett 2005; Reference MahoneyGoertz and Mahoney 2012). More formally, set-theoretic process tracers account for equifinality by emphasizing that, in most circumstances, “necessary” conditions or events are actually INUS conditions – individually necessary components of an unnecessary but sufficient combination of factors (Reference Mahoney and VanderpoelMahoney and Vanderpoel 2015: 15–18).

One of the reasons why processualist social scientists increasingly take equifinality seriously is the recognition that causal mechanisms may be context-dependent. Sewell’s work stresses that “the consequences of a given act … are not intrinsic to the act but rather will depend on the nature of the social world within which it takes place” (Reference SewellSewell 2005: 9–10). Similarly, Reference Falleti and LynchFalleti and Lynch (2009: 2; 11) argue that “causal effects depend on the interaction of specific mechanisms with aspects of the context within which these mechanisms operate,” hence the necessity of imposing “scope conditions” on theory building. One implication is that the exact same sequence of events in two different settings may produce vastly different causal outcomes. The flip side of this conclusion is that we should not expect a given outcome to always be produced by the same sequence of events.

For example, consider Sewell’s critique of Reference SkocpolSkocpol (1979)’s States and Social Revolutions for embracing an “experimental temporality.” Skocpol deploys Millian methods of case selection to theorize that the great social revolutions – the French, Russian, and Chinese revolutions – were caused by a conjunction of three necessary conditions: “(1) military backwardness, (2) politically powerful landlord classes, and (3) autonomous peasant communities” (Reference SewellSewell 2005: 93). Yet to permit comparison, Skocpol assumes that the outcomes of one revolution, and the processes of historical change more generally, have no effect on a subsequent revolution (Reference SewellSewell 2005: 94–95). This approach amounts to “cutting up the congealed block of historical time into artificially interchangeable units,” ignoring the fatefulness of historical sequences (Reference SewellSewell 2005). For example, the Industrial Revolution “intervened” between the French and Russian Revolutions, and consequently one could argue that “the revolt of the Petersburg and Moscow proletariat was a necessary condition for social revolution in Russia in 1917, even if it was not a condition for the French Revolution in 1789” (Reference SewellSewell 2005: 94–95). What Sewell is emphasizing, in short, is that peasant rebellion is an INUS condition (as is a proletariat uprising), rather than a necessary condition.

Another prominent example of equifinality is outlined by Reference CollierCollier’s (1999: 5–11) review of the diverse pathways through which democratization occurs. In the elite-driven pathway, emphasized by Reference O’Donnell and SchmitterO’Donnell and Schmitter (1986), an internal split amongst authoritarian incumbents emerges; this is followed by liberalizing efforts by some incumbents, which enables the resurrection of civil society and popular mobilization; finally, authoritarian incumbents negotiate a pacted transition with opposition leaders. By contrast, in the working-class-driven pathway, emphasized by Reference Rueschemeyer, Stephens and StephensRueschemeyer, Stephens, and Stephens (1992), a shift in the material balance of power in favor of the democracy-demanding working class and against the democracy-resisting landed aristocracy causes the former to overpower the latter, and via a democratic revolution from below a regime transition occurs. Crucially, Reference CollierCollier (1999: 12) emphasizes that these two pathways need not be contradictory (or exhaustive): the elite-driven pathway appears more common in the Latin American context during the second wave of democratization, whereas the working-class-driven pathway appears more common in Europe during the first wave of democratization.

What is crucial is that Millian case selection is premised on there being a single cause underlying the outcome of interest. As a result, Millian methods risk dismissing a set of events as causally irrelevant ex ante in one case simply because that same set of events fails to produce the outcome in another case (see Figure 7.6). Unlike ordered and paced processes, there is no clear way to leverage deductive theorizing to reconfigure Millian methods for case selection and accommodate equifinality. However, I argue that the presence of equifinal pathways can be fruitfully probed if we embrace a more inductive approach to comparative case selection, as the next section outlines.

Figure 7.6 Equifinal causal processes risk being ignored by a Millian setup

7.4.1 Tempering the Deductive Use of Millian Methods

To begin, one means to ensure against a Millian case selection design overlooking an ordered, paced, or equifinal causal process (in the absence of deductive theorizing) is to be wary of leveraging the methods of agreement and difference to eliminate potential explanatory factors (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 225–226). That is, the decision to discard an explanatory variable or historical event as causally unnecessary (via the method of agreement) or insufficient (via the method of difference) may be remanded to the process-tracing stage, rather than being made ex ante at the case selection stage.

Notice how this recommendation is particularly intuitive in light of the advances in process-tracing methods. Before this burgeoning literature existed, Millian methods were called upon to accomplish two things at once: (1) provide a justification for selecting two or more cases for social inquiry, and (2) yield causal leverage via comparison and the elimination of potential explanatory factors as unnecessary or insufficient. But process-tracing methodologists have showcased how the analysis of temporal variation disciplined via counterfactual analysis, congruence testing, and process-tracing tests renders within-case causal inference possible even in the absence of an empirical comparative case (Reference George and BennettGeorge and Bennett 2005; Reference GerringGerring 2007; Reference CollierCollier 2011; Reference MahoneyMahoney 2012; Reference Beach and PedersenBeach and Pedersen 2013; Reference Bennett and CheckelBennett and Checkel 2015; Reference LevyLevy 2015). That is, the ability to make causal inferences need not be primarily determined at the case selection stage.

The foregoing implies that if a researcher does not take temporal dynamics into account when developing their theory, the use of Millian methods should do no more than to provisionally discount the explanatory purchase of a given explanatory factor. The researcher should then bear in mind that as the causal process is reconstructed from a given outcome, the provisionally discounted factor may nonetheless be shown to be of causal relevance – particularly if the underlying process is ordered or paced, or if equifinal pathways are possible.

Despite these limitations, Millian methods might fruitfully serve additional functions from the standpoint of case selection, particularly if researchers shift (1) when and (2) why they make use of them. First, Millian methods may be as – if not more – useful after process tracing of a particular case is completed rather than to set the stage for within-case analysis. Such a chronological reversal – process tracing followed by Millian case selection, instead of Millian case selection followed by process tracing – inherently embraces a more inductive, theory-building approach to case study research (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 229–231) which, I suspect, is far more commonly used in practice than is acknowledged. I refer to this approach as the method of inductive case selection, wherein “theory-building process tracing” (Reference Beach and PedersenBeach and Pedersen 2013: 16–18) of a single case is subsequently followed by the use of a most-similar or most-different cases design.

7.4.2 Getting Started: Selecting the Initial Case

The method of inductive case selection begins by assuming that the researcher has justifiable reasons for picking a particular case for process tracing and is subsequently looking to contextualize the findings or build a theory outwards. Hence, the first step involves picking an initial case. Qualitative methodologists have already supplied a number of plausible logics for selecting a single case, and I describe three nonexhaustive possibilities here: (1) theoretical or historical importance; (2) policy relevance and salience; and (3) empirically puzzling nature.

First, an initial case may be selected due to its theoretical or historical importance. Reference Eckstein, Greenstein and PolsbyEckstein (1975), for example, defines an idiographic case study as a case where the specific empirical events/outcome serve as a central referent for a scholarly literature. As an illustration, Reference Gerring and CojocaruGerring and Cojocaru (2015: 11) point to Reference North and WeingastNorth and Weingast (1989)’s influential study of how the Glorious Revolution in seventeenth-century Britain favorably shifted the constitutional balance of power for the government to make credible commitments to protecting property rights (paving the way for the financial revolution of the early eighteenth century). Given that so much of the scholarly debate amongst economic historians centers on the institutional foundations of economic growth, North and Weingast’s case study was “chosen (it would appear) because of its central importance in the [historical political economy] literature on the topic, and because it is … a prominent and much-studied case” (Reference Gerring and CojocaruGerring and Cojocaru 2015: 11). In other words, Reference North and WeingastNorth and Weingast (1989)’s study is idiographic in that it “aim[s] to explain and/or interpret a single historical episode,” but it remains “theory-guided” in that it “focuses attention on some theoretically specified aspects of reality and neglects others” (Reference LevyLevy 2008: 4).

While the causes of the Glorious Revolution are a much-debated topic amongst economic historians, they have less relevance to researchers and practitioners focused on assessing the effects of contemporary public policy interventions. Hence, a second logic for picking a first case for process tracing is its policy relevance and salience. Reference George and BennettGeorge and Bennett (2005: 263–286) define a policy-relevant case study as one where the outcome is of interest to policy-makers and its causes are at least partially amenable to policy manipulation. For example, one recent World Bank case study (Reference El-Saharty and NagarajEl-Saharty and Nagaraj 2015) analyzes how HIV/AIDS prevalence amongst vulnerable subpopulations – particularly female sex workers – can be reduced via targeted service delivery. To study this outcome, two states in India – Andhra Pradesh and Karnataka – were selected for process tracing. There are three reasons why this constitutes an appropriate policy-relevant case selection choice. First, the outcome of interest – a decline in HIV/AIDS prevalence amongst female sex workers – was present in both Indian states. Second, because India accounts for almost 17.5 percent of the world population and has a large population of female sex workers, this outcome was salient to the government (Reference El-Saharty and NagarajEl-Saharty and Nagaraj 2015: 3). Third, the Indian government had created a four-phase National AIDS Control Program (NACP) spanning from 1986 through 2017, meaning that at least one set of possible explanatory factors for the decline in HIV/AIDS prevalence comprised policy interventions that could be manipulated.Footnote ⁶

A third logic for picking an initial case for process tracing is its puzzling empirical nature. One obvious instantiation is when an exogenous shock or otherwise significant event/policy intervention yields a different outcome from the one scholars and practitioners expected.Footnote ⁷ For example, in 2004 the federal government of Nigeria partnered with the World Bank to improve the share of Nigeria’s urban population with access to piped drinking water. This partnership – the National Urban Water Sector Reform Project (NUWSRP1) – aimed to “increase access to piped water supply in selected urban areas by improving the reliability and financial viability of selected urban water utilities” and by shifting resources away from “infrastructure rehabilitation” that had failed in the past (Reference Hima and SantibanezHima and Santibanez 2015: 2). Despite $200 million worth of investments, ultimately the NUWSRP1 “did not perform as strongly on the institutional reforms needed to ensure sustainability” (Reference Hima and SantibanezHima and Santibanez 2015). Given this puzzling outcome, the World Bank conducted an intensive case study to ask why the program did “not fully meet its essential objective of achieving a sustainable water delivery service” (Reference Hima and SantibanezHima and Santibanez 2015).Footnote ⁸

The common thread of these three logics for selecting an initial case is that the case itself is theoretically or substantively important and that its empirical dynamics – underlying either the outcome itself or its relationship to some explanatory events – are not well understood. That being said, the method of inductive case selection merely presumes that there is some theoretical, policy-related, empirical, or normative justification to pick the initial case.

7.4.3 Probing Generalizability Via a Most-Similar Cases Design

It is after picking an initial case that the method of inductive case selection contributes novel guidelines for case study researchers by reconfiguring how Millian methods are used. Namely, how should one (or more) additional cases be selected for comparison, and why? This question presumes that the researcher wishes to move beyond an idiographic, single-case study for the purposes of generating inferences that can travel. Yet in this effort, we should take seriously process-tracing scholars’ argument that causal mechanisms are often context-dependent. As a result, the selection of one or more comparative cases is not meant to uncover universally generalizable abstractions; rather, it is meant to contextualize the initial case within a set or family of cases that are spatiotemporally bounded.

That being said, the first logical step is to understand whether the causal inferences yielded by the process-traced case can indeed travel to other contexts (Reference GoertzGoertz 2017: 239). This constitutes the first reconfiguration of Millian methods: the use of comparative case studies to assess generalizability. Specifically, after within-case process tracing reveals a factor or sequence of factors as causally important to an outcome of interest, the logic is to select a case that is as contextually analogous as possible such that there is a higher probability that the causal process will operate similarly in the second case. This approach exploits the context-dependence of causal mechanisms to the researcher’s advantage: Similarity of context increases the probability that a causal mechanism will operate similarly across both cases. By “context,” it is useful to follow Reference Falleti and LynchFalleti and Lynch (2009: 14) and to be

For this approach to yield valuable insights, the researcher focuses on ‘controlling’ for as many of these contextual explanatory factors (crudely put, for as many independent variables) as possible. In other words, the researcher selects a most-similar case: if the causal chain similarly operates in the second case, this would support the conclusion that the causal process is likely at work across the constellation of cases bearing ‘family resemblances’ to the process-traced case (Reference SoiferSoifer 2020). Figure 7.7 displays the logic of this design:

Figure 7.7 Probing generalizability by selecting a most-similar case

As in Figure 7.7, suppose that process tracing of Case 1 reveals that some sequence of events (in this example, event 4 followed by event 5) caused the outcome of interest. The researcher would then select a most-similar case (a case with similar values/occurrences of other independent variables/events (here, IV1–IV3) that might also influence the outcome). The researcher would then scout whether the sequence in Case 1 (event 4 followed by event 5) also occurs in the comparative case. If it does, the expectation for a minimally generalizable theory is that it would produce a similar outcome in Case 2 as in Case 1. Correlatively, if the sequence does not occur in Case 2, the expectation is that it would not experience the same outcome as Case 1. These findings would provide evidence that the explanatory sequence (event 4 followed by event 5) has causal power that is generalizable across a set of cases bearing family resemblances.

For example, suppose a researcher studying democratization in Country A finds evidence congruent with the elite-centric theory of democratization of Reference O’Donnell and SchmitterO’Donnell and Schmitter (1986) described previously. To assess causal generalizability, the researcher would subsequently select a case – Country B – that is similar in the background conditions that the literature has shown to be conducive to democratization, such as level of GDP per capita (Reference Przeworski and LimongiPrzeworski and Limongi 1997; Reference Boix and StokesBoix and Stokes 2003) or belonging to the same “wave” of democratization via spatial and temporal proximity (Reference Collier, Rustow and EricksonCollier 1991; Reference HuntingtonHuntington 1993). Notice that these background conditions in Case B have to be at least partially exogenous to the causal process whose generalizability is being probed – that is, they cannot constitute the events that directly comprise the causal chain revealed in Case A. One way to think about them is as factors that in Case A appear to have been necessary, but less proximate and important, conditions for the outcome. Here, importance is determined by the “extent that they are [logically/counterfactually] present only when the outcome is present” (Reference Mahoney, Kimball and KoivuMahoney et al. 2009: 119), whereas proximity is determined by the degree to which the condition is “tightly coupled” with the chain of events directly producing the outcome (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 233).

An example related to the impact of service delivery in developmental contexts can be drawn from the World Bank’s case study of HIV/AIDS interventions in India. Recall that this case study actually spans across two states: Andhra Pradesh and Karnataka. In a traditional comparative case study setup, the selection of both cases would seem to yield limited insights. After all, they are contextually similar: “Andhra Pradesh and Karnataka … represent the epicenter of the HIV/AIDS epidemic in India. In addition, they were early adopters of the targeted interventions”; and they also experience a similar outcome: “HIV/AIDS prevalence among female sex workers declined from 20 percent to 7 percent in Andhra Pradesh and from 15 percent to 5 percent in Karnataka between 2003 and 2011” (Reference El-Saharty and NagarajEl-Saharty and Nagaraj 2015: 7; 3). In truth, this comparative case study design makes substantial sense: had the researchers focused on the impact of the Indian government’s NACP program only in Andhra Pradesh or only in Karnataka, one might have argued that there was something unique about either state that rendered it impossible to generalize the causal inferences. By instead demonstrating that favorable public health outcomes can be traced to the NACP program in both states, the researchers can support the argument that the intervention would likely prove successful in other contexts to the extent that they are similar to Andhra Pradesh and Karnataka.

One risk of the foregoing approach is highlighted by Reference SewellSewell (2005: 95–96): contextual similarity may suggest cross-case interactions that hamper the ability to treat the second, most-similar case as if it were independent of the process-traced case. For example, an extensive body of research has underscored how protests often diffuse across proximate spatiotemporal contexts through mimicry and the modularity of repertoires of contention (Reference TillyTilly 1995; Reference TarrowTarrow 1998). And, returning to the World Bank case study of HIV/AIDS interventions in Andhra Pradesh and Karnataka, one concern is that because these states share a common border, cross-state learning or other interactions might limit the value-added of a comparative design over a single case study, since the second case may not constitute truly new data. The researcher should be highly sensitive to this possibility when selecting and subsequently process tracing the most-similar case: the greater the likelihood of cross-case interactions, the lesser the likelihood that it is a case-specific causal process – as opposed to cross-case diffusion mechanism – that is doing most of the explanatory work.

Conversely, if the causal chain is found to operate differently in the second, most-similar case, then the researcher can make an argument for rejecting the generalizability of the causal explanation with some confidence. The conclusion would be that the causal process is sui generis and requires the “localization” of the theoretical explanation for the outcome of interest (Reference TarrowTarrow 2010: 251–252). In short, this would suggest that the process-traced case is an exceptional or deviant case, given a lack of causal generalizability even to cases bearing strong family resemblances. Here, we are using the ‘strong’ notion of ‘deviant’: the inability of a causal process to generalize to similar contexts substantially decreases the likelihood that “other cases” could be explained with reference to (or even in opposition to) the process-traced case.

There is, of course, the risk that by getting mired in the weeds of the first case, the researcher is unable to recognize how the overall chronology of events and causal logics in the most-similar case strongly resembles the process-traced case. That is, a null finding of generalizability in a most-similar context calls on the researcher to probe whether they have descended too far down the “ladder of generality,” requiring more abstract conceptual categories to compare effectively (Reference SartoriSartori 1970; Reference Collier and LevitskyCollier and Levitsky 1997).

7.4.4 Probing Scope Conditions and Equifinality Via a Most-Different Cases Design

A researcher that has process-traced a given case and revealed a factor or sequence of factors as causally relevant may also benefit from leveraging a most-different cases approach. This case selection technique yields complementary insights to the most-similar cases design described in the previous section, but its focus is altogether different: instead of uncovering the degree to which an identified causal process travels, the objective is to try to understand where and why it fails to travel and whether alternative pathways to the same outcome may be possible.

More precisely, by selecting a case that differs substantially from the process-traced case in background characteristics, the researcher maximizes contextual heterogeneity and the likelihood that the causal process will not generalize to the second case (Reference SoiferSoifer 2020). Put differently, the scholar would be selecting a least-likely case for generalizability, because the context-dependence of causal mechanisms renders it unlikely that the same sequence of events will generate the same outcome in the second case. This would offer a first cut at establishing “scope conditions” upon the generalizability of the theory (Reference TarrowTarrow 2010: 251) by isolating which contextual factors prevented the process from producing the outcome in the most-different case.

Figure 7.8 provides a visual illustration of what this design could look like. Suppose, once more, that process tracing in Case 1 has revealed that some event 4 followed by event 5 generated the outcome of interest. To maximize the probability that we will be able to place scope conditions on this finding, we would select a comparative case that is most different to the process-traced case (a case with different values/occurrences of other independent variables/events [denoted as IV1–IV3 in Figure 7.8] that might also influence the outcome) but which also experienced the sequence of event 4 followed by event 5. Given the contextual differences between these two cases, the likelihood that the same sequence will produce the same outcome in both is low, which then opens up opportunities for the researcher to probe the logic of scope conditions. In this endeavor, temporality can serve as a useful guide: a means for restricting the set of potential contextual factors that prevented the causal process from reproducing the outcome in Case 2 is to identify at what chronological point the linkages between events 4 and 5 on the one hand and the outcome of interest on the other hand branched off from the way they unfolded in Case 1. The researcher can then scout which contextual factors exuded the greatest influence at that temporal location and identify them as central to the scope conditions to be placed upon the findings.

Figure 7.8 Probing scope conditions by selecting a most-different case

To provide an example for how this logic of inquiry can work, consider a recent case study focused on understanding the effectiveness of Mexico’s conditional cash transfer program – Opportunitades, the first program of its kind – in providing monetary support to the female heads of Indigenous households (Reference Alva Estrabridis and Ortega NietoAlva Estrabridis and Ortega Nieto 2015). The program suffered from the fact that Indigenous beneficiaries dropped out at higher rates than their non-Indigenous counterparts. In 2009 the World Bank spearheaded an Indigenous Peoples Plan (IPP) to bolster service delivery of cash transfers to Indigenous populations, which crucially included “catering to indigenous peoples in their native languages and disseminating information in their languages” (Reference Alva Estrabridis and Ortega NietoAlva Estrabridis and Ortega Nieto 2015: 2). A subsequent impact evaluation found that “[w]hen program messages were offered in beneficiaries’ mother tongues, they were more convincing, and beneficiaries tended to participate and express themselves more actively” (Reference Alva Estrabridis and Ortega NietoAlva Estrabridis and Ortega Nieto 2015; Reference Mir, Gámez, Loyola, Martí and VerazaMir et al. 2011).

Researchers might well be interested in the portability of the foregoing finding, in which case the previously described most-similar cases design is appropriate – for example, a comparison with the Familias en Accion program in Colombia may be undertaken (Reference Attanasio, Battistin, Fitzsimons, Mesnard and Vera-Hernandez.Attanasio et al. 2005). But they might also be interested in the limits of the policy intervention – in understanding where and why it is unlikely to yield similar outcomes. To assess the scope conditions upon the “bilingualism” effect of cash transfer programs, a most-different cases design is appropriate. Thankfully, conditional cash transfer programs are increasingly common even in historical, cultural, and linguistic contexts markedly different from Mexico, most prominently in sub-Saharan Africa (Reference Lagarde, Haines and PalmerLagarde et al. 2007; Reference Garcia and MooreGarcia and Moore 2012). Selecting a comparative case from sub-Saharan Africa should prove effective for probing scope conditions: the more divergent the contextual factors, the less likely it is that the policy intervention will produce the same outcome in both contexts.

On the flip side, in the unlikely event that part or all of the causal process is nonetheless reproduced in the most-different case, the researcher would obtain a strong signal that they have identified one of those rare causal explanations of general scope. In coming to this conclusion, however, the researcher should be wary of “conceptual stretching” (Reference SartoriSartori 1970: 1034), such that there is confidence that the similarity in the causal chain across the most-different cases lies at the empirical level and is not an artificial by-product of imprecise conceptual categories (Reference Bennett and CheckelBennett and Checkel 2015: 10–11). Here process tracing, by pushing researchers to not only specify a sequence of “tightly-coupled” events (Reference Falleti, Mahoney, Mahoney and ThelenFalleti and Mahoney 2015: 233), but also to collect observable implications about the causal mechanisms concatenating these events, can guard against conceptual stretching. By opening the “black box” of causation through detailed within-case analysis, process tracing limits the researcher’s ability to posit “pseudo-equivalences” across contexts (Reference SartoriSartori 1970: 1035).

Selecting a most-different case vis-à-vis the process-traced case is also an excellent strategy for probing equifinality – for maximizing the likelihood that the scholar will be able to probe multiple causal pathways to the same outcome. To do so, it is not sufficient to merely ensure divergence in background conditions; it is equally necessary to follow Mill’s method of agreement by ensuring that the outcome in the process-traced case is also present in the second, most-different case. By ensuring minimal variation in outcome, the scholar guarantees that process tracing the second case will lead to the desired destination; by ensuring maximal variation in background conditions, the scholar substantially increases the likelihood that process tracing will reveal a slightly or significantly different causal pathway to said destination. Should an alternative route to the outcome be found, then its generalizability could be assessed using the most-similar cases approach described previously.

Figure 7.9 visualizes what this case selection design might look like. Here, as in previous examples, suppose process tracing in Case 1 provides evidence that event 4 followed by event 5 produced the outcome of interest. The researcher then selects a case with the same outcome, but with different values/occurrences of some independent variables/events (in this case, IV1–IV3) that may influence the outcome. Working backwards from the outcome to reconstruct the causal chain that produced it, the researcher then probes whether (i) the sequence (event 4 followed by event 5) also occurred in Case 2, and (ii) whether the outcome of interest can be retraced to said sequence. Given the contextual dissimilarities between these most-different cases, such a finding is rather unlikely, which would subsequently enable to the researcher to probe whether some other factor (perhaps IV2/event 2 in the example of Figure 7.9) produced the outcome in the comparative case instead, which would comprise clear evidence of equifinality.

Figure 7.9 Probing equifinality by selecting a most-different case with the same outcome

To return to the concrete example of Mexico’s conditional cash transfer program’s successful outreach to marginalized populations via bilingual service provision, an alternative route to the same outcome might be unearthed if a cash transfer program without bilingual outreach implemented in a country characterized by different linguistic, gender, and financial decision-making norms proves similarly successful in targeting marginalized populations. Several factors – including recruitment procedures, the size of the cash transfers, the requirements for participation, and the supply of other benefits (Reference Lagarde, Haines and PalmerLagarde et al. 2007: 1902) – could interact with the different setting to produce similar intervention outcomes, regardless of whether multilingual services are provided. Such a finding would suggest that these policy interventions can be designed in multiple ways and still prove effective.

To conclude, the method of inductive case selection complements within-case analysis by supplying a coherent logic for probing generalizability, scope conditions, and equifinality. To summarize, Figure 7.10 provides a roadmap of this approach to comparative case selection.

Figure 7.10 Case selection roadmap to assess generalizability, scope conditions, equifinality

In short, if the researcher has the requisite time and resources, a multistage use of Millian methods to conduct four comparative case studies could prove very fertile. The researcher would begin by selecting a second, most-similar case to assess causal generalizability to a family of cases similar to the process-traced case; subsequently, a third, most-different case would be selected to surface possible scope conditions blocking the portability of the theory to divergent contexts; and a fourth, most-different case experiencing the same outcome would be picked to probe equifinal pathways. This sequential, four-case comparison would substantially improve the researcher’s ability to map the portability and contours of both their empirical analysis and their theoretical claims.Footnote ⁹

8.3.1 Conventional Footnotes

The simplest and most widespread instruments of transparency used today in social science are citations found in footnotes, endnotes, and the text itself. Yet the current state of citation practice demonstrates the flaws in this approach. Basic citations in published work are often incomplete or incorrect, particularly if they appear as brief in-text “scientific citations” designed for a world in which most (quantitative) analysts use footnotes to acknowledge other researchers rather than cite evidence. Such citations do not provide either data access or analytic transparency. Scientific citations are often incomplete, leaving out page numbers and failing to specify the concrete textual reference within an article or on a page that the author considers decisive. Even if a citation is precise, most readers will be deterred by the need to locate each source at some third location, perhaps a library or an archive – and, in many cases, as with interviews and records of focus groups, the source material may not be available at all.Footnote ¹⁵ Even more troubling, conventional citations offer no analytical transparency whatsoever: the reader knows what is cited, but generally much less about why.

In theory, an attractive solution would be to return to the traditional method of linking evidence and explanation in most scholarly fields: long discursive footnotes containing extended quotations with interpretive annotations. Discursive footnotes of this kind remain widespread in legal academia, history, some humanities, and a few other academic disciplines that still prize qualitative transparency. In legal academia, for example, where fidelity to the precise text and rigorous interpretation are of great academic and practical value, articles may have dozens, even hundreds, of such discursive footnotes – a body of supplementary material many times longer than the article itself. The format evolved because it can enhance all three dimensions of transparency. The researcher is often obliged to insert extensive quotations from sources (data access); annotate those quotations with extensive interpretation of how, why, and to what extent they support a claim made in the text and how they fit into the broader context (analytic transparency); and discuss issues of data selection and opposing evidence (production transparency). At a glance, readers can scan everything: the main argument, the citation, the source material, the author’s interpretation, and information about how representative the source is. In many ways, discursive footnotes remain the “best practice” instruments for providing efficient qualitative transparency.

Yet recent trends in formatting social science journals – in particular, the advent of so-called scientific citations and ever-tighter word limits – have all but banished discursive footnotes. This trend is not methodologically neutral: it privileges quantitative research that employs external datasets and cites secondary journals rather than data, while blocking qualitative research from citing and interpreting texts in detail. As a result, in many social sciences, we see relatively little serious debate about the empirics of qualitative research. Replication or reanalysis is extremely difficult, and extension or secondary analysis almost impossible.Footnote ¹⁶ Given the economics of social science journals, this trend is unlikely to reverse. Practitioners and policy analysts face similar constraints, because they often aim their publications, at least in part, at nonexperts. Memos and reports have been growing shorter. Long discursive footnotes pose a visual barrier, both expanding the size of a text, and rendering it less readable and accessible. In sum, conventional footnotes and word limits are part of the problem, not the solution.

8.3.2 Hyperlinks to Online Sources

Some suggest that a simple digital solution would be to link articles and reports to source documents already posted online. Many government reports, journalistic articles, contemporary scholarship, and blogs often do just this. Yet this offers an inadequate level of research transparency, for three basic reasons. First, much material simply cannot be found online: Most primary field research evidence (e.g., interviews) is not there, and despite the efforts of archives to digitalize, we are far from having all documents online even in the most advanced industrial democracies, let alone elsewhere. Even journalistic articles and secondary scholarly works are unevenly available, with much inaccessible online (or hidden behind paywalls), in foreign languages, or buried within longer documents. Second, links to outside sources are notoriously unstable, and subject to “link rot” or removal.Footnote ¹⁷ Attempts to stabilize links to permit cross-citation have proven extremely challenging even when they focus on a very narrow range of documents (e.g., academic medical journals), and it is nearly impossible to do so if one is dealing, as policy analysts do, with an essentially unlimited range of contemporary material of many types and in many languages. Third, even when sources are available online – or when we place them online for this purpose – hyperlinks provide only data transparency, not analytical and process transparency. We learn what source a scholar cited but not why, let alone how he or she interpreted, contextualized, and weighed the evidence. This undermines one of the distinctive epistemological advantages of qualitative research.

8.3.3 Archiving Evidence in a Centralized Database

For many from other research traditions, data archiving may seem at first glance the most natural way to enhance transparency. It is, after all, the conventional solution employed by statistical researchers, who create centralized, homogeneous “datasets” where all evidence is stored, connected to a single set of algorithms used to analyze it. Moreover, data repositories do already exist for textual material, notably the Qualitative Data Repository for social science materials recently established with NSF funding at Syracuse University.Footnote ¹⁸ Data archiving is admittedly essential, especially for the purpose of preserving complete collections of new field data drawn from interviews, ethnographic notes, primary document collections, and web-searches of manageable size that are unencumbered by human subject or copyright restrictions.Footnote ¹⁹ Archiving full datasets can also help create a stronger bulwark against selection bias (“cherry-picking” or constructing biased case studies by selecting only confirming evidence) by obliging qualitative scholars to archive “all” their data.

Yet, while data archiving can be a useful ancillary technique in selected cases, it is unworkable as a general “default” approach for assuring qualitative research transparency because it is both impractical and inappropriate. Archiving is often impractical because ethical, legal, and logistical constraints limit the analyst’s ability to reveal to readers all the interviews, documents, or notes underlying qualitative research. Doing so often threatens to infringe the confidentiality of human subjects and violates copyright law limiting the reproduction of published material.Footnote ²⁰ Sanitizing all the interviews, documents, and notes (i.e., rendering them entirely anonymous and consistent with confidentiality agreements) is likely to impose a prohibitive logistical burden on many research projects. These limitations become much greater when the researcher seeks to archive comprehensive sets of complete documents, as opposed to just releasing quotations or summaries, as some other transparency strategies require. This is often particularly problematic for policy practitioners, perhaps more so than scholars, because policy case studies and histories, after-action reports, and interview or focus-group analyses so commonly contain sensitive information.

Archiving is also inappropriate because it dilutes the distinctive epistemological advantages of qualitative research. The notion that archiving documents in one large collection generates transparency overlooks a distinctive quality of case study analysis. A qualitative analyst does not treat the data as one undifferentiated mass, analyzing all of it at once using a centralized algorithm, as in a statistical study. Instead, he or she presents and interprets individual pieces of data one at a time, each linked to a single step in the main narrative.Footnote ²¹ Qualitative analysts enjoy considerable flexibility to assign a different location, role, relative weight, reliability, and exact meaning to each piece of evidence, depending on its logical position in a causal narrative, the specific type of document it is, and the textual content of the quotation within that document. This type of nuanced and open-ended, yet rigorous and informed, contextual interpretation of sources is highly prized in fields such as history, law, anthropology, and the humanities. Any serious effort to enhance qualitative transparency must thus make clear to the reader how the analyst interprets each piece of data and exactly where in the narrative it fits. Simply placing all the evidence in a single database, even where it is logistically and legally feasible, does not help the reader much.Footnote ²² Links from citations to archived material are, at best, cumbersome. Moreover, as with hyperlinks and conventional citations, archiving fails to specify particular passages and provides little analytic transparency, because it fails to explain why each source supports the underlying argument at that point in the narrative. To achieve qualitative transparency, a less costly approach is required – one that reveals the inferential connection between each datum and the underlying analytical point in the narrative.

8.3.4 Active Citation/ATI: A “Best Practice” Standard of Qualitative Transparency

Given the practical and epistemological constraints outlined above, social scientists have recently agreed that the best way to enhance transparency is to exploit recent innovations in internet formatting and software engineering. These technologies permit us to create new digital formats that can reestablish the high levels of qualitative transparency afforded by discursive footnotes in a more efficient and flexible way. Active Citation (AC) and Annotation for Transparency Initiative (ATI) are two related, digitally enhanced transparency standards designed do just this. They are practical and epistemologically appropriate to qualitative research.

AC/ATI envisages a digitally enabled appendix to research publications and reports. Rather than being an entirely separate document, however, the appendix embeds each source and annotation in an entry linked to a specific statement or citation in the main narrative of a research article or report. These may take the form of numbered hyperlinks from the article to an appendix or, in the ATI version, a set of annotations that overlay the article using a separate but parallel software platform. Unlike modern in-text footnotes, hyperlinks, and archiving, AC/ATI reinforces the epistemological link between narrative, data, and interpretation central to qualitative research. This author-driven process of annotation and elaboration via a separate document assures the same (or greater) levels of data, analytical, and production transparency as discursive footnotes, but with greater flexibility and no constraint on overall length. Moreover, it reduces the logistical difficulties by leaving the existing format of basic digital or paper articles and reports completely unchanged. Indeed, AC/ATI has the advantage that some audiences can simply skim or read the article without any additional materials, while those with a desire for more information can activate the additional materials.

Two ways exist to implement the AC/ATI standards. One, initially proposed by advocates of AC, obliges authors to design standardized entries that promote realistic levels of data, analytic, and production transparency in a relatively structured way. Accordingly, AC prescribes that researchers link each annotation that concerns an “empirically contestable knowledge claim” to a corresponding appendix entry. Of course, this still leaves tremendous leeway to the author(s), who decide (as with any footnote or citation) what is sufficiently “empirical” or “contestable” to merit further elaboration. Once an author decides that further elaboration is required, each entry would contain three mandatory elements and room for one more optional one – though, again, the author would decide how detailed and lengthy this elaboration needs to be.

An examination of the four elements in an AC entry shows how, in essence, this system simply updates the centuries-old practice of discursive footnoting in a flexible, author-driven, and electronic form appropriate to a digital age.Footnote ²³ The four elements that can be in each entry are:

1. 1) A textual excerpt from the source. This excerpt is presumptively 50–100 words long, though the length is ultimately up to the author. It achieves basic qualitative data transparency by placing the essential textual source material that supports the claim “one click away” from the reader. Sources subject to human subject or copyright restrictions can be replaced with a sanitized version, a summary, or a brief description, as is feasible. This provides a modest level of prima facie data transparency, while minimizing the logistical demands on authors, the ethical threats to subjects, and the potential legal liability.

2. 2) An annotation. This length of interpretive commentary explains how, why, to what extent, and with what certainty the source supports the underlying claim in the main text. This provides basic analytic transparency, explaining how the author has interpreted the source. In this section, the author may raise not just the analysis of a given source, but its interpretive context, its representativeness of a broader sample, the existence of counterevidence, how it should be read in broader context, how it was translated, etc. This annotation can be of any length the author believes is justified.

3. 3) A copy of the full footnote citation, sufficient to locate the document. This is critical because authors may seek to use the appendices independently of the text – for example, in a bibliography or database. Also, it assures that, whatever the format being employed in the main report, a genuine full citation exists somewhere, which is far from true today.

4. 4) An optional link to (or scan of) the full source. A visual copy of the source would provide more context and unambiguous evidence of the source, as well as creating additional flexibility to accommodate nontraditional sources such as maps, charts, photographs, drawings, video, recordings, and so on. This option can be invoked, however, only if the author has the right to link or copy material legally and the ability to do so cost effectively, which may not always be the case – and doing so at all remains at the discretion of the author.

Of course, the de facto level of transparency that an author chooses to provide in any specific case will still reflect other important constraints. One constraint is ethical. The active citations cannot make material transparent that would harm research subjects or that is subject to confidentiality agreements. Ethical imperatives obviously override transparency.Footnote ²⁴ A second constraint is legal. The content of the entries must respect intellectual property rights. Fortunately, small citations of most published material (except artistic or visual products) can be cited subject to “fair use” or its equivalent in almost all jurisdictions – but in cases of conflict, legal requirements override transparency. A third constraint is logistical. The amount of time and effort required to provide discursive notes of the type AC envisages is surely manageable, since discursive footnotes with roughly the same content were the norm in some academic disciplines and were widely used in the social sciences until a generation ago – and still appear in many published books. Today, the advent of electronic scanning and word processing make the process far easier. One can readily imagine situations in which that would create excessive work for the likely benefit. This is yet another reason why the decision of how many annotations to provide and how long they are remains primarily with individual authors, subject to guidance from relevant research communities, as is currently the case with conventional citations. Ultimately, the number of such entries, and their length and content, remain essentially up to the author, much as the nature of footnotes is today.

ATI offers the slightly different prospect of a more flexible, open-ended standard. ATI’s major innovation is to use innovative software provided by the nonprofit firm hypothesis.Footnote ²⁵ In lieu of storing the annotated source entries in a conventional appendix (akin to existing practice with formal and quantitative research) and hyperlinking individual entries to selected citations, as AC initially recommended, ATI allows the annotations to be written at will, stored in a separate program, and seamlessly layered on top of a PDF article by running the two programs simultaneously. ATI software makes the annotated sections appear as highlighted portions of the article, and when one clicks on a section of highlighting, the additional material appears in a box alongside the article. ATI provides a particularly efficient and manipulable means of delivering these source material and annotations, and it provides almost infinite flexibility to authors. In trials, authors use the software to add annotations as they see fit. This type of software option also allows for separate commentary by readers. One might imagine the social sciences moving forward for a time with a set of such experiments that recommend no specific set of minimum standards for transparency but permit authors to define their own digital options. In a number of large test studies, dozens of younger scholars have tried ATI out with considerable enthusiasm, and this approach is in the process of adoption by major university presses that publish journals. This, it seems, is the future.