The overarching goal of public health is to maximise the health of the population, and to achieve this we need evidence about what works and what does not work. Good studies are difficult to design and implement, and interpretation of their results and conclusions is not always as straightforward as we might hope. How, then, can we make the best use of this information? In the next three chapters we look at ways to identify, appraise, integrate and interpret the literature to generate the evidence we need to inform policy and practice. In this chapter we focus on interpreting the results from a single study, because if they are not valid they will be of limited value. The central question we have to answer when we read a study report is, ‘Are the results of the study valid?’

Peer review supports decisions related to publications, grant proposals, awards, or personnel selection. Independent of the specific occasion, we propose validity as a chief evaluation criterion for reviews. While applicable to all occasions, the principles of validity-oriented quality control are particularly suited for journal reviews. Beyond evaluating validity and the scientific potential of a given piece of research, we address how peer reviewing serves important functions and is accountable for the growth of science at a more superordinate level. We also provide guidelines and concrete recommendations for how a good peer review may serve these functions. Good peer review, thereby, fosters both the advancement of scientific research and the quality, precision, and sincerity of the scientific literature. The end of the chapter is devoted to a core set of good reviewer practices, conceived as an essential feature of academic culture.

Like Chapter 7, this chapter also discusses the sequencing of case study designs. Here we particularly focus on the deductive–inductive sequencing. Using an exemplar case study, we discuss how prediction outliers (deviant cases) identified during the initial study can guide the sequencing of designs in further stages. In the context of the example, we discuss the research question, theoretical sampling, cases, levels of analysis, and the potential requirement for additional data collection. Furthermore, we discuss the issue of omitted variable bias and internal validity in the context of sequenced case study designs. We end the chapter with a discussion on how to report sequenced case studies following deductive–inductive reasoning.

We discuss the single embedded case study design in this chapter. We deliberate how this design is different from multiple and single holistic designs in terms of the levels of analysis and the nature of replication. The selection rationale and sampling are discussed next. Afterwards, we move on to the longitudinal and/or cross-sectional single embedded designs. The strengths and the weaknesses of the design in terms of internal validity, external validity, and the number of variables are discussed subsequently. This chapter also discusses the (mis)conception regarding longitudinal designs and temporal embedded units.

We introduce and define the single holistic case study design in this chapter. The strengths of the design are discussed in detail, with examples. In particular, we discuss the potential of single holistic design in providing a detailed explanation of processes. Single holistic case studies also explore the theorizing potential of unique cases which hold the potential to reveal new dimensions of a phenomenon or falsify/refute an existing theory. Relatively high data access, construct validity, potential to include an unlimited number of variables, etc., are some other strengths that we discuss. The weaknesses of the design (i.e. low internal and external validity) are discussed afterwards. The chapter also addresses some common (mis)conceptions regarding single holistic designs and their external validity.

Here we discuss the multiple embedded case study design and the levels and cases involved in it. The design’s capacity to undertake between- and within-case comparative analyses are deliberated upon afterwards. Multiple embedded designs offer a variety of design possibilities, which we discuss in this chapter. The potential of the design for high internal validity is considered afterwards. Lastly, the design’s weaknesses, including the lack of contextual depth, the smaller number of explanatory variables, and the access difficulties to the cases/embedded units, are detailed.

The final chapter in this book discusses some methodological considerations and debates surrounding case study research and its quality. In particular, we revisit the topic of research paradigms (i.e. positivism and interpretivism). Relatedly, we discuss different quality criteria as proposed by prior researchers from both paradigmatic camps. In particular, we focus on the rigor versus trustworthiness discussion and the internal versus external validity debate. Afterwards, we briefly discuss the iterative cycles of data collection and analysis one would encounter during a qualitative case study research process. We end the chapter (and subsequently the book) with a guiding framework which will help researchers in sequencing case study designs by acknowledging the weaknesses of individual designs and leveraging their strengths. The framework can be adopted and adapted to suit the specific research objectives of the study in hand.

We discuss multiple case studies in this chapter. We start off with a discussion of theoretical sampling and replication logic. We specifically discuss literal and theoretical replication (LR and TR) in connection with multiple case studies. The strengths and limitations of LR and TR are discussed thereafter. In particular, we deliberate upon the potential of TR to enhance the internal and external validity of a case study. Henceforth, we address some common (mis)conceptions regarding replication logic, internal validity, external validity (generalizability), and reliability. We also discuss how multiple case studies might need to sacrifice the depth of observation for breadth. Other potential weaknesses, such as the smaller number of independent variables and the difficulty in controlling context, are also discussed thereafter.

This chapter provides a comprehensive overview of the verbal-guise technique (henceforth VGT), a variant of the speaker evaluation paradigm in which guises representing different language varieties are produced by different speakers, each speaking in their habitual language variety. First, key features of the VGT are discussed. Second, a brief historical sketch of the technique’s introduction, development, and proliferation in language attitudes research is offered. Third, key advantages (e.g. speaker authenticity, ease of implementation) and disadvantages (e.g. lack of full experimental control) of the technique are reviewed. Fourth, various practical considerations surrounding research planning and design are described and several recommendations are offered, including: (a) matching speakers on various demographic factors (e.g. sex, age), (b) matching speakers on extraneous vocal characteristics not of interest in the study (e.g. pitch), and (c) using multiple speakers to represent each variety of interest. Fifth, the main concerns surrounding the analysis and interpretation of data obtained using the VGT are discussed. Finally, a brief sketch is provided of the methodological considerations that were involved in designing a recent study utilising the VGT, which examined Americans’ attitudes toward standard American English and nine foreign accents: Arabic, Farsi, French, German, Hindi, Hispanic, Mandarin, Russian, and Vietnamese.

Chapter 3 focuses on how to think when evaluating experiments. This includes a discussion of realism, particularly why mundane realism or resemblance to the “real world” receives far too much attention, as well as an overview of how to design experimental treatments. The chapter then turns to validity issues, offering a new way to think about external validity in assessing experiments. This includes a detailed discussion of sampling and why the onus should be more on critics of an experimental sample than on the experimentalist him/herself (i.e., to justify a sample).

Experimental approaches are gaining in popularity across disciplines, ranging from behavioural sciences to economics. In this chapter, we discuss the advantages and disadvantages of field experiments and review their use by scholars to study routine dynamics. Based on these, we suggest that field experiments hold further promise to study routines given their potential to develop and test theory, while achieving internal and external validity. To further the adoption of field experiments to study routines, we outline a five-step procedure, including research questions and hypotheses, context and research setting, treatment and design, measurement and statistical tests, and managing field experiments. We conclude by discussing potential research questions and contexts suitable for field experiments.

The conclusion begins by exploring questions of internal and external validity. In the latter case, the concept of religious group identity is applied to faiths other than Islam and is found to extend to other egalitarian faiths, including Judaism and Buddhism. In addition, cross-national data on vote volatility confirm that the trust problem invoter coordination extends beyond the Turkish case. Delving into some out-of-sample predictions, I consider where in the Muslim worldIslamic groups might be particularly successful, based on a combination of low trust and salient Islamic identity. I also explorewhat mightexplain the strange combination of low trust and high honesty in Muslim countries. To address this trust deficit, I suggest that over-bearing institutions may play a key role in not allowing citizens to learn who among them can really be trusted. Finally, I consider what factors from within my theory could explain the eventual decline of Islamic-based groups, in Turkey and elsewhere, before posing some questions for future lines of inquiry.

This chapter guides the reader in moving from a general interest in neuroscience to a specific testable hypothesis.