Research with Human Participants: Why IRB Approval Isn’t the Last Word

In the United States, conversations about the ethics of research with human participants often start with the Federal Policy for the Protection of Human Subjects (the Common Rule), which includes the requirements and guidelines for Institutional Review Boards (IRBs), the university-based bodies that review research proposals. The Common Rule governs federally funded biomedical and behavioral research with human participants; many institutions review all research, not just that which is federally funded. Therefore, US-based political scientists who work with human research participants have become familiar with navigating IRB review processes, and many journals use IRB approval as a shorthand means of ensuring that research participants’ rights and interests have been respected. On the surface, this may seem sensible. Journals do not have the capacity to vet research projects directly, and IRBs are widely available in higher educational institutions in the US. However, scholars have noted that the IRB model, which was initiated in response to concerns about biomedical research, may be a poor fit for our work, given the wide variety of approaches that social scientists use in their research (Yanow and Schwartz-Shea Reference Yanow and Schwartz-Shea2008, 483). In addition, because IRBs are American institutions, making IRB review the norm is problematic for a journal that publishes research from an international pool of authors and hopes to speak to an international community.

Moreover, review by an IRB or a non-US equivalent is neither sufficient nor necessary to ensure that research is ethical. IRB approval is not sufficient because, although IRBs are designed to identify and address individual risks that research participants may encounter, they do not consider a range of important and broader questions, such as whether the research might have an effect on a community beyond the individuals who participate in a given project or whether it might affect political outcomes that do not map readily as risks to the immediate subjects of the research. While IRBs require the reporting of unanticipated harms that occur during the research process, research proposals that are submitted for IRB review cannot anticipate all of the ethical dilemmas that may arise in the course of the research. Also, IRB review is unnecessary to ensure that research is ethical, as ethical researchers in countries without review board requirements can attest. We cannot reasonably require IRB (or non-US equivalent) review of submissions from authors at institutions without an ethical review body, so instead we require that they include ample discussion of research practices that are relevant to human participants and any potential for broader social or political effects. Although we consider IRB approval if it helps to answer our questions about the research process, authors should understand that simply submitting an approved research proposal is not enough and may not be necessary.

American Political Science Association Principles and Guidance

The American Political Science Review aspires to publish the best political science research in a global discipline. Indeed, the last editorial team was based entirely outside of the United States, and two members of the current team reside in Canada. Our reviewers, authors, and readers are an international community, and our thinking about the ethics of research begins from this recognition.

Rather than starting with the Common Rule, IRBs, or other nationally equivalent review boards, our ethical review process begins with the APSA Principles and Guidance. Although they are the result of an APSA initiative, they express long-standing and commonly held principles and norms. Beginning in 2017, the APSA convened an ad hoc committee to consider questions around the ethics of research involving human participants in political science. The committee’s membership included scholars who were experts in a wide range of research methods and individuals from outside the US. The committee’s charge was to “identify broad principles of ethics relating to research on human subjects that can guide individual scholars in the design and practice of their research and informing institutional review boards about current standards and practices in Political Science research” (APSA n.d.). Over the course of nearly 3 years, the committee carefully considered these issues and solicited input from APSA members, including non-US scholars, at multiple conferences, and from the APSA Council, a process that culminated in the 2020 Principles and Guidance. The document identifies broad areas of ethical concern that are keyed to the questions that political scientists ask and the ways that we conduct research. It also encourages thoughtful engagement across the discipline about how best to navigate the dilemmas that we face as researchers, reviewers, and journal editors.

The document consists of 12 principles and related guidance (APSA 2020, 2–20). Three of these principles are general: the first advocates respecting autonomy, considering participants’ well-being, and maintaining openness about ethical issues; the second underlines the responsibilities that researchers bear to consider ethics; and the third notes that deviations from the principles may be appropriate in some circumstances, but a reasoned justification must be given in presentations and publications (APSA 2020, 2). The fourth principle asks researchers to attend to power differentials between themselves and research participants and to take these differences into account in understanding “the voluntariness of consent and the evaluation of risk and benefit” (APSA 2020, 2–3).

The next several principles press for the consideration of the interests of research participants. The fifth one encourages researchers to seek informed and voluntary consent, especially for research that poses risks or for research in which one might expect people to decline participation if they were informed about it. The sixth principle addresses deception in research, pressing authors to consider whether the deception is necessary, what risks it imposes, and how it relates to consent and power. The seventh and eighth principles direct our attention to the harms and traumas that research may cause, pressing for evaluation of the potential harm to participants and also to those indirectly affected by the research. Research that has the potential to cause harm or trauma must be thoroughly justified and explained. The ninth principle encourages confidentiality and, where necessary, anonymity (APSA 2020, 5–9). These Principles broaden the scope of research ethics beyond those considered by IRBs; “participants” is a broader category than “subjects,” and potential harms go beyond physical and psychological harms.

The final three principles further broaden the scope of research ethics. The tenth principle presses for consideration of potential risk beyond that to which individual participants may be exposed. It encourages researchers to “consider the broader social impacts of the research process,” discouraging research that may “compromise the integrity of political processes” without securing consent of the participants (APSA 2020, 13). The eleventh principle cautions that researchers should be aware of relevant laws and regulations and encourages compliance with the sponsoring university’s regulatory framework as well as with local rules, regulations, and laws, while acknowledging that exceptions to local review requirements may be ethical (APSA 2020, 17). The twelfth principle emphasizes shared responsibility for promoting ethical research, beyond that of individual researchers and research teams (APSA 2020, 20). It encourages advisors, mentors, and graduate programs to promote the consideration of ethics and to provide instruction in how to navigate ethical dilemmas.

Importantly for our work at APSR, the twelfth principle directs journal editors and reviewers to “encourage researchers to be open about the ethical decisions they made in conducting their research; encourage research on research ethics; and provide editorial expressions of concern or solicit independent commentaries when publishing ethically troubling research.” Journal editors are also encouraged to “incorporate ethical commitments into their mission, bylaws, instruction, practices, and procedures.”

Incorporating the Principles into the Review Process

Our team has made it a priority to translate the Principles into concrete expectations for our authors and reviewers and to develop processes to consider these challenging questions. Authors who submit articles that draw on research with human participants will encounter questions that prompt reflection about ethics. Our submission interface prompts authors to review the Principles and either affirm that their research conforms to all of them or to affirm that they acknowledge the Principles and provide a reasoned justification for any exception(s) in the main text of the submitted article and, if need be, in an appendix. The prompt is broadly worded to signal that it is relevant for all types of research including ethnography, interviews, original surveys, and experiments. Authors whose research engages with human participants are also asked whether they “encounter[ed] any ethical issues or perceptions of ethical issues unrelated to the study of human subjects.”

As is the practice with most political science journal editors, we conduct technical checks on manuscripts to ensure that they meet word limits and threshold technical specifications and also to verify that they fall within the journal’s remit before deciding whether to desk reject them or send them out for review. At this stage, we also check to see whether the manuscript discusses research ethics concerning human participants based on the Principles. If the discussion is insufficient, we send the article back to the authors and ask them to address them before moving forward. For instance, we sometimes receive submissions in which the author has replied to the prompts stating that they did not engage human participants in the research process, perhaps assuming that their IRB’s determination that their research was exempt from review would suffice to satisfy our ethical review. If so, and if upon looking at the article we can see that human participants were involved, we send the manuscript back to them for clarification before considering it for review.

Not all issues are this simple, however. Because we were aware that more challenging ethical issues might arise, when we created our Editorial Board, we identified a small cohort of members who, together, form our Ethics Advisory Board. We are fortunate to have among this group both Scott Desposato and Trisha Phillips, who cochaired the Ad Hoc Committee that developed the Principles. The other members—Catherine Boone, Macartan Humphreys, Lauren MacLean, Layna Mosley, and Peregrine Schwartz-Shea—are well situated to advise us based on their collective experience with a broad array of political science research methods and their long-standing concern with ethics. Indeed, many also served on the Ad Hoc Committee. These experts assist the editorial team in determining whether submitted manuscripts align with the Principles and what kinds of reasoned justifications can support deviations from the Principles. They also assist us in developing advice for authors whose work raises ethical concerns. Although we reserve the discretion to decline to consider manuscripts that pose ethical concerns that the authors cannot resolve, in the vast majority of cases we have been able to help authors clarify and justify their research processes sufficiently to enable us to consider their work.

Once we have determined that a manuscript is suitable for peer review and are ready to invite reviewers to help us evaluate it, we include in our invitation letter a specific question: “Do you have any concerns about the ethics of the research conducted?” We ask for explanations if the answer is “yes” and offer reviewers the opportunity to explain their concerns to the editors. We see this question as a critical means of supplementing our internal reviewing and also as a way of engaging our reviewers actively in our project of raising the salience of research ethics.

As journal editors, our input comes at the end of the research process, long after researchers have designed their studies, conducted their work, and analyzed and reported the results. While we are currently reviewing research carried out prior to APSA’s adoption of the Principles, the fundamental norms governing research with human participants have not changed, although they are now foregrounded in the Principles. In all cases, we consider carefully how to balance fair consideration for projects initiated and completed in the earlier, less reflective ethical environment with ensuring that the research we consider adheres to the Principles (or offers a reasoned justification for deviations).

Conflicts of Interest and Scholarly Integrity

This note has focused primarily on the ethics of research involving human participants. But other more familiar ethical issues can arise and remain important. We will touch briefly on two: conflicts of interest and plagiarism, including self-plagiarism.

Like most journal editors, we attend to the potential for conflicts of interest in the review process. The members of the editorial team will not submit manuscripts to the journal during our term, nor will any individual editor participate in discussions or decisions about any manuscript authored or coauthored by someone with whom she has a significant personal or professional relationship.

We provide authors with the opportunity to let us know if there are reviewers to whom we should not send their manuscripts. We do not want to invite reviewers who will be biased for or against a project, so we appreciate our authors letting us know about people who either have already given extensive feedback or may be inclined against a project at the outset. We also count on reviewers to let us know if they perceive conflicts of interest that may not be visible to us and ask them specifically in the review questionnaire to disclose any perceived conflicts, declining to review if a conflict is significant enough. Once we receive reviews, we read them carefully. If the review itself signals either positive or negative bias, we take this into account in the extent to which it influences our decision on the manuscript.

At the same time, we expect our authors to abide by high standards in their own dealings with potential conflicts of interest. Upon submission, we ask authors whether any agency, organizational, or institutional funding supported the research and whether the authors have any conflicts of interest relating to any funding. We seek full transparency about relationships that authors may have with funding agencies, employers, other institutions, or individuals who might influence their research and a frank discussion of how the authors navigated these relationships to ensure their research was not compromised.

We use but do not rely unquestioningly upon programs that check submitted manuscripts for similarity to previously submitted manuscripts or to other published work. We consider only work that has not previously been published, and our team will not revisit judgments made by prior editorial teams. We rely on our expert reviewers to identify possible scholarly integrity issues with the manuscripts they consider. In the event that a problem arises, we rely on the guidelines developed by the interdisciplinary Committee on Publication Ethics (COPE), an organization that develops guidelines and other resources to educate scholarly publishers and editors about best practices for handling ethical dilemmas. The COPE guidelines recommend a sensible sequence of procedural steps to follow to address and resolve questions. We can thus apply a uniform process to address, for example, situations in which plagiarism is suspected or manuscripts that rely heavily on or reiterate the author’s own previously published work. In developing standards to address conflicts of interest and plagiarism, we look to the broad range of standards that prevail in different methodological and epistemological circles across the discipline.

Transforming Ethics as a Disciplinary Project

We believe our engagement with research ethics provides an important opportunity for scholars—authors, readers, reviewers, and editors—to participate in the thoughtful development of community norms to guide the conduct of research with human participants. We have already had several exchanges with authors that have left both us and the authors more deeply informed about and sensitive to ethics. We anticipate that some of the articles that we will soon publish provide excellent models for how to navigate ethical considerations in challenging settings. We hope that others will benefit from these exchanges among authors, reviewers, and editors, which will ultimately leave their mark on the manuscripts that we publish.

With all of this in mind, we encourage mentors, postdoctoral supervisors, and faculty who are instructing graduate and undergraduate students to make ethical considerations as important as questions about research design or data collection. We hope that the work we have done and the work our authors and reviewers are doing now will help to move our disciplinary conversations forward.