Teaching undergraduate students research methods is a priority for many political science departments (Ishiyama and Breuning Reference Ishiyama and Breuning2008). Understanding methods enables students to become better consumers of scholarship and to write stronger research papers and theses. Yet, explicit training in research methods for undergraduates—particularly in qualitative methods—remains rare (Parker Reference Parker2010).Footnote 1 As a result, students often encounter difficulties when beginning their own research projects. Even those students who have excelled in substantive courses may struggle when attempting to work with primary sources or to generate original data for the first time. In our own experiences of teaching senior thesis courses, we also found many students to be unaware of the wide range of research methods open to them within political science.
This article describes a methods lab approach that can be used to introduce research-methods skills to undergraduates in the context of senior thesis, research methods, or substantive courses in political science. In this approach, students are given explicit instruction on research methods and the opportunity to practice them before conducting their own research. This training helps students to better understand the rules and conventions that govern the practice of different types of research in political science and why those rules exist (Elman, Kapiszewski, and Kirilova Reference Elman, Kapiszewski and Kirilova2015). Giving students the opportunity to practice using research methods before embarking on their own research projects can help them to craft more creative research designs. It also can help them to understand the time required and potential pitfalls associated with each type of research—making the subsequent process of writing a research paper or thesis easier.
Giving students the opportunity to practice using research methods before embarking on their own research projects can help them to craft more creative research designs. It also can help them to understand the time required and potential pitfalls associated with each type of research—making the subsequent process of writing a research paper or thesis easier.
This article presents two sample methods labs that we used in an honors thesis seminar for government majors at our institution, and it provides step-by-step instructions to implement them. In the first lab, students practiced drawing on archival materials to evaluate a set of competing hypotheses; in the second, they developed their own research question and practiced crafting both survey and interview questions to answer it. We discuss our experiences in implementing the labs and provide tips for instructors who are interested in developing their own labs or in integrating the approach into other types of courses.
PROCESS AND IMPLEMENTATION
We piloted the methods lab approach in an honors thesis seminar for majors. In this seminar, methods instruction occurred alongside the mentorship of students as they embarked on year-long thesis projects covering a range of topics across all subfields of political science. We taught the honors seminar using the labs for two years; currently, the lab approach is being incorporated into a dedicated course on research methods in our department. Because each lab was designed to work as a self-contained lesson, these sample labs also could be incorporated into substantive political science courses.
A key feature of the lab approach was that it did not form part of a student’s own individual research. Rather, we gave explicit instruction about a methodological tool or technique; selected a topic and gave the students a sample of existing scholarly literature; and then had them practice using the method in the context of the selected research topic. Students were allowed five days to complete the labs; they reported spending about 5 to 7 hours outside of class working on each lab.
Another key feature of the approach was that students completed the work in groups, beginning the lab during class and completing it outside of class. This approach had several benefits. We observed students collaborating successfully, both during the in-person class and in the course’s virtual space. The group format also reduced the grading burden and allowed us to assess assignments more carefully. Although we taught the labs in the context of a relatively small seminar, the group format makes them also suitable for larger courses.
The final feature of the methods lab approach was the debriefing. When grading the assignments, we identified common problems and used them as the foundation of our debriefing lesson. This was an opportunity to teach the method again after the students had practiced using it. We found that the debriefing led to more student questions and deeper discussion because they had experienced roadblocks and wanted to discuss them. It also allowed us to highlight examples of student creativity.
When grading the assignments, we identified common problems and used them as the foundation of our debriefing lesson. This was an opportunity to teach the method again after the students had practiced using it.
METHODS LAB #1: USING ARCHIVAL MATERIALS TO EVALUATE COMPETING HYPOTHESES
The analysis of archival material is increasingly possible for undergraduates as important archival collections become digitized. This methods lab introduced students to an important and widely used archival collection—the Foreign Relations of the United States (FRUS) series onlineFootnote 2—and had them practice using it to evaluate competing hypotheses about why the United States conducted covert operations to overthrow regimes in Latin America during the Cold War.
Pre-Reading
Students were assigned two readings in preparation for the lab. The first reading, a chapter from Howard’s (Reference Howard2017) Thinking Like a Political Scientist entitled “Using Documents as Evidence,” introduced students to strengths and potential pitfalls of working with archival documents. It emphasizes attention to authority, bias, and currency in written records, and it describes the process through which scholars can triangulate different sources of evidence. The second reading, the introduction to O’Rourke’s (Reference O’Rourke2018) book entitled Covert Regime Change: America’s Secret Cold War, provided substantive background for this lab.
Instruction on Working with Archival Evidence
Before receiving the assignment, students were given explicit instruction on conducting archival research (see the online appendix). We discussed advantages and limitations of archival research as well as how to become oriented to the organizational scheme of an archive, interpret individual documents, and use documents as evidence in their analysis. We provided practical tips for working with large amounts of archival material. The lesson also highlighted several collections of archival materials that have been digitized, and we provided a list of additional resources on archival research that were well suited for undergraduates.
Introducing the Lab
We assigned students into groups of four or five and distributed the Archival Methods Lab assignment (table 1). The lab asked students to draw on an assigned set of documents from the FRUS series to evaluate competing hypotheses about what motivated American regime-change efforts in the Dominican Republic in the 1960s. We also gave them a handout adapted from O’Rourke’s (Reference O’Rourke2018) book that explains each hypothesis and provides brief background information on the politics of the Dominican Republic (see the online appendix).
Table 1 Archival Methods Lab
The lab instructed students to proceed in three steps: (1) specify the observable implications of each potential hypothesis; (2) read and analyze the assigned set of documents; and (3) write a memo drawing on their documents to evaluate the plausibility of each hypothesis. In class discussion, we encouraged students to think about whether the material in their set of documents would “be consistent with” or “cast doubt on” potential explanations instead of “proving” that one was correct. We allowed students about 30 minutes of class time to begin working on the lab. This gave them an opportunity to ask questions of the instructors and to work out a plan with their group for tackling the assignment. Each group was assigned a specific set of documents from the FRUS series (see the online appendix) and had five days to complete the assignment.
Debriefing
In debriefing the lab in the following class session, we asked students to describe their approach and identify any challenges. We then shared common themes that we observed in their memos. We found that the groups that struggled the most were those who did not clearly specify observable implications in advance or were too broad in what they understood to constitute evidence in support of a particular hypothesis. For example, some groups counted any mention of American firms as evidence that economic motives drove US actions, whereas other groups made a similar leap with references to the Soviet Union and the hypothesis about superpower rivalry. As a possible solution, we discussed looking at an initial sample of documents to refine the observable implications before reviewing the remaining documents. This discussion led to (1) the insight that actions taken to “preserve regional hegemony” and “preempt a Soviet threat” might appear similar to one another; and (2) the conclusion that the sample hypotheses must be specified more clearly in advance to be distinguished.
Another challenge students faced was in providing sufficient context for archival evidence cited in the memos. It was common for memos to include a quotation without indicating the type of archival document from which it originated or the author. In the debriefing discussion, students recognized that some familiarity with the actors and institutions mentioned in a document was a prerequisite to understanding its meaning and purpose. For example, documents with “talking points” for telephone calls with counterparts in the Dominican Republic differed significantly from intelligence briefings intended for internal consumption. We emphasized to students that providing this type of context was crucial when using quotations in support of their claims. We also discussed what students did with conflicting evidence; asked them to reflect on which perspectives were missing from the material they reviewed; and asked them to brainstorm other potential sources that they might use to triangulate evidence if they were to continue this research. After the conclusion of the lab, students reported greater comfort working with archival materials; several incorporated archival sources into their theses; and we observed an improvement in their ability to specify observable implications.
METHODS LAB #2: DEVELOPING SURVEY AND INTERVIEW QUESTIONS
In this lab, students learned how to design survey and interview questions. Because both methods require students to work with human subjects, understand research ethics, and apply for Institutional Review Board (IRB) approval, we decided to group them together. Both methods also share an important focus on question design yet call for different styles of questions. Finally, undergraduate research-methods textbooks commonly cite interviews and surveys as potentially useful tools for students who are conducting primary research (e.g., Baglione Reference Baglione2020; Johnson, Reynolds, and Mycoff Reference Johnson, Reynolds and Mycoff2019).
Pre-Reading
In preparation for this lab, students were required to read the Tomz and Weeks (Reference Tomz and Jessica2013) article, “Public Opinion and the Democratic Peace,” which used survey data to explore how individuals in the United States and the United Kingdom view military action against democracies. They also were required to read the associated appendix, which included survey questions and how they were coded for analysis. We also asked students to read the introduction and appendix from Mosley’s (Reference Mosley2013) edited volume, Interview Research in Political Science. Mosley’s introduction sets up the usefulness of interviews in political science research, describes examples of their use in the discipline, and discusses decisions and challenges related to using interviews. The appendix includes examples of multiple scholars’ interview materials, such as sample consent scripts and questionnaires.
Instruction on Interviews and Surveys
As we did with the archival lab, we began the survey and interview research lab with concrete instruction on each method (see the online appendix). We discussed research ethics and explained the process required for our institution’s IRB approval to conduct research with human subjects. We then discussed surveys, explaining their advantages and disadvantages, with a primary focus on the practical aspect of survey design. Finally, we discussed the benefits and drawbacks of interviews, highlighting how the two methods require the researcher to craft different types of questions. This lab illustrated how scholars can approach the same research question with different research methods.
Introducing the Lab
In this lab, students first crafted a research question to build on the existing scholarship on attitudes toward immigration summarized in Hainmueller and Hopkins’s (Reference Hainmueller and Hopkins2014) article, “Public Attitudes Toward Immigration.” Second, they developed survey questions that could help them answer their question. Third, they brainstormed a relevant group of interview subjects who were accessible to them and crafted a set of interview questions that would shed light on their question. Fourth, students prepared answers to several questions on our institution’s IRB form and drafted an informed consent letter (table 2).
Table 2 Survey and Interview Methods Lab
Debriefing
When we graded the interview and survey labs, we noted strengths in particular groups’ write-ups as well as common problems. We found that our students were able to develop a research question to address a gap in the literature based on the information that they were given.Footnote 3 However, they struggled to frame questions in ways that would give them useful survey data and fruitful interview material. They had trouble remembering that they should not ask their participants to directly answer their research question but rather should ask questions that help them—as researchers—to answer the question. In the surveys, this meant that students often were drafting questions that asked respondents to directly assess cause and effect (e.g., “On a scale of 1 to 5, how much do you think the media you consume has impacted your policy views?”) as opposed to crafting questions that capture a single concept. They needed to be reminded that survey analysis allows them to explore whether different questions have correlated responses.
For the interview questions, students had trouble crafting open-ended questions and often phrased them in ways that might cause a participant to feel as if they were being quizzed (e.g., “What did you learn about immigration in school?”). They needed to be reminded that interviews are best for generating in-depth explanations about motivations, reasoning, and processes and that the best way to elicit this type of information is to craft questions that encourage participants to tell stories. We were able to use the debriefing to brainstorm ways to refine the questions the students had developed. At the end of the discussion, students displayed a stronger understanding of the different strengths and limitations of interviews and survey work. Some students subsequently incorporated these methods into their thesis projects.
ADAPTING THE LAB APPROACH FOR OTHER RESEARCH METHODS AND TYPES OF COURSES
Table 3 draws on our experience in developing labs focused on archival, interview, and survey research to provide practical suggestions for instructors interested in developing additional labs. The methods lab approach could be used to introduce many other types of research accessible to undergraduates, such as working with focus groups composed of their peers, conducting content analysis on published speeches, developing online survey experiments, and/or building an original dataset using newspaper sources or human rights reports. Each lab could introduce a specific research method or be designed to illustrate how different methods can shed light on the same question—similar to our approach in combining interview and survey methods.
The methods lab approach could be used to introduce many other types of research accessible to undergraduates, such as working with focus groups composed of their peers, conducting content analysis on published speeches, developing online survey experiments, and/or building an original dataset using newspaper sources or human rights reports.
Table 3 Tips for Developing New Methods Labs
Although we taught these labs in the context of a thesis course, teaching methods as a series of lab modules could provide the structure for a dedicated research-methods course. This would allow students to practice various methodological skills without the burden of fitting new skills into a substantive research project. The lab approach also is flexible enough to incorporate into a course dedicated to substantive topics in political science. In the context of substantive courses, the labs could be built around research questions drawn from course material.
CONCLUSIONS
During our two-year pilot program using this approach in an honors thesis seminar, we observed students develop more creative senior thesis projects with more varied research methods than those developed in previous years. In addition, whereas we had been cautioned that students would be resistant to being taught research methods, many found the lab approach quite engaging. As one student wrote in a teaching evaluation: “I found the methods labs the best part of this class—they were genuinely fun and challenging tasks.” Although not all students were as enthusiastic, both informal and formal feedback included two common refrains: (1) the labs helped them to feel better prepared to tackle their thesis projects, and (2) this type of explicit methods instruction would have been even more helpful earlier in their college career. In summary, the methods lab approach can serve as an engaging, hands-on way for instructors to introduce students to the mechanics of conducting research in political science.
SUPPLEMENTARY MATERIALS
To view supplementary material for this article, please visit http://doi.org/10.1017/S1049096522001366.
CONFLICTS OF INTEREST
The authors declare that there are no ethical issues or conflicts of interest in this research.
