UGR PROJECT PLANNING AND ORGANIZATION

In the Fall 2014 semester, 17 students enrolled in my upper-level “Parties, Campaigns, and Elections” course at the liberal arts college where I work in Central Kentucky. I had taught this course previously but was unsatisfied with the rigor of the out-of-class written assignments. (Course evaluations revealed that my students were equally unimpressed.) In response, I wanted to improve the out-of-class component and decided to replace the previous set of assignments with a meaningful and substantive UGR experience. I previously had not attempted this in a semester-long course, so it was essentially an experiment. For guidance, I consulted the formalized UGR SLOs that my institution had recently adopted. I designed the project with those outcomes in mind, which include the following:

1. 1. Identify and apply the tools, methods, sources, concepts, and ethical standards appropriate to the discipline.

2. 2. Communicate—both orally and in written form and to diverse audiences—the ways in which their intellectual or creative work contributes to broader frameworks, expressions, and discussions in the discipline.

3. 3. Effectively connect multiple ideas and approaches to bring new insights to questions at hand, overcome barriers, and develop an appreciation for the complexity and ambiguity inherent in the research process.

4. 4. Reflect on their work to assess progress toward personal, career, or post-graduation goals.

5. 5. Work independently while also identifying when input, guidance, and feedback are needed.

The “vision” for the project was introduced to my students on the first day of class. ^{Footnote 1} I explained that we would do an “undergraduate research project” and that, organized into research teams, we would empirically investigate an important political science question from a variety of different perspectives. The question posed to my students was: “Why are there more ticket-splitters in Kentucky than in other states? ^{Footnote 2} ” I told them that this is a question I had been thinking about since I moved to the state five years before and that although I had some ideas, I still did not have a satisfactory answer. In other words, the explicit framing for the project was: “Here is a puzzle about how people in Kentucky vote: they vote differently in federal elections than they do in state and local elections. Why? I’m not sure! Even your professor doesn’t have all of the answers. This semester we are going to apply social science tools to answer an interesting and important question about politics here in our community.”

I also informed the students that the project would have an audience outside of the classroom. We would present their research results at the Kentucky Political Science Association (KPSA) conference to be held the following spring. Thus, from “Day One,” this was a serious research endeavor for a public audience that would accomplish more than simply earning a grade for the course.

The first task was to assign students to different research teams. I divided the 17 students into five teams of three or four students each. Each team would be responsible for creating one final product: a research paper investigating the question of Kentucky ticket-splitting. This gave students an opportunity to work in a collaborative setting with one another (and reduced the number of projects for me to supervise from 17 to five). The groups were assigned on the basis of which methodological approach they would take to our collective research question. I determined them in advance and cast a broad array of both quantitative and qualitative approaches to both primary and secondary evidence. Group 1 would perform a quantitative analysis of local exit-poll survey data. Group 2 would perform a quantitative analysis of secondary data sources. Group 3 could perform either a quantitative or qualitative analysis of historical evidence (or both). Group 4 would perform a qualitative analysis of interviews with political elites, scholars, and journalists in Kentucky. Group 5 would perform a qualitative analysis of interviews with “average” Kentucky voters.

It is important to note that most students had previously taken an empirical analysis course from the political science, history, or economics department. Thus, I assumed that they had basic social science analysis skills and that significant class time on the specifics of writing literature reviews, citing sources, analyzing evidence, and so on would not be necessary. (I recommended chapters from the Baglione Reference Baglione2011 text for various parts of the project if they needed a refresher.) I ensured that at least one or two students in each group had previously taken one of these courses; they then were assigned to groups based on their skill set and their indicated first and second preferences. I emphasize that I could not have executed the project in the same way if students did not have at least a basic familiarity with empirical research methods. (See Elman, Kapiszewski, and Kirilova Reference Elman, Kapiszewski and Kirilova2015, who discuss the role of methods training for undergraduates for success in subsequent UGR experiences.)

After the groups were organized, one student in each group was designated as the “team leader” responsible for submitting assignments and organizing team efforts. We used one class session to discuss group dynamics: managing time, being proactive and reliable participants, avoiding the “free-rider” temptation, and so on. ^{Footnote 3}

UGR PROJECT EXECUTION

The first step in the project was for each team to produce a literature-review essay. Each group investigated the “ticket-splitting” phenomenon to learn what political scientists already know about why people ticket-split, as well as why this might be more prevalent specifically in Kentucky than in other states. The groups were given significant leeway in how they decided to collectively draft and write the literature-review essays (see UGR SLO #5).

After the five essays were submitted, I compiled the various “schools of thought” on ticket-splitting from each essay to review as a class. Students had identified 11 distinct explanations and/or hypotheses from their review of the literature, including candidate-centered voting, partisan ambivalence and cross-pressures, political sophistication and socioeconomic dynamics, balancing theory, off-year election effects, post–Civil War Reconstruction effects, and the presence of strong coal mining unions in Kentucky. In class, we summarized each potential explanation and then collectively discussed which methodological approach would be best suited to test specific hypotheses (see UGR SLO #3). This also gave students the opportunity to consider the strengths and weaknesses of quantitative versus qualitative approaches (see UGR SLO #1). After deciding the most appropriate hypothesis for each methodological approach, I assigned them to one or two specific groups in charge of that particular approach(es).

When this step was completed, students then had approximately six weeks to finish the “research-design” and “data-and-analysis” steps of their project (i.e., the former step was due about a month before the latter step). Again, because most students had already taken an empirical-analysis course, I did not use much class time on instruction for these assignments. I was consistently available outside of class, however, to consult with students as they worked on the assignments (see UGR SLO #5). ^{Footnote 4}

After students had performed the data-analysis part of their project and drafted their paper, the next step was to engage in a “peer review” of one another’s full essay. This provided an opportunity to discuss the norms of peer review in academia as well as for hands-on experience (see UGR SLO #1). Each research team was responsible for writing a detailed critique of three other groups’ papers.

The final portion of the project in the last week of the semester was for each team to make a formal oral presentation of their findings to the entire class. This was an opportunity to discuss professional “conference-paper presentation” norms as well as to gain practice for the presentation that they would make the following spring at the KPSA conference (see UGR SLOs #1 and #2). Each team received a grade according to an oral-presentation rubric.

After the presentations, we spent one class in a “debriefing” session to hear results of the various research projects. We compared them to the original hypotheses to determine whether each group had confirmed or disconfirmed it with their research. There were clear patterns, with some hypotheses receiving support across methodological approaches, whereas others received support with one methodological approach but not when tested with another. This was an excellent outcome to illustrate to the students that academic research is not always a “neat and tidy” process; on the contrary, it often is “messy” with many false starts and partial or even contradictory answers (see UGR SLO #3).

Despite occasional snags and delays in the process, each group produced a high-quality original UGR paper that used empirical evidence to test a clear social science hypothesis derived from political science literature. The following spring, one student from each group was accepted to present their paper at a roundtable panel at the 2015 KPSA conference (see UGR SLOs #1 and #2).

GRADING COLLECTIVE UGR EXPERIENCES

A difficult component of this project was how to fairly and meaningfully grade every student on their contribution to the group research projects. My goals were for them to (1) have an experience working together and receiving a collective evaluation as a group (to prepare for future post-graduation experiences likely to be encountered in the workforce; see UGR SLO #4); and (2) hold individual students accountable for their contribution (or lack thereof) to the group as a whole. In total, the project counted for 35% of the final course grade. The grading was weighted as follows:

1. 1. My assessment of your group’s literature-review assignment (5%); theory, hypothesis, and research-design assignment (5%); and peer-review assignment (5%) (each team member receives the same grade).

2. 2. My assessment of your group’s final paper and presentation (10%) (each team member receives the same grade).

3. 3. Your assessment of your own contribution to the group’s efforts, assignments, and final paper and presentation (2.5%) (based on a scoring rubric).

4. 4. Your group members’ individual confidential assessments of your contribution to the group’s efforts, assignments, and final paper and presentation (7.5%) (based on a scoring rubric).

For the self-evaluation and team-member evaluation, I used the rubric available at http://jfmueller.faculty.noctrl.edu/crow/altmangroupprocessrubric.pdf. Students completed this rubric for one another’s efforts; to ensure confidentiality, they were instructed to keep them in a sealed envelope and submit them to me privately.

In general, students seemed to appreciate that there would be a mechanism to ensure a consequence for students who did not “pull their weight” in the group; they reported that it kept themselves accountable as well. Students gave one another frank and honest assessments, equally willing to praise and criticize as warranted. I noted that the peer assessments matched fairly well with the self-assessments; however, on average, students rated themselves slightly higher than their peers rated them.

STUDENT ASSESSMENT OF THE UGR PROJECT

At the conclusion of the semester, I administered an anonymous evaluation instrument specifically designed to measure the effectiveness of the project in achieving various learning outcomes, including the institutional UGR SLOs. In general, student responses were positive: 88% either strongly or somewhat agreed that their “ability to write an academic research paper improved as a result of this experience.” All but one student reported that they either strongly or somewhat agreed that their “ability to problem-solve and overcome obstacles has improved.” Every student agreed that their “ability to effectively connect multiple ideas and approaches to bring new insights to research questions has improved.” Students were less optimistic about the extent to which the experience “helped me productively reflect on or make progress toward a personal, career, or post-graduation goal,” with only 59% either somewhat or strongly agreeing. Overall, however, every student agreed that “this was an effective and educational research experience” (approximately two thirds strongly agreed and one third somewhat agreed).

A common theme in the open-ended qualitative responses was that the group-based organization of the project enabled the students to effectively share tasks among group members. More than half wrote a variant of the following comment: “[A helpful aspect was that] being able to split the workload, compartmentalize, and become an expert in one area allowed for higher quality papers.” A few students appreciated that, although difficult, the group-based project provided insight to their future work environments after graduation. For instance: “[T]he entire group aspect was the most important factor in this whole experience. I think that finally, I’ve done something in class here that I can see immediate applications for in the world waiting for me outside the classroom.” Another student stated: “It is always good to practice working with other people since many careers require working with groups.”

In terms of challenges that the students faced, the single most common theme in the open-ended comments was that it was difficult for them to meet regularly to accomplish the group tasks, given that they were involved in extracurricular activities and also had homework for other courses. A representative comment was: “Since everyone is so busy, it was hard to find times to meet and to have everyone turn in their part in a timely fashion so that the whole group would edit it.” Another comment stated: “We had to meet A LOT. With busy schedules (for all of us) this can get tricky, but we made it work.” I am not persuaded that it is a problem when students must expend significant hours on a project outside of class. However, in the future, I will be more deliberate about providing guidance in terms of time management and group-work dynamics.

The final open-ended question asked for students’ suggestions on how to improve the project in the future. A key recommendation was that the groups conducting qualitative interviews could benefit from additional group members. The interview and transcription process was (arguably) more time consuming than the data-collection and analysis process for the quantitative groups. This is a fair critique that I intend to address in future courses.