Grade Inflation and Distributional Changes

Juola (Reference Juola1979) shows grade increases of approximately 0.4 grade points using data from 1965–1973 with the largest increases occurring in between 1968 and 1970. Studies using data from the 1970s and early 80s show either a decrease or only a small increase in GPA (Juola, Reference Juola1979; Bejar and Blew, Reference Bejar and Blew1981; Adelman, Reference Adelman2004). Later studies suggest grade inflation renewed in the late 1980s (Kuh and Hu, Reference Kuh and Hu1999; Rojstaczer and Healy, Reference Rojstaczer and Healy2010; Kostal, Kuncel, and Sackett, Reference Kostal, Kuncel and Sackett2016). Grade inflation leads to changes in grade distributions.

Suslow (Reference Suslow1976) notes there has been a shift in grade distributions with a decrease in the proportion of C’s, D’s, and F’s and an increase the proportion of A’s and B’s. Rojstaczer and Healy (Reference Rojstaczer and Healy2012) suggest that since the early 1980s, the largest change in grade distribution is an increase in proportion of As with corresponding decrease in Bs and Cs. The proportion of Cs shows the largest decrease. Proportion of Ds and Fs also show a slight decrease. The decrease in Ds and Fs may be a function of increasing withdrawals from classes. Adelman (Reference Adelman2004), for example, mentions that an important issue in change in the distribution of grades is a doubling of the percent of withdrawal or noncredit between 1972 and 1992 cohorts. Jewell, McPherson, and Tieslau (Reference Jewell, McPherson and Tieslau2013) suggest grade inflation maybe nonlinear, they find trend, and trend squared variables are significant in explaining grades. In contrast, Kezim, Pariseau, and Quinn (Reference Kezim, Pariseau and Quinn2005) suggest a linear trend in grades over a 20-year period for a private college business school.

Not all studies, however, agree with these findings of grade inflation, suggesting grade inflation does not exist or is a non-issue (Adelman, Reference Adelman2008; Brighouse, Reference Brighouse2008; Pattison, Grodsky, and Muller, Reference Pattison, Grodsky and Muller2013). McAllister, Jiang, and Aghazadeh (Reference McAllister, Jiang and Aghazadeh2008), for example, show increasing grades in an engineering department match increasing achievement potential as measured by standardized test scores. Along these lines, Kostal, Kuncel, and Sackett (Reference Kostal, Kuncel and Sackett2016) note it is hard to indicate the extent of grade inflation, because of the many other factors that can contribute to raising grades; one must remove the influence of these factors to determine the severity of grade inflation. Kuh and Hu (Reference Kuh and Hu1999) explore grade increases after controlling for student characteristics, such as family educational background, academic achievements in high school, and socioeconomic status. They show grade increases net of these specific controls is slightly larger than without controls. Kostal, Kuncel, and Sackett (Reference Kostal, Kuncel and Sackett2016) results, however, show an opposite effect of including control factors. These studies suggest the need to control for institutional, instructor, and student factors when examining potential grade inflation.

Institutional Factors

Studies such as Diette and Raghav (Reference Diette and Raghav2015) and Hernández-Julián and Looney (Reference Hernández-Julián and Looney2016) use fixed effects models to show there are differences associated with students’ grades and their major. Danilowicz-Gösele et al. (Reference Danilowicz-Gösele, Lerche, Meya and Schwager2017) suggest one reason for differences in departments maybe student self-selection. Further, Diette and Raghav (Reference Diette and Raghav2015) and Hernández-Julián and Looney (Reference Hernández-Julián and Looney2016) discuss different departments have different grading norms that may be used to manage demand for courses and majors. Bond and Mumford (Reference Bond and Mumford2019) examine increase in grades across various colleges at Purdue University. Their results suggest differences exist between colleges. They show much of the increases in grades in the college of agriculture are driven by increasing student ability. Department/colleges characteristics and differences, however, are difficult to quantify. As such, to account for department-specific characteristics, departments are added as a level in the models.

Empirical studies and theoretical reviews find quantifiable variables under institutional control are important in student success (McElroy and Mosteller, Reference McElroy and Mosteller2006; Cuseo, Reference Cuseo2007; Kokkelenberg, Dillon, and Christy, Reference Kokkelenberg, Dillon and Christy2008; Henebry, Reference Henebry2010; Wang and Degol, Reference Wang and Degol2016). Cuseo (Reference Cuseo2007, p. 12) in his review on class size concludes

He goes on to state the only argument for large class size is fiscal. Class meeting times may also influence learning. McElroy and Mosteller (Reference McElroy and Mosteller2006) believe “optimal learning times” depends on the students’ circadian rhythm. Skinner (Reference Skinner1985) finds class mean GPAs are lower for morning classes compared to afternoon and evening classes. Henebry (Reference Henebry2010) finds students are more opt to pass a class that meets more than once a week. Skinner (Reference Skinner1985), however, finds an opposite effect on the number of class meeting times. He suggests better attendance in classes meeting only once a week may improve grades. Student performance differs for length of class and meeting times be it because of student circadian rhythms or instructor ability.

Instructor-Specific Factors

The importance of the instructor and instructional method in students’ learning is well documented (Kezim, Pariseau, and Quinn, Reference Kezim, Pariseau and Quinn2005; Kuh et al., Reference Kuh, Kinzie, Buckley, Bridges and Hayek2006; Joyce et al., Reference Joyce, Crockett, Jaeger, Altindag and O’Connell2014). Hoffmann and Oreopoulos (Reference Hoffmann and Oreopoulos2009 p. 491) note “Hard-to-measure instructor qualities may matter more in predicting achievement, even for instructors that exhibit the same age, salary, rank, and gender.” Given the importance and difficulty to measure these qualities especially for data going back to 1985, instructors are modeled as a level in the mixed effect models.

In addition to individual instructor effects, other quantifiable characteristics are shown in the literature to influence grades. Studies (Moore and Trahan, Reference Moore and Trahan1998; Ronco and Cahill, Reference Ronco and Cahill2004; Kezim, Pariseau, and Quinn, Reference Kezim, Pariseau and Quinn2005; Sonner, Reference Sonner2000) suggest grade inflation may be related to the increase in the use of non-tenure track faculty (term used loosely here as different studies used different terminology and length of contract). Kezim, Pariseau, and Quinn (Reference Kezim, Pariseau and Quinn2005) and Sonner (Reference Sonner2000) indicate the use of non-tenured faculty is a cause of grade inflation, whereas Figlio, Schapiro, and Soter (Reference Figlio, Schapiro and Soter2015) find students learn more from non-tenured faculty, which would not be grade inflation. On the other hand, Chen, Hansen, and Lowe (Reference Chen, Hansen and Lowe2021) state instructors hired on a temporary, part-time basis assign higher grades than full-time instructors without a discernable increase in learning. Hoffmann and Oreopoulos (Reference Hoffmann and Oreopoulos2009) and Solanki and Xu (Reference Solanki and Xu2018) find instructor gender plays a small but statistically significant role in student achievement. Both Hoffmann and Oreopoulos (Reference Hoffmann and Oreopoulos2009) and Solanki and Xu (Reference Solanki and Xu2018) indicate a same-sex instructor increases grade performance. Kapitanoff and Pandey (Reference Kapitanoff and Pandey2017) suggest at least for the first exam, females underperform at a higher rate than males if the instructor is also female. Stratton, Myers, and King (Reference Stratton, Myers and King1994) find as instructors’ experience measured as years teaching increases, grades also increase. In summary, both quantifiable and unquantifiable instructor characteristics have been shown to influence student’s performance.

Student Specific Factors

Kuh et al. (Reference Kuh, Kinzie, Buckley, Bridges and Hayek2006) review theoretical perspectives on student learning including sociological, organizational, psychological, cultural, and economic perspectives. They conclude the students “… wide path with twists, turns, detours, roundabouts, and occasional dead ends …” is more realistic than a direct route to student achievement (Kuh et al., Reference Kuh, Kinzie, Buckley, Bridges and Hayek2006, p. 7). In their framework, besides institutional factors, pre-college experiences and student behaviors or engagement also matter in student success. Pre-college experiences include enrollment choices, academic preparations, aptitude and college readiness, educational aspirations, family and peer support, motivation to learn, gender, race, ethnicity, and demographics. Of these experiences, Reference Kuh, Kinzie, Buckley, Bridges and HayekKuh et al. (2006, p. 31) conclude, “In fact, the best predictor of college grades is the combination of an individual student’s academic preparation, high school grades, aspirations, and motivation.” Kobrin et al. (Reference Kobrin, Patterson, Shaw, Mattern and Barbuti2008) and Westrick et al. (Reference Westrick, Le, Robbins, Radunzel and Schmidt2015) indicate the importance of pre-college preparation in explaining the first year success of college students. Rothstein (Reference Rothstein2004), however, finds that much of the SAT predictive power on grades is derived from SATs correlation with high school demographics. Studies also suggest changing demographics may lead to higher grades (Kuh and Hu, Reference Kuh and Hu1999; Kostal, Kuncel, and Sackett, Reference Kostal, Kuncel and Sackett2016). Kuh and Hu (Reference Kuh and Hu1999) suggest increased female enrollments in colleges may lead to higher grades because females tend to receive higher grades, while Bergtold, Yeager, and Griffin (Reference Bergtold, Yeager and Griffin2016) did not find any significant differences between performances of male and female students.

Although brief, this review shows the importance of controlling for factors that may influence grades, therefore, creating a better picture whether grade inflation is occurring. Further, it is not just the institution, instructor, or student factors that determines student success, but rather the interaction of all of these factors.

Data

Data for 14 departments (13 departments and one degree which is considered a separate department, but classes are taught by various departments) in COALS from Spring 1985 through Fall 2019 (the last pre-COVID-19 semester) are utilized giving 35 years of data, far more than many studies. The data starts in fall 1985 because this is the first semester grade data were available. The primary reason behind selecting COALS is access to the data. Another important factor is the wide range of diverse disciplines within COALS which ranges from basic science and technology to economics and agricultural communication. This wide spectrum of disciplines helps make the findings relevant to other universities. In addition, methods utilized are applicable to other universities. Data from the University are supplemented with information from undergraduate catalogs, faculty and department websites, e-mails to instructors, as well as discussions with staff and faculty in the various departments. Departments changing names and combining departments resulted in no data for Forestry and Rangeland Science Departments. A total of 17,696 usable observations are obtained. All data is at the class level. Grading scheme used by TAMU is a four-point scale based on receiving four points for an A, three for a B, two for a C, one for a D, and zero for an F (TAMU does not use grade modifiers of plus or minus). Class mean GPA is based on the number of students receiving each grade. Grade distribution is based on the standard deviation of the grades received in the class. Individual projects, study abroad, and internship classes are not included. These classes are eliminated because students tend to receive A’s or credit for completion in these classes. Distant learning or in-absentia classes are eliminated because they do not have a specific meeting time and number of classes per week, variables used in the model. Classes with variable credits are also eliminated. Further for privacy reasons, information for any classes of less than five students. are not available. Summer classes are also eliminated from the analysis, because most summer courses, especially early in the data set, fall into one of the categories of classes that are dropped. Further it is felt summer courses being shorter in length either five or 10 weeks instead of 14 or 15 weeks (changed over the data set) may not be directly comparable. Data from Texas A&M were given by sections. In some cases, classes in COALS might have different sections taught at the same time by the same instructor. Sections that were held at the same time and by the same professor were merged to reflect the fact that the sections meet as one class. The different sections are generally for accounting and reporting purposes and not a difference in the class delivery.

Student grade information is highly protected by students’ privacy rights laws. Because of this privacy, this research uses class level data rather than individual student data. Over the data period, changes occurred in the data reporting formats and requirements from year to year, specifically in student admissions information and information reported in university catalogs. As such, in many cases, department and administration leadership were contacted to understand data changes. Further, much of the faculty information was obtained by internet searches or contacting instructors directly. These sources of information were particularly used for early years and non-tenure track instructors.

Variable Descriptions and Summary Statistics

Based on the literature, a theoretical model of the three sets of factors influencing grades is presented in Figure 1. Institutional, instructor, and student factors come together to determine the class mean GPA and standard deviation. Empirical variables used as proxies for the theoretical model, their descriptions, and summary statistics are presented in Table 1 and Figure 1. Most of the variables are self-explanatory. Class averages for variables that are based on student information, such as SAT scores, are based on all students in the class that had such information reported. TAMU, COALS, and departments early in the data set did not require students to provide SAT scores or their rank in high school for admission to the University. As such, the class average may be based on less students than in the class, especially in the early years. To account for this non-reporting, a qualitative 0–1 variable for the years 1985–1988 is included with one indicating the years 1985–1988. This variable is then interacted with SAT and High School rank. Variables associated with Association of American Universities (AAU) and foreign universities are included as a proxy for potential differences in instructor background (not quality). AAU is comprised of some of the leading universities in the U.S. and Canada.

Figure 1. Institutional, instructor, and student variables that my affect a students’ grade.

Table 1. Variable description and summary statistics

Qualitative variables’ (listed as 0–1 variables in Table 1) mean values can be interpreted as the share of class in the category. As to institutional variables, most classes meet either in the morning or afternoon, meet twice a week, and are upper division classes. Average class size is 51 students but ranges from five to 349 students. Approximately 78 percent of the instructors are male. Few instructors’ last degree is from a foreign institution. At the time of their graduation, slightly more instructors received their degree from non-AAU-affiliated institutions than from AAU-affiliated universities. Average percent of male students in class is slightly higher than females. SAT scores are somewhat above the national average (College Board, 2015). High school rank average is approximately the 79 percentile (or top 21% of the graduating class in the high school). Students took on average 14 credits per semester with only a small percentage of students dropping or receiving a no grade in a class.

In Figure 2, mean GPA and standard deviation are graphed by semester for all classes. Mean weighted GPA overall years is 3.08. In fall 1985, mean weighted GPAs is 2.88. By fall 2019, GPA has increased to 3.28. Standard deviation declined from 0.85 to 0.73. Examining the percentage of letter grades helps explain the above increase in GPA and decrease in standard deviation (Figure 3). The percentage of A’s increases from about 32% to over 52%. All other letter grades percentages have decreased over the time frame with the percentage of C’s showing the largest decrease (from approximately 22% to a little more than 13%). Mean GPAs and standard deviations by department show considerable variation, with mean GPAs, ranging from 2.79 for the Department of Renewable and Natural Resources to 3.46 for the Department of Agricultural Leadership Education and Communication (Figure 4). Standard deviation and GPA show an inverse relationship caused by the increase in A’s and decrease in F’s. Such departmental differences provide support for the inclusion of departments as a level in the model.

Figure 2. Weighted (by class size) average COALS GPA and standard deviation by semester from fall 1985 to fall 2019.

Figure 3. Percentage of letter grades in COALS in all courses from fall 1985 to fall 2019.

Figure 4. Overall weighted by student mean GPA and standard deviation by department in COALS for the fall 1985 to fall 2019.

Estimation Methodology

When discussing observations that are a part of one group or a cluster, there is risk of drawing inferences about the groups based on simple aggregation of the individual data. Diez-Roux (Reference Diez-Roux1998) argues the relationships observed at the individual level may not hold between the same variables observed in a group, which will result in inferential errors. At the group level, there may be specific group properties that will affect the variables in the same way, regardless of the individual characteristics. In the scope of this study, there may be department-specific characteristics that might follow the same pattern within the same department. This means that data within the same department might be correlated, whereas data from different departments may be considered independent. A similar argument holds for instructors.

Multilevel mixed modeling accounts for both within-individual and between-individual variations (Douglas, Reference Douglas2004; Wu, Reference Wu2019; Shinn and Rapkin, Reference Shinn, Rapkin, Shinn and Rapkin2000). Following Kokkelenberg, Dillon, and Christy (Reference Kokkelenberg, Dillon and Christy2008), Beenstock and Feldman (Reference Beenstock and Feldman2016), and Hernández-Julián and Looney (Reference Hernández-Julián and Looney2016), a mixed effect model is used to incorporate the variations of grading between the departments and individual instructors, as well as to account for the internal correlation while controlling for other factors.

The model is a three-stage hierarchical model with two levels, where the first stage measures the fixed effect or within-individual variation, while the second and third stages measure the random effect or the between- individual variations for instructors and for departments. The estimation consists of separate sets of regressions nested within each other (Greene, Reference Greene2012). Combining the stages, one obtains

(1) $$y_{ijk}=\beta _{0}+X_{k}\beta _{k}+u_{i}+u_{ij}+\epsilon _{ijk}$$

where

k indicates an individual class;

j = 1, …, 1359 instructors nested within i = 1, …,14 departments;

y _ijk —is the individual class mean GPA or class GPA standard deviation;

β ₀—is the intercept to be estimated;

β _k —is a vector of coefficients to be estimated associated with the fixed part of the model;

X _k —is the matrix of independent variables described in Table 1;

u _i —is the particular department GPA (standard deviation) variation from the mean GPA (standard deviation) in the fixed part of the model;

u _ij —is the variation of the instructor mean GPA (standard deviation) from the department mean GPA (standard deviation) in the random part of the model; and

∈ _ijk —is the variation of each class GPA (standard deviation) from the instructor mean GPA (standard deviation) in the random part of the model.

The model is estimated using mixed command in Stata 15 (StataCorp, 2017).