Introduction
Fewer than two thirds of US high-school seniors enroll in college immediately after completing high school (NCHEMS, n.d.). This share is much lower for students from low-income families, for those whose parents did not go to college, and for those from underrepresented minority groups (Wilbur & Roscigno, Reference Wilbur and Roscigno2016). Indeed, the shares of high-achieving, low-income students who go to college are lower than the equivalent shares of lower-achieving students from higher-income families (Fox et al., Reference Fox, Connolly and Snyder2005).
Cost is a key barrier to college access. Rapidly increasing college tuition affects both the decision to go to college in the first place, leaving some students out of college completely, and college selection, leading some students to enroll at less selective options with lower tuition – sometimes referred to as the ‘sticker price’ – despite evidence that attending more selective schools is good for long-run outcomes (Cohodes & Goodman, Reference Cohodes and Goodman2014; Dynarski et al., Reference Dynarski, Libassi, Michelmore and Owen2021). In response, policymakers have made significant investments in financial aid programs that aim to increase college access, particularly at selective, 4-year institutions. Total grant aid to students is roughly $125 billion per year, quadruple (in real terms) its level a decade ago (Seltzer, Reference Seltzer2017; Trends in Student Aid, 2017). Nevertheless, aid programs fail to reach all students who would benefit from them. One analysis found that more than 20% of California community college students who were eligible for a Pell Grant did not receive it, leaving almost $130 million in financial aid on the table in a single semester (Martorell & Friedmann, Reference Martorell and Friedmann2018). Another study finds that students who do not fill out the FAFSA leave $24 billion on the table (Kofoed, Reference Kofoed2017).
One potential explanation for this gap in take-up is the burdens associated with applying for and receiving financial aid, a process that is notoriously complex and difficult to navigate. Herd and Moynihan (Reference Herd and Moynihan2019) provide a useful framework for understanding these and other types of administrative barriers. First, there may be learning costs: students may not take up financial aid for which they are eligible because they may incorrectly calculate the value of going to college or the true cost of going to college for them. The latter is, in itself, a function of how aid is presented. Second, there may be compliance costs: even for students who understand the rules and availability of aid, the numerous actions required to qualify for financial aid may create too many hurdles. Students must complete FAFSA forms, much more complex than annual income tax returns (The Hamilton Project, 2007); comply with requests to verify their financial and family details; and collect and combine grant and loan aid offers from federal, state, and institutional sources that are not coordinated. Last, students may face psychological barriers when making high-stakes college decisions that limit their ability to take up aid and have long-lasting effects on their lives.
Behavioral science evidence is mixed on whether outreach efforts to reduce such barriers and increase take-up could impact student decision-making. Some studies have found that direct assistance with aid applications (Bettinger et al., Reference Bettinger, Long, Oreopoulos and Sanbonmatsu2012) and a clear guarantee about the specific amounts of aid availability (Dynarski et al., Reference Dynarski, Libassi, Michelmore and Owen2018) have meaningful effects on college enrollment decisions, even without changes in the underlying structure of aid. It is noteworthy that both of these successful interventions tackle compliance burdens as well as informational barriers. In the former, direct assistance is provided in filling out the FAFSA; in the latter, a guarantee of a full ‘scholarship’ eliminates the need to go through the complex aid process. On the other hand, recent attempts to scale up successful ‘nudge’ interventions aimed at encouraging financial aid applications have shown no impact on take-up (Bird et al., Reference Bird, Castleman, Denning, Goodman, Lamberton and Rosinger2019), enrollment (Hyman, Reference Hyman2020), or school choice (Gurantz et al., Reference Gurantz, Howell, Hurwitz, Larson, Pender and White2019).
We contribute to this literature by explicitly testing whether nudges that reduce psychological and learning barriers can impact decision-making on three margins – accessing financial aid, going to college, and choosing which school to attend – in a setting where we can test these barriers head-to-head and in combination. Unlike other studies, we can directly compare the effects of interventions that reframe college aid with those providing concrete and personalized cost information, and can estimate effects both on the outcomes being nudged and on longer-term outcomes of greater long-run importance. This allows us to assess the relative importance of different barriers at one key stage in the process and to understand whether previous evidence that nudges fail (Bird et al., Reference Bird, Castleman, Denning, Goodman, Lamberton and Rosinger2019; Gurantz et al., Reference Gurantz, Howell, Hurwitz, Larson, Pender and White2019; Hyman, Reference Hyman2020) reflects the specific nudges that were selected or represents a limit to what is possible through outreach campaigns.
In collaboration with the California Student Aid Commission (CSAC), which administers state-level financial aid for California (known as the ‘Cal Grant’), we ran two large-scale field experiments (total N = 265,570) that aimed to increase knowledge and take-up of financial aid. The Cal Grant both lowers the cost of college and changes the relative cost of different school options, and past studies have found impacts on enrollment, college completion, and earnings (Kane, Reference Kane2003; Bettinger et al., Reference Bettinger, Gurantz, Kawano, Sacerdote and Stevens2019).
We test several variants of notification letters sent to eligible students, in the middle of their senior years of high school. We find striking evidence that simplified, behaviorally informed letters lead to increased Cal Grant account registration rates. In Year 1, simplifying the letter and adding language emphasizing social belonging significantly increase registrations by 9% and 11% (5.5 and 6.8 percentage points), respectively, compared to the baseline letter. In Year 2, we find that both a belonging message and a social norm message have small positive, albeit statistically insignificant, effects on account registration, of 1.6% and 2.6% (1.1 and 1.7 percentage points), respectively, compared to the baseline simplified letter. However, combining the belonging language with individualized information on net costs significantly increases registration by 4.6% (3.0 percentage points) over the simplified letter.
When we turn to Cal Grant payout, a proxy for enrollment, we do not see statistically significant increases in overall payouts from any of the letter variants. That is, despite sizeable changes in behavior in the first stage of the process – registering for a Cal Grant account – that indicates that one administrative barrier was meaningfully lowered, we do not see second stage impacts on overall take-up of the grant. However, we find evidence that the net cost letters, which aimed to also provide useful information about later stages of the process, significantly changed decision-making on school choice. Specifically, these letters caused increases in enrollment at community colleges, at the lowest cost college of the student's indicated options, and at colleges where they can live at home. While this is only partly in line with the program's goals,Footnote 1 it does indicate that when students are given transparent, clear information that provides individualized estimates of cost differences, they adjust their decision-making. This suggests that both behavioral barriers and information constraints play roles in student decisions.
Our findings have both theoretical and practical implications. Contributing to the literature on administrative burdens, we show that there are clear learning costs and psychological barriers that prevent students from navigating the financial aid process and that can be addressed through simple interventions. Affirming belonging and strengthening a positive social norm can effectively nudge more students to take an important proximate step in accessing financial aid for which they are eligible. Yet, our studies show that unlocking these psychological barriers and jump-starting the process is not sufficient to overcome future barriers, such as understanding complex and personalized details about the costs of tuition. It is only when psychological nudges are combined with salient information on net costs that school choice is affected. A strategy for increasing take-up of student aid and other public benefit programs may involve combining nudges at multiple key pain points with simplification of the overall process to reduce the number of such points and with clearer concrete information on cost and benefits at early stages.
Setting and methods
The sample for these studies includes all letters mailed to students who were eligible for the Cal Grant A and the Cal Grant B High School Entitlement Awards in 2017–2018 and again in 2018–2019. These awards cover all tuition and fees at public 4-year colleges in California for eligible students, with partial coverage at private colleges and small stipends at community colleges,Footnote 2 and are renewable for up to 4 years. Students with high-school grade point averages above 3.0 and family incomes under $100,000 are eligible for Cal Grant A, while those with GPAs above 2.0 and incomes under $50,000 are eligible for the Cal Grant B. The CSAC mails notification letters to all high-school students meeting these criteria on a rolling basis, beginning in mid-November. A letter is triggered when (1) a high-school submits a student's name as meeting the GPA criterion and (2) the CSAC receives the student's FAFSA, indicating that the income criterion is met. The earliest submitted FAFSAs arrive at the CSAC in mid-November and the initial letters are mailed shortly thereafter.
Treatment design
In the 2 years of experiments, the CSAC randomly allocated students to receive letters that varied in language and content. Samples of each letter variant are included in the Supplementary Material. While the letters alone do not reduce compliance costs, the specific letter variants combined insights from behavioral science with evidence on school choice to address various forms of learning and psychological costs.
The baseline (control) letter was a notification letter produced by the CSAC that described the program and instructed recipients to register for Cal Grant accounts on a website, WebGrants4Students, that the CSAC maintains. The CSAC worked internally to clarify the language and presentation of the notification letter from letters used in prior years for the 2017–18 version. Despite this, the letter remained quite dense and contained several undefined acronyms and terms. Moreover, figuring out what one needed to do next to obtain a Cal Grant award required careful reading.
In Year 1, we tested variations of the baseline letter (T1) that measured the impact of simplification (T2) and affirming belonging (T3). T2 drew on evidence from a wide range of behavioral science and communication literature, indicating that simplification can have perhaps the biggest proportional change on behavioral outcomes (e.g. Bhargava & Manoli, Reference Bhargava and Manoli2015; DellaVigna & Linos, Reference DellaVigna and Linos2020). Specifically, by making it easy to understand the substance of the letter and, importantly, making it easy to take the next step, simplification both reduces learning costs and removes frictions that may otherwise lead to inaction. As such, T2 contained the same basic information as the baseline letter, but presented it in a dramatically simplified manner, with far less text and with graphical design features that drew attention to the specific action that students needed to take. In particular, the Cal Grant was described as a scholarship, and the call to action to visit WebGrants4Students was enclosed in a red box in the middle of the letter, along with the ID numbers that students would need to create accounts.
T3, a letter aimed at affirming belonging, used the simplified language from T2 but added sentences that emphasized that the addressee belonged in college and that the CSAC perceived them as a likely college graduate. This treatment drew from studies that show that reducing belonging uncertainty – the anxiety related to cues of non-belonging for otherwise underrepresented groups – disproportionately improves educational outcomes for African American students and women in STEM fields (Walton & Cohen, Reference Walton and Cohen2007; Walton et al., Reference Walton, Logel, Peach, Spencer and Zanna2015). Although most of this literature emphasizes impacts on college performance, the hypothesized mechanism operates through self-perceptions of belonging in college and institutional expectations about a student's potential success. We anticipated that these mechanisms could be strong at earlier stages in the selection process where students are asked to sort themselves into institutions where they may or may not feel that they belong.
Study 2, the following year, was planned after preliminary results from the first study were available. The early results, confirmed by the longer-run results presented below, indicated that the Simplified letter produced dramatically higher account registrations than the baseline letter, and that the Simplified+Belonging letter improved even further on this. Accordingly, in 2018–2019, the baseline letter from the initial study was discarded. The Simplified (T4) and Simplified+Belonging letters (T5) were retained, the first with a slight modification to remove some wording that could imply belonging in the simplified letter, and new variants were added to test additional behavioral hypotheses. We use the Simplified letter (T4) as our control condition in Study 2. Because treatment contrasts were much smaller than in Study 1, we anticipated smaller impacts on outcomes.
To explore additional behavioral interventions in Year 2, we replaced some sentences of the ‘belonging’ intervention with a clear descriptive social norm (T6), emphasizing that many other high-school students were enrolling and utilizing the Cal Grant. A rich behavioral science literature provides wide-ranging evidence that descriptive social norms can impact behavior in various policy areas, especially in contexts where that behavior is relatively invisible to the target population (Cialdini & Trost, Reference Cialdini, Trost, Gilbert, Fiske and Lindzey1998; Goldstein et al., Reference Goldstein, Cialdini and Griskevicius2008; Gerber & Rogers, Reference Gerber and Rogers2009; Hallsworth et al., Reference Hallsworth, List, Metcalfe and Vlaev2017). Whether these nudges are effective depends, in part, on whether the target population thinks of the social norm as applicable to them and on whether it changes expectations. As such, it was unclear a priori whether a social norm intervention would be effective in this context.
Last, we tested the impact of going beyond a framing nudge to provide new information about college costs that might help students understand the aid landscape. Letter T7 looked similar to the belonging (T5) letter, but included on the back a table of the amount of aid available, and the resulting net costs (including tuition and living expenses), for the specific public colleges and universities in California that students listed on their FAFSAs to receive their financial information (which we interpret as a proxy for application). The table had one row for each school listed on the student's FAFSA and columns showing:
1. The student's planned living situation (on-campus, off-campus, with parents).
2. Estimated tuition, fees, housing, and other costs.
3. Estimated total grant aid.
4. Estimated net cost, the difference between columns 2 and 3.
Column 1 was taken from the student's FAFSA, while columns 2–4 were taken from colleges’ cost calculators, populated with the living situation and personalized family and financial information from the student's FAFSA (see Supplementary Material for details). We did not gather the cost information from private colleges, which used widely varying forms that were often not easily scrapable and often requested information not on the FAFSA; from five public colleges in California whose calculators we were not able to scrape, or from out-of-state colleges. When students listed these colleges on their FAFSAs, the table included rows for them but cost information was listed as ‘not available.’
Financial aid is highly individualized, and learning about the net costs of different types of colleges may be an important barrier. As such, we expected that most of the students receiving CSAC notification letters overestimate the cost of attendance and underestimate the aid available to them (e.g., Grodsky & Jones, Reference Grodsky and Jones2007; Scott-Clayton, Reference Scott-Clayton2012). Our letters generally arrived several months before colleges’ official aid offers and were designed to help students be more informed in their planning and thinking about college costs. All of the information that we provided in the comparison table was already available to students – it was taken from public net cost calculators on each college's website, mandated under a signature Obama Administration initiative that aimed to support more informed decisions – but we suspected that many students had not accessed the calculators and that many would have been confused by them if they had (Hopkins, Reference Hopkins2011; Nelson, Reference Nelson2012).Footnote 3 As such, the table provided in T7 may have provided net cost information that was not otherwise in students’ consideration (Table 1).
Table 1. Description of treatments.
All the notification letters focused attention on one key decision, registering for accounts on the CSAC website. While there are many additional compliance hurdles that students face in accessing aid (see Supplementary Material for details), registering for an account is a key step to receiving a Cal Grant scholarship and is a trigger for additional communications about the process. Accordingly, the primary outcome of interest in this study is registration for an account.
However, the main goal of the aid process is not to generate WebGrants4Students accounts but to help students go to and pay for college, and the CSAC letters aim to help students better understand their choices. As secondary outcomes, we measure whether a Cal Grant was paid out for the student in the Fall of the following academic year, and, if so, to what school. Payout of a grant at a particular school implies that the student was admitted and decided to enroll, that she completed all verification processes for the Cal Grant over the summer after high-school graduation, and that the college claimed the Cal Grant on her behalf (which is intended to be automatic, but may in practice require some intervention by the student). By improving students’ understanding of the process, the letters could plausibly affect all but the admissions decision, although they were not timed to affect decisions on where to apply. We outline the process in greater detail in the Supplementary Material.
Experimental design
Both studies were pre-registered on Open Science Foundation before any outcome data were available, on December 27, 2017 (registration redacted for review) and November 29, 2018 (registration redacted for review), respectively.
Students were randomly assigned to receive one of three (in 2017–2018) or four (2018-2019) letter variants. In both experiments, randomization occurred at the school level, and all students within a school received the same letter variant. This was done in order to reduce the potential for contamination. Random assignment was within eight strata, based on the high school's count of Cal Grant-eligible students and the share whose awards were paid out in the previous 2 years. Within each stratum, one third (Study 1) or one quarter (Study 2) of high schools were assigned to each treatment arm. Randomization was independent in the 2 years (Supplementary Table S2), and schools were assigned to treatment groups before the first letters in that study were mailed.
We consider all letters sent by June 1st. In Study 1, this was roughly 134,000 letters; in Study 2, it was 131,000. Letters were mailed in batches, and, due to mailroom constraints, the timing of mailing sometimes varied by a few days across treatment arms. We discuss this at greater length in the Supplementary Material; we find no evidence that the timing of mailing affected our outcomes or that it confounds our estimates of treatment effects.
The CSAC sent a reminder e-mail in early February to all students who had received letters to date, encouraging them to register for their account if they had not already done so. This e-mail was identical for all students across treatments. The CSAC also makes other efforts to reach out to students, including encouraging high-school counselors to contact students who have not yet registered (which the counselors can check via their own WebGrants accounts). These too are likely to be similarly distributed across treatment groups.
We estimate experimental impacts via simple OLS (linear probability) regressions of each outcome on the assigned treatment, with fixed effects for assignment strata. Studies 1 and 2 are analyzed separately, and for each, the standard errors are clustered at the level of the unit of assignment, the high school.
Most analyses use the full sample of students who received notification letters. In some of our analyses of college choice, we limit the sample based either on the set of colleges listed on the FAFSA (determined before the letter was sent) or on students whose Cal Grants were paid out at some college. The latter is a post-treatment outcome, though as we show there is no effect of the treatment assignment on this outcome.
Outcomes
We present results for three pre-registered primary outcomes: registration on the CSAC online portal, Cal Grant payout, and choice of a specific college. Our pre-registered analysis plan included a fourth primary outcome, enrollment in college, which we cannot distinguish in the administrative data from payouts.Footnote 4 As such, we do not consider it separately here. Registration and payout are coded as binary outcomes. When we examine college choice, we estimate impacts on enrollment at a college in each of several specific segments (e.g., the UC), as well as enrollment (proxied by Cal Grant payout) at the college that had the lowest indicated net cost among all those listed on the student's FAFSA for which we were able to obtain net cost information.
The last of these outcomes was the most complex to construct, as the CSAC scraped college cost calculators only for students in the Net Cost treatment arm. To examine college choices, we need comparable measures for students in all arms. We, therefore, need to impute net costs based on the information in the Net Cost treatment arm. Fortunately, this is not too difficult, as the net cost calculators are deterministic so it is necessary only to recover the formula that they use. To do this, we fit flexible statistical models to the data from the Net Cost arm, then use these models, along with inputs to the cost calculators (which we have for all students), to impute net costs. To ensure comparability across treatment arms, we then use the imputed cost information for all students, even those for whom non-imputed data is available.
The 34,610 students assigned to the net cost treatment listed 169,093 (non-unique) colleges on their FAFSAs, of which 120,636 (71%) were among the 138 public colleges where we could scrape net cost information. For each of these 138 colleges, we fit a random forest prediction model for net costs, using only students in the Net Cost treatment arm who listed that college and using the FAFSA information that was used to populate the calculators as explanatory variables. These random forest models were highly successful at uncovering the college formulae, even in cases where we had relatively few observations – the predictions from the random forest models correlate 0.996 with the cost calculator output in a hold-out sample. We then use these models to predict net costs for each student-college combination in all treatment arms and use these predictions to identify the lowest cost school among all those that a student listed for whom net cost information was available. Students who attended schools other than this school, whether net cost information was available or not, are coded as not attending the lowest net cost school.
Results
Summary statistics
Table 2 shows summary statistics for the samples, separately for the two yearly cohorts. In our sample, mean GPAs are a bit above 3.0, while mean family incomes are under $30,000. Nearly 90% of students are eligible for Cal Grant B, aimed at students with family incomes below about $50,000 (depending on family size), while a little under 60% of students are eligible for Cal Grant A, which has a higher income limit but more demanding GPA requirements. The typical student attends a high school where two thirds of students are eligible for free- or reduced-price lunches. 57% are eligible for Cal Grant A, while close to 90% are eligible for Cal Grant B.
Table 2. Summary statistics.
Notes: Parent incomes reported as below 0 or above 100,000 are set to missing. This accounts for 1.1% of observations each year. Colleges listed on FAFSAs are classified into four segments: California community colleges, California State University, University of California, and private. Net cost information is available only for the first three of these.
The net cost treatment (T7) relies on information that students provide about which colleges they are considering. Students list colleges on the FAFSA to designate that their information be shared with those colleges for the calculation of financial aid offers. The average student lists around five colleges. In Year 2, when we incorporated net cost information, we were able to calculate net costs for an average of 3.8 colleges for each student, all public. We explain the process of obtaining net cost information in the Methods section and in the Supplementary Material in more detail.
The last panel of the table shows average outcomes. About two thirds of students registered for Cal Grant accounts, our primary outcome. In Year 1, 61% had Cal Grants paid out in the fall, with an additional 1.5% paid out in the spring. In Year 2, we have only fall payout data, with notably lower total rates than even the fall data from Year 1, perhaps attributable to delayed reporting from colleges.
Columns 3 and 6 of the table report p-values for tests that the indicated variable has identical means across treatment arms. These are well above standard thresholds for all pretreatment variables, indicating that randomization was successful. Previewing our main results, we reject the null hypothesis of equality for account registration, but not for grant payout.
Main outcomes
Table 3 presents our main results. In Year 1, 62% of students who received the control letter created accounts. The Simplified letter (T2) increased this by 8.9% (5.5 percentage points), while the Simplified+Belonging letter (T3) increased it by 11% (6.8 percentage points). Both are highly statistically significant, individually and jointly. Effects of the two treatments on Fall Cal Grant payout are much smaller, around 0.5 percentage point, and not statistically significant. Estimates are nearly identical when we include Spring payouts, shown in the Supplementary Material. In Year 2, the baseline letter (T4) was the Simplified letter from Year 1. The Simplified+Belonging (T5) and Simplified+Social Norm (T6) treatments had small, statistically insignificant effects on account registration relative to this, with point estimates of 1.1 and 1.7 percentage points, respectively.
Table 3. Effects on account registration and Cal Grant Payout.
Notes: All specifications include stratum fixed effects. Standard errors are clustered at the high-school level. The control group in Year 2 received a similar letter as the Simplified treatment group in Year 1.
The combined Net Cost letter (T7), however, did raise account creation by 4.6% (3.0 percentage point) over the baseline T4 letter. This is statistically significant on its own; in addition, a joint test of the significance of the three treatment effects together, which is not subject to multiple testing issues, is significant at the 1% level. The incremental effect of adding net price information to a letter that includes Simplified+Belonging language is twice as large as the incremental effect of Simplified+Belonging relative to Simplified in either Year 1 or Year 2. We can also reject the null hypothesis that the Simplified+Belonging (T6) and Net Cost (T7) effects are the same in Year 2, with a p-value of 0.03.
When we turn to effects on Cal Grant payout in Year 2, we again see small and statistically insignificant effects. Suggestively, the largest point estimate, 0.9 percentage point, comes from the Net Cost letter, but this is not statistically significant.
College choice
The results in Table 3 indicate that our behaviorally informed letters were effective at getting students’ attention and inducing them to register accounts, but they do not support the hypothesis that the letters increased overall college enrollment. As per our pre-registration, we next investigate further whether the letters impacted school choice. We are particularly interested here in the Net Cost letter, which aimed to make cost differences among colleges that the student was already considering more transparent and more salient.
Table 4 reports summary statistics for the gross prices and net costs that students faced, at the student-by-school level. The average total price of college before aid is over $23,000 per year, but the Cal Grant and other grant aid bring this down to a net cost of just over $9000. We also show estimates separately for each of California's three public education segments: the 2-year community colleges (CCs); the 4-year, moderately selective California State University (CSU); and the 4-year, more selective University of California (UC). Gross prices vary dramatically across the segments, from just over $10,000 at the CCs, where students are more likely to plan to live with parents, to nearly $35,000 at the UC. Net costs are much less variable, but on average are also lower at CCs ($5500 vs. around $10,000 at the 4-year segments). This masks a fair amount of variability, however: for 41% of students who listed a UC school, the lowest price was in this segment, falling to 33% when we exclude students who listed no CSU or CC schools.
Table 4. Detail about net prices.
Notes: Estimates are based on random forest predictions of costs and aid reported by college cost calculators. Lowest costs are defined only over colleges for which calculator output is available.
Table 5 presents estimates of the impact of the letters on Cal Grant payout in each of the three public segments and at private colleges. We present results for both Year 1 and Year 2, first for the full samples and then, to examine college choice as distinct from college enrollment, for the subsample of students whose Cal Grants were paid somewhere. We find evidence that the Net Cost letter (T7) shifted students toward community colleges, both from the other public segments and from private colleges. This is consistent with the evidence that community colleges are often the cheapest option and the hypothesis that the Net Cost letter (T7) made comparable costs more salient.
Table 5. Effects on school choice.
Notes: All specifications are linear probability models for payout of the Cal Grant at a school in the indicated category, including strata fixed effects. Samples in columns 5–8 include only those students whose Cal Grants were paid out. Standard errors are clustered at the level of the high school.
To probe this further, Table 6 presents estimates of the effect of our various Year-2 treatments on the likelihood of enrolling at the school with the lowest net cost among those listed on the FAFSA. Students who did not enroll at all, or who enrolled at a school other than those listed schools for which we could calculate net costs, are treated as failures for this outcome. In column 1, we see that the net cost letter raised the probability of choosing the lowest net cost option by 10.4% (1.6 percentage point). This coefficient is statistically significant considered on its own (p = 0.03), though the joint test of all of the treatment effects being zero is only marginally significant (p = 0.07). The effect is similar when we limit the sample to students who listed schools from at least two segments on their FAFSAs, who typically face larger contrasts in net costs (column 2). When we separate the sample by whether the lowest net cost option was a community college (column 3) or some other choice (column 4), we see that the effect is concentrated in the former.Footnote 5 The effects generally grow when we limit our sample to students who enrolled at some college, so that we are examining only the intensive margin of choice between colleges rather than the extensive margin of going to college (panel B).
Table 6. Effects on the likelihood of enrolling at the lowest net price option.
Notes: All specifications are linear probability models, including stratum fixed effects. In panel B, samples are limited to students whose Cal Grants were paid out somewhere. Standard errors are clustered at the level of the high school.
The Supplementary Material include additional specifications probing the role of living situations in driving these results. Net costs are generally lower when students indicate that they will live at home than when they list the same schools but indicate plans to live on- or off-campus. The net cost letter causes students to shift their enrollment toward schools where they plan to live at home, and we find marginally significant evidence that it increases overall Cal Grant payouts for students who listed at least one live-at-home option on their FAFSAs.
Heterogeneity
In exploratory analyses, we examined the heterogeneity of effects along several dimensions. Table 7 presents estimates separately for students eligible for Cal Grant B, with family incomes under $50,000 and high-school GPAs over 2.0, and those who are eligible for Cal Grant A but not Cal Grant B. Cal Grant B students come from lower-income households. As such, they are likely to be more reliant on financial aid, and perhaps less well informed, than non-Cal Grant B students. We find that the letters’ effects on account registration are somewhat larger for the Cal Grant B students, particularly in Year 2. Effects on enrollment at the cheapest college are also concentrated in this group.
Table 7. Heterogeneity by Cal Grant B eligibility.
Notes: All specifications are linear probability models, including stratum fixed effects. Standard errors are clustered at the level of the high school.
Table 8 presents estimates of variation by the date that the notification letter was sent. We distinguish letters sent before and after February 1. We hypothesized that later notification letters might have less effect on college choice, as they may have arrived after, or only shortly before, the financial aid letters that they were meant to preview. On the other hand, late FAFSA filers may be substantively different on unobservables than their early filer counterparts. For example, late FAFSA filers may be less familiar with the overall aid process, may be less settled in their college-going plans, may be more present biased, or may be less organized. The evidence is more supportive of the latter hypothesis. Effects of letters on account registration are somewhat larger for the late filers than for those who filed earlier. Similarly, the effect on payout at the cheapest option is driven by the late FAFSA filers. In addition, for this subgroup (though not for the overall population), we observe a statistically significant positive effect on Cal Grant payout.
Table 8. Heterogeneity by FAFSA filing date.
Notes: All specifications are linear probability models, including stratum fixed effects. Standard errors are clustered at the level of the high school. Early and late FAFSA filers are defined based on letters sent before and after February 2.
Discussion
Students finishing high school must make highly consequential decisions about whether to attend college and where to enroll, with enormous financial implications for themselves and their families. Many lack access to quality college counseling, and may not have accurate information about the costs of their different options or about how to access available aid. Like many other financial aid programs, the Cal Grant program is intended to make college affordable for low- and moderate- income students. However, it can only have limited effects on these students’ educational attainment if students do not know it exists, understand how to access it, understand how it affects their net costs of enrollment, and believe it is for them.
We conducted two large-scale randomized controlled trials to explore how low-cost letter interventions could impact some of these barriers. Evidence from both studies suggests that the language and framing of the letters can meaningfully impact student decision-making. Those who received simplified letters were substantially more likely to register for WebGrants4Students accounts by June of their senior year of high school. Letters that added language emphasizing that the student belonged in college were even more effective at prompting registration than the basic simplified letter. Changing letter language is free, so the resulting 7–9% increases in take-up came at zero cost. Importantly, those who received personalized information on the net cost of attending the colleges they had chosen were not only much more likely to register, but they were also significantly more likely to choose a low-cost school.
Our findings contribute to three literatures. First, we contribute to a growing literature on administrative burdens. We focus on financial aid, but the gap between financial aid availability and aid take-up mirrors administrative burdens in other policy areas: 12.5% of eligible SNAP recipients do not take up SNAP (Crouse & Macartney, Reference Crouse and Macartney2020), 20% of EITC-eligible households do not take up the Earned Income Tax Credit (Iselin et al., Reference Iselin, Mackay and Unrath2021); and 75.5% of TANF-eligible individuals do not take up that program (Crouse & Macartney, Reference Crouse and Macartney2020). Our findings suggest that vastly simplifying communications and directly addressing psychological barriers may be critical to helping people start a long and complicated process. This is particularly important for the types of administrative processes where an early pain point (like registering on a website, or registering to vote) can severely limit options down the line. Moreover, although many recent studies show the impact of reducing learning costs by informing people they are eligible to receive a program, our study emphasizes a very different type of learning barrier: being able to carefully and correctly compare the costs and benefits of different types of program offerings. In our setting, reducing learning costs and psychological costs in tandem affects both short-term and longer-term behavior.
Second, our experiments point to both the promise and potential limitations of using low-cost nudges to move behavior. Our findings are largely optimistic. A zero-cost tweak to letters significantly increases the percentage of students who take the desired action, in this case registering for an account. The magnitude of the observed effect on our primary outcome is four to five times larger than the average effect of a government nudge in the USA (DellaVigna & Linos, Reference DellaVigna and Linos2020). Yet a successful nudge that moves proximate behavior does not automatically affect behavior down the line. Future unaddressed compliance costs can still limit the impact of a behavioral intervention. Making costs more salient, in combination with other nudges, on the other hand, not only moves immediate behavior, but it also has long-term consequences on school choice.
Last, our findings contribute directly to the growing literature on the role of costs in college decision-making. The explanations for why high-achieving low-income students do not go to college at rates that seem optimal span a wide range of literatures. Some include emphasis on present bias (Dynarski et al., Reference Dynarski, Libassi, Michelmore and Owen2021); a scarcity mindset (Mullainathan & Shafir, Reference Mullainathan and Shafir2013); misunderstanding of the benefits (Hoxby & Turner, Reference Hoxby and Turner2013), or negative social identities (Lavecchia et al., Reference Lavecchia, Liu, Oreopoulos, Hanushek, Machin and Woessmann2016). While many of these factors play a critical role in whether students take action in a complicated process, we show that at least among students who have shown some interest in going to college, and have already navigated the complex FAFSA process, at the end of the day, cost is still a primary driver of school choice. Whether or not students should be encouraged to attend the lowest-cost university is a policy question outside the scope of this study – though enabling informed decision-making was part of the intent of the policy mandating creation of the cost calculators upon which we rely. Rather, this study points to the relative importance of cost when costs are provided in a comparable and clear way.
Our studies have several limitations. First, we could not vary the timing of communication. The process of personalized communication from the CSAC begins after a student has already filled out the FAFSA, a complicated first step of the process that may deter many students. As such, we cannot generalize our results beyond a population of already motivated students who have managed to navigate the first step in a long process. An earlier intervention may have been more effective or may have captured a different subpopulation of high-achieving low-income students. Second, while we can observe Cal Grant payouts, we cannot observe important related components of enrollment – whether students were admitted to the colleges of their choice, whether they enrolled but did not take up the Cal Grant, and whether they received other financial aid for which they were eligible. Last, while we used existing information from college websites to populate our Net Cost letters, we cannot separately verify if the college calculators correctly identify the various costs associated with going to a specific college, and it is possible that more accurate information would have larger effects.
Importantly, this study has practical implications for policymakers eager to use insights from behavioral science to improve service delivery. While our effect sizes are particularly large, relative both to the broader nudge literature and to their cost, we show that different interventions may be most effective at targeting different barriers in a larger, extremely complex administrative process. A range of nudges have positive effects on the proximate outcome, account registration, but only our treatment making comparable information about net costs more salient had an impact on college choice. A policymaker aiming to fundamentally shift college access may consider interventions at various pain points, which also consider different types of learning, compliance, or psychological barriers at each stage in the process. These could include direct assistance in filling out the FAFSA (Bettinger et al., Reference Bettinger, Long, Oreopoulos and Sanbonmatsu2012); early and inclusive language on financial aid availability, such as that presented in this article; as well as comparable and salient information on costs either from financial aid agencies, as in this article, or from the colleges themselves (Dynarski et al., Reference Dynarski, Libassi, Michelmore and Owen2021). Policymakers could design and test improvements to the compliance process itself, such as reducing the number of steps required to receive financial aid, and introducing personalized targeting significantly earlier in the process. Last, given the importance of cost comparisons in student decision-making, policymakers may choose to emphasize the potential benefits of 4-year colleges in their communications if they aim to encourage attendance at 4-year institutions.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/bpp.2022.1.
Acknowledgements
We are very grateful to the CSAC for its role in this project. Lupita Alcalá, David O'Brien, Catalina Mistler, Jessica Moldoff, and Patrick Perry, in particular, were close collaborators at every stage. We also thank Christian Osmeña for his assistance in fostering this collaboration. Miranda Boyden and Samantha Fu provided excellent research assistance. We are grateful to Laura Szabo-Kubitz, Jessica Thompson; seminar participants at the University of Nevada, Reno; and participants at a conference sponsored by The Institute for College Access and Success for feedback. This research was made possible through the California Policy Lab's data infrastructure. This work reflects the views of the authors and not necessarily the views of our funders, the Regents of the University of California, or the CSAC.
Financial support
Funding for this work came from the Abdul Latif Jameel Poverty Action Lab, Arnold Ventures, the University of California Office of the President, Multicampus Research Programs and Initiatives, MRP-19-600774 and M21PR3278, and The James Irvine Foundation.
Competing interest
The authors declare that they have no competing interests.
Open access
