1. Conceptualizing bureaucratic capacity

Bureaucratic capacity describes an agency's ability to complete the tasks delegated to it by Congress and the president. The most nuanced conceptualizations acknowledge that capacity has multiple dimensions, each of which must be assessed and measured separately (Geddes, Reference Geddes1994; Fukuyama, Reference Fukuyama2013).Footnote ¹ Despite emerging discussions of “capacity” in the American literature, scholars have made few efforts to conceptualize or measure the concept as it pertains to the US bureaucracy. This study focuses on a key aspect of capacity—human capital—because scholars generally agree that the quality and management of an agency's workforce play a pivotal role in bureaucratic performance. All else equal, agencies whose workforces exhibit higher levels of human capital have a greater likelihood of enacting regulations, distributing benefits, and enforcing the law (Bolton et al., Reference Bolton, Potter and Thrower2015; Drolc and Keiser, Reference Drolc and Keiser2020; Fisher and Shapiro, Reference Fisher and Shapiro2020).

Like the broader concept of capacity, human capital cannot be easily distilled to a single characteristic. A high-capacity workforce will have (1) substantive and procedural expertise, (2) an ability to recruit and retain skilled employees, and (3) an ability to organize itself for efficient team production. Many scholars describe these individual attributes in passing but few—if any—have sought to synthesize them as attributes of a single, latent concept.

For many scholars, bureaucratic capacity means expertise. Expertise describes the knowledge and skills possessed by the agency's workforce. But expertise also has various understandings (Fisher and Shapiro, Reference Fisher and Shapiro2020). Expertise may describe the scientific or technical knowledge brought to the agency by employees’ educations and past careers (Miller and Whitford, Reference Miller and Whitford2016). It may also encompass the practical knowledge attained while working in public service. Experience hones the bureaucrat's professional judgment and teaches them how to use government procedures to advance the agency's interests (Gailmard and Patty, Reference Gailmard and Patty2007; Potter, Reference Potter2019; Fisher and Shapiro, Reference Fisher and Shapiro2020). As Mashaw (Reference Mashaw1983) states, experience teaches bureaucrats “what works and what doesn't when ferreting out information, what evidence is reliable and what is not—things that are an implicit part of the culture of the system but are not to be found in manuals or regulations” (67). These forms of expertise are not necessarily interchangeable and, therefore, a high-capacity agency will have both an educated and experienced workforce.

The agency must also have the ability to recruit and retain skilled, public-service motivated employees. At baseline, an effective system of administrative governance must provide career civil servants with tenure (Geddes, Reference Geddes1994; Weber, Reference Weber1978). The United States's robust system of civil-service laws offers that protection (Skowronek, Reference Skowronek1982). However, agencies must also provide adequate financial remuneration to avoid rent-seeking behavior and corruption. Indeed, the Weberian ideal necessitates the employment of civil servants who need not rely on rent-seeking behaviors (Weber, Reference Weber1978). Even absent rent-seeking behaviors, the lack of competitive salaries makes employment in the private sector a more attractive alternative to government service (Gailmard and Patty, Reference Gailmard and Patty2007). The lack of competitive salaries for many civil servants poses a possible threat to bureaucratic capacity and, therefore, this threat should be minimized in agencies that offer higher salaries (Lewis, Reference Lewis2019).

A subtler attribute of human capital describes the managerial and organizational structures that allow the agency to make use of the diverse expertise within its workforce. Agencies must organize the workforce in ways conducive to team production (Williams, Reference Williams2021). Agencies organize their workforces in ways that further their primary mission, neglecting tertiary tasks they see as less aligned with their core principles (Wilson, Reference Wilson1989; Barkow, Reference Barkow2013; Hickman, Reference Hickman2016). These structures are part cultural and part managerial. Culturally, agencies cultivate esprit de corps to unite the workforce behind the agency's mission. Managerially, the agency must train employees, organize them into teams, and find ways to maximize the exchange of information within the workforce. A poorly organized workforce may fail to see a key aspect of a problem confronting the agency or may move slowly in conveying information across teams.

A broader conceptual issue emerges when attempting to translate this definition into measures. We often think of capacity as an amorphous concept that describes the agency's ability to do something. Too often, however, we fail to ask, “The capacity to do what?” While a workforce may have sufficient expertise and experience to complete one type of task (e.g., policymaking), it may lack the requisite capacity for a different type of task (e.g., law enforcement or adjudication). For example, the Federal Communications Commission (FCC)—an agency charged with regulating telecommunications—may operate comfortably with 1500 employees with law and telecommunications backgrounds. However, the same workforce would struggle to deliver 162 million pieces of mail to over 41,000 zip codes each day. Meanwhile, there is no guarantee that the Postal Service's 330,000 mail carriers would have sufficient expertise to manage the radio spectrum. Does the Postal Service have greater capacity than the FCC because it has more employees? Not necessarily. We can reduce errors in our estimates of capacity by tailoring measures to a single type of activity undertaken by a variety of agencies.

In addition, one should avoid confusing the concept of capacity with bureaucratic performance. Unlike performance, capacity concerns the agency's prospective ability to complete a task. The prospective focus of capacity means that an agency may have a latent ability to complete a particular task even if it chooses not to undertake the activity. Experienced careerists may use their knowledge of administrative procedure to delay policy proposals they dislike (Potter, Reference Potter2019). Alternatively, Congress and the president may enact laws that limit the agency's authority to take certain actions. For example, subject to notable exceptions, only the Department of Justice has the authority to litigate on behalf of federal agencies (Selin and Lewis, Reference Selin and Lewis2018). Yet the nature of administrative work still requires many agencies to employ experienced litigators who would excel at defending their agency's actions in court. Capacity spills over from one task to another. Indeed, Congress and the president delegate new programs to existing agencies to harness this latent capacity rather than create new agencies from scratch (Hickman, Reference Hickman2016).

Finally, scholars ought to remain mindful of agency workloads when studying the relationship between bureaucratic capacity and performance. Poor performance may occur when an agency's workload exceeds its capacity. All else equal, two agencies with the same levels of capacity may experience different outcomes as a result of different workloads. Empirically, many studies of performance require scholars to control for both the capacity of the agency to perform a particular task and the size of the workload attributable to that task. This study provides measures of human capital for the policymaking context, but scholars will also need measures of workload tailored to their particular task of interest.

2. Existing measures of human capital

Existing measures of human capital come in two varieties. The crudest measures rely on the number of full-time employees within an agency (see Bolton et al., Reference Bolton, Potter and Thrower2015; Potter and Shipan, Reference Potter and Shipan2019). Although workforce size is an important component of an agency's ability to complete many tasks, it does not capture the level of expertise or experience within the workforce. For many tasks, such as policymaking, expertise and experience play a greater role in task completion than workforce size. Moreover, measuring the entirety of the workforce introduces measurement error by failing to control for what proportion of the overall workforce is capable of performing the task of interest.

More sophisticated measures rely on surveys of civil servants. For example, Richardson et al. (Reference Richardson, Clinton and Lewis2017) present federal executives with a list of federal agencies and ask, “In your view, how skilled are the workforces of the following agencies?” The authors then aggregate the responses to create measures of workforce skills. Surveys capture concepts, like teamwork, that researchers cannot easily discern from observational data. At the same time, however, they measure perceptions of capacity. Political leanings, recent events, and turf wars influence these perceptions. Moreover, survey-based measures are time bound and often suffer from low response rates within particular government agencies. Unless researchers regularly survey the federal government about its capacity, these measures provide limited utility outside of specific contexts.

The need for new measures of capacity is apparent in the literature. In recent years, theorists have formalized predictions about the effects of capacity on a variety of political phenomena (see Huber and Sanford, Reference Huber and Sanford2004; Ting, Reference Ting2011; Turner, Reference Turner2020). Moreover, the recent experience of the Trump Administration has raised questions about what strategies presidents use to build and deplete capacity within the civil-service workforce (see Benn, Reference Benn2019; Bednar and Lewis, Reference Bednar and Lewis2023). The measures developed here allow scholars to probe differences in human capital to better understand how capacity emerges within the federal bureaucracy and affects policy creation and outcomes.

3. Measuring policymaking capacity

The development of new measures is possible given the wealth of data on the US civil service. The scores developed here measure the ability of an agency's workforce to make policy. All agencies engage in some level of policymaking. Formally, agencies make policy by promulgating rules, drafting legislation or executive orders, and issuing enforcement guidance. They also make informal policy decisions by interpreting vague mandates in statutes to discern the extent of their authority and their obligations under the law. Agencies must adhere to similar procedures when engaged in certain forms of policymaking (see Potter, Reference Potter2019). The high degree of standardization of policymaking activities within the executive branch makes it an ideal task for comparing capacity across agencies.

The Office of Personnel Management (OPM)—the federal government's human-resources department—publishes annually its Enterprise Human Resources Integration (EHRI) database, which includes records of civilian employees working in federal agencies.Footnote ² From 1998 to 2021, EHRI includes over 190 million personnel records.Footnote ³ Each entry in EHRI includes information about the employee's agency, salary, education, occupation, and length of service in a given year. The regular publication of EHRI permits the development of time-series measures of human capital by aggregating these data into agency–year units.

To allow for meaningful cross-agency comparisons, the measures must adequately control for differences in employees’ responsibilities across agencies. Reliance on OPM's occupational classifications addresses this concern. OPM standardizes occupations for all federal agencies based on the knowledge and skills required for the job, its responsibilities, and the level of supervision received by the employee. Federal law requires most agencies to use these classifications when creating new positions and setting salaries. Certain occupations play a particular role in administrative policymaking (Walker, Reference Walker2013). I use OPM's Handbook of Occupational Groups and Families to code occupations that mention responsibilities related to policymaking, such as “developing regulations” or “planning policies” (O.A.2).

Identifying agency policymakers based on standardized job descriptions has three main advantages. First, it ensures that individuals employed in the same occupation have similar responsibilities regardless of which agency employs them. Second, it sets a baseline standard for the qualifications and salary of a given occupation across all agencies. Third and finally, it allows us to identify individuals who perform tasks related to policymaking and remove employees in non-policymaking occupations from the sample. The chosen employees all have responsibilities related to policymaking, and OPM's standardization of their occupations creates a comparable level of necessary qualifications and compensation across agencies.Footnote ⁴

Drawing on the broader conceptualization of human capital, I create five agency–year indicators correlated with the workforce's ability to make policy. Plausible alternatives exist for each indicator, but the measures are robust when estimated with these alternatives. I discuss other possible specifications in the Appendix (O.A.1).

To make policy, agencies need employees with scientific and technical expertise related to their policy domains. Bureaucrats generally acquire this scientific and technical expertise through postsecondary education (Miller and Whitford, Reference Miller and Whitford2016; Fisher and Shapiro, Reference Fisher and Shapiro2020). I measure the scientific and technical expertise within an agency's policymaking workforce as the average proportion of policymaking employees with a college education (mean: 0.74, SD: 0.17).Footnote ⁵ Employees with more experience in the federal government have a greater understanding of how to navigate complex procedures and reconcile competing political and policy considerations (Potter, Reference Potter2019; Fisher and Shapiro, Reference Fisher and Shapiro2020). I measure the experience within the agency's policymaking workforce as the mean length of service of policymaking employees (mean: 15.56 years, SD: 3.31 years).Footnote ⁶

Agencies must be able to recruit and retain these expert and experienced civil servants. To do this, they must offer sufficient salaries to prevent civil servants from engaging in rent-seeking behavior or pursuing other career paths (Weber, Reference Weber1978; Gailmard and Patty, Reference Gailmard and Patty2007). I measure the ability of the agency to recruit and retain civil servants using the average salary of the agency's policymaking employees (mean: $115.85 thousand (2020 USD),  SD: $21.56 thousand (2020 USD)).Footnote ⁷ In addition to the employee's base salary, the salary incorporates any locality adjustment. The locality adjustment ensures that individuals in the same occupation receive comparable wages regardless of the employee's geographic location.

An agency must organize its policymaking workforce in ways conducive to team production. Rarely does a single bureaucrat provide all of the expertise and experience the agency needs to complete a particular task. Instead, bureaucrats work in teams, bringing their own unique perspectives and knowledge to the task (Williams, Reference Williams2021). Measuring the management and organization of an agency's workforce is difficult with EHRI. The personnel records do not tell us anything about whether the agency effectively manages or organizes its workforce. We know, however, that agencies organize their workforces around tasks more central to their missions (Wilson, Reference Wilson1989; Barkow, Reference Barkow2013; Hickman, Reference Hickman2016). Consistent with the approach of other scholars (Bersch et al., Reference Bersch, Praça and Taylor2017), I measure the centrality of policymaking to the agency's workforce as the proportion of total agency employees in policymaking occupations (mean: 50.40, SD: 0.24). This indicator assumes that an agency with a higher proportion of policymaking employees is more likely to have organized the workforce in ways that promote effective policymaking. This indicator positively correlates with other plausible measures of agency management and organization.Footnote ⁸ In addition, validation of the policymaking-capacity scores with time-series surveys of civil servants shows a positive correlation with the reported level of cooperation within the agency, suggesting the measures adequately capture these organizational concepts (O.A.3).

Finally, the number of policymakers within an agency informs how many policies the agency may pursue at one time. Empirical research shows that the number of employees within an agency is positively correlated with policymaking productivity (Bolton et al., Reference Bolton, Potter and Thrower2015; Potter and Shipan, Reference Potter and Shipan2019). Accordingly, the measure includes the total number of policymaking employees (logged) in the agency (mean: 6.28, SD: 2.31).

A Bayesian model aggregates these five variables to produce 5590 yearly policymaking-capacity scores for 261 agencies from 1998 to 2021 (O.A.1). For the 15 cabinet departments, I create both aggregated and bureau-level measures. The resulting scores range from −2.38 (low capacity) to 3.59 (high capacity) (mean: 0, SD: 0.68). The results are robust to other specifications. Measuring these concepts with different indicators or incorporating other possible concepts into the measure produces highly correlated results (O.A.1).

3.1 Content and discriminant validation

The value of the scores depends on their validity relative to other plausible measures of policymaking capacity. I validate the content of the policymaking-capacity scores against the number of policymaking employees in each agency. I use case-oriented content validation to check whether the measures comport with expectations about which agencies have higher levels of policymaking capacity (Adcock and Collier, Reference Adcock and Collier2001). To do so, I draw a set of priors about the levels of policymaking capacity across the administrative state:

1. 1. Agencies whose core missions center around regulation should have higher scores than agencies whose core missions center around other activities, such as law enforcement or benefits distribution.

2. 2. Agencies whose policy domains necessitate high levels of technical expertise should have higher scores.

3. 3. Agencies with known patterns of mismanagement should have lower scores.

These heuristics are not infallible, but they provide a set of criteria by which to compare the two measures.

Figure 1 compares the average policymaking-capacity score for the 15 executive departments and the agencies classified as large independent agencies by OPM (N = 34) against the number of policymaking employees (logged) in the same subset of agencies. The policymaking-capacity scores better comport with our expectations than the measures of workforce size.

Figure 1. Policymaking capacity estimates and total policymaking employees for major agencies.

I begin by explaining the descriptive findings of the policymaking-capacity scores. First, eight of the top ten agencies have strong regulatory missions. These agencies enact regulations that govern nuclear energy and the environment (the Nuclear Regulatory Commission, the Environmental Protection Agency, the Department of Energy), finance and securities (Securities and Exchange Commission, Federal Trade Commission, Consumer Financial Protection Bureau), telecommunications (Federal Communications Commission), and labor (National Labor Relations Board). Although not a regulator, the Office of Management and Budget (OMB) acts as the nucleus of policymaking in the executive branch by reviewing agencies’ proposed policies. Many of these agencies, such as NASA and the Department of Energy, are known for their employment of highly educated experts. For example, the Department of Energy's National Laboratories founded the Human Genome Project and identified the cause of the Cretaceous–Paleogene extinction event.

Many of the bottom ten agencies have missions that revolve around law enforcement, benefits distribution, and other non-policymaking tasks. Although all of these agencies engage in some form of policymaking, policymaking is tertiary to their main missions, especially when compared to the missions of the top ten agencies. This does not mean that these agencies lack the capacity to perform their primary functions. However, it does mean that these agencies are less equipped to make policy relative to their high-capacity peers. The agency with the lowest average score—the Department of Homeland Security (DHS)—polices American borders, waterways, and airports while also adjudicating immigration applications and distributing benefits following disasters. The fact that DHS lacked a policy shop until 2005 highlights the relative inattention it gives to this function (McIntire, Reference McIntire2005). Many of the bureaus within the Department of the Treasury and the Department of Justice (DOJ) also have strong enforcement missions. The majority of Treasury's activities involve auditing tax returns, distributing refunds, and protecting the fisc (Hickman, Reference Hickman2016). DOJ's enforcement bureaus, such as the Bureau of Prisons and the Drug Enforcement Administration, employ over half of its workforce classified as policymaking employees (58.2 percent). DOJ's strong enforcement culture has caused it to neglect many other functions (Barkow, Reference Barkow2013).

Mismanagement has wracked many of these lower-ranked agencies. Both the Federal Emergency Management Agency and Secret Service within DHS have been the subjects of highly publicized scandals and performance failures (Lewis, Reference Lewis2008; Leonnig, Reference Leonnig2021). In the last decade, the largest employer within Treasury—the Internal Revenue Service—has suffered from a significant loss of employees brought about by an aging workforce, government shutdowns, and budget cuts (Weiner, Reference Weiner2013; Kiel and Eisinger, Reference Kiel and Eisinger2018). Since 2010, IRS's appropriations have fallen by 20 percent, and the agency has lost 22 percent of its total staff (Congressional Budget Office, 2020). Likewise, commentators have raised concerns about the ability of certain bureaus within DOJ to make policy. Following President Biden's promise to use the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) to create gun-control policies, commentators expressed doubts about the capacity of the agency's workforce to accomplish the administration's objectives. Former ATF officials expressed “that the agency need[ed] to be restructured, modernized, given adequate resources and managed in a more proactive and aggressive way” if the agency hoped to implement the policies desired by the administration (Thrush et al., Reference Thrush, Hakim and McIntire2021).

By contrast, the size of the agency's policymaking workforce displays almost completely opposite trends. Enforcement agencies with known problems of mismanagement—DHS, DOJ, and Treasury—appear toward the top of the list. In addition, the agency with the greatest number of policymaking employees, the Department of Veterans Affairs, experienced one of the largest scandals related to workforce attrition and recruitment in the last two decades (Katz, Reference Katz2022). The regulatory agencies that appear on the top of the left-hand plot appear on the bottom of the right-hand plot. The Office of Management and Budget—the overseer of all policymaking in the executive branch—appears at the very bottom of the list.

Comparisons of the two measures suggest that workforce size does not adequately capture the concept of policymaking capacity as scholars describe it. The agencies with the highest number of policymakers are not those that scholars describe as more expert or carefully managed. By contrast, the policymaking-capacity scores seem to comport with prior expectations of which agencies have the workforce and management needed to engage in policymaking.

3.2 Construct validation

An alternative means of validating the policymaking-capacity scores is to examine a well-supported hypothesis in the literature (Adcock and Collier, Reference Adcock and Collier2001). Scholars of agency rulemaking have identified a positive relationship between workforce size and successful completion of the rulemaking process (see Potter and Shipan, Reference Potter and Shipan2019). Specifically, Bolton et al. (Reference Bolton, Potter and Thrower2015) demonstrate that the Office of Information and Regulatory Affairs (OIRA) takes longer to review rules when it has fewer employees. This result should extend to all rulemaking agencies. Rulemaking provides an appropriate test of the policymaking-capacity scores because rulemaking is a notoriously arduous process that requires significant technical expertise and experience (see Potter, Reference Potter2019). If the scores properly measure policymaking capacity, then agencies with higher scores should take less time to finalize their rules.

The dataset consists of substantive rulemakings begun during the first terms of the Bush, Obama, and Trump Administrations. I measure Duration of Rulemaking as the number of days between the issuance of the agency's Notice of Proposed Rulemaking and its final submission of the final rule to the OIRA (mean: 488.34 days, SD: 662.84 days).Footnote ⁹ I use the date of the agency's final submission to OIRA because, at that point, the agency has completed its work on the rulemaking. Like most scholars studying rulemaking (Potter, Reference Potter2019; Potter and Shipan, Reference Potter and Shipan2019), I examine the subset of rules deemed “significant” by the agency. These rules have greater policy implications compared to other rules and represent the greatest test of the agency's policymaking capabilities. I discuss the research design in greater detail in the Appendix (O.A.4).

The explanatory variable of interest is the agency's Capacity. I measure the rulemaking agency's level of Capacity in the fall before the start of the new presidential administration. I control for other common predictors of rulemaking success. Agencies may choose to slow down the rulemaking process when they disagree with the proposals of the current president (Potter, Reference Potter2019). I use measures of agency ideology to construct a binary indicator of whether the agency aligns with the current president (Aligned President: 41.6 percent) (Richardson et al., Reference Richardson, Clinton and Lewis2017). I also control for Agency Ideology separately because left-leaning agencies may have a stronger regulatory bent than right-leaning agencies (Potter and Shipan, Reference Potter and Shipan2019). A negative value indicates a more liberal agency and a positive value indicates a more conservative agency. Agencies with greater structural independence often face fewer procedural requirements in the rulemaking process because they are not required to submit their rules to the White House for review. I measure Independence using Selin's measure of decision-maker independence (Selin, Reference Selin2015). A negative value indicates an agency with less independence and a positive value indicates an agency with more independence. Consistent with the need to consider capacity in the context of agency workloads, I control for the agency's workload using the logged number of entries in the Unified Agenda in the fall before the inauguration. I stratify the results by presidential administration to control for time-varying policies within administrations that affect rulemaking procedures.Footnote ¹⁰ To allow for ease of comparison, I normalize all continuous variables prior to estimation.

As Duration of Rulemaking exhibits right censoring for uncompleted or withdrawn rules, I estimate the model using a Cox proportional-hazard model. Because presidents care about finalizing rules so that they can claim credit for the resulting policy, I also estimate the model by censoring all rules not completed by the end of the president's first term in office. A positive coefficient indicates that the agency takes less time to finalize its rules.

Table 1 reports the results. In all three models, Capacity is a positive and significant predictor of rulemaking finalization. For model (2), an agency with Capacity one standard deviation above the mean is 12 percent (hazard ratio: 1.12) more likely to complete an ongoing rulemaking at a fixed point in time. By contrast, an agency with Capacity one standard deviation below the mean is 10 percent (hazard ratio: 0.90) less likely to complete a rulemaking at a fixed point in time. The effect size is larger when examining the agency's ability to complete the rule within a president's first term. The results mirror the expectations set by Bolton et al. (Reference Bolton, Potter and Thrower2015), extending their findings to show that capacity has a broad effect on rulemaking duration in all agencies—not just OIRA. Overall, these results suggest that the policymaking-capacity scores measure the concept of interest: an agency's ability to produce policy.

Table 1. Estimated days to final rule: first term of the Bush, Obama, and Trump administrations

Note: Standard errors clustered at the agency level. All continuous variables normalized prior to estimation.

* p < 0.05; **p < 0.01; ***p < 0.001.

4. Bureaucratic autonomy and policymaking capacity

How do bureaucratic institutions acquire and retain this policymaking capacity? A common theory attributes the acquisition and erosion of human capital within an agency to bureaucratic autonomy (see Gailmard and Patty, Reference Gailmard and Patty2007; Richardson, Reference Richardson2019). Autonomy describes the degree of freedom that agencies enjoy from Congress and the president. Civil servants in policymaking positions select into government service to influence policy in the direction of their sincerely held preferences. Scholars identify two possible mechanisms by which autonomy increases bureaucratic capacity. First, autonomy encourages civil servants to invest in their own expertise to attain their preferred policy outcomes while working for government (Gailmard and Patty, Reference Gailmard and Patty2007). Second, autonomy induces civil servants to remain in public service, foregoing more lucrative private-sector jobs in favor of the opportunity to influence policy without political interference (Gailmard and Patty, Reference Gailmard and Patty2007; Richardson, Reference Richardson2019). The policymaking-capacity scores afford scholars the opportunity to test the relationship between bureaucratic autonomy and capacity.

Agencies derive their autonomy, in part, from the structures that govern their interactions with Congress and the president. Structural independence insulates agencies from congressional and presidential control. Most independent agencies exist outside of the executive departments, removed from the president's direct control (Lewis, Reference Lewis2003). The most insulated agencies are independent regulatory commissions. The commissioners for these agencies serve for fixed terms and have for-cause removal protections (Selin and Lewis, Reference Selin and Lewis2018). In designing these agencies, Congress expressed a belief that independence would give commissioners “an opportunity to acquire the expertness in dealing with these special questions concerning industry that comes from experience” (S. Rep. No. 63–597, 11 (1914)).

Recent work by Devins and Lewis (Reference Devins and Lewis2023) calls into question whether these insulating structures promote capacity building. Devins and Lewis (Reference Devins and Lewis2023) use the Richardson et al. (Reference Richardson, Clinton and Lewis2017) survey measures to compare the levels of workforce skills between independent commissions and other agencies. Devins and Lewis find that there is no statistically distinguishable difference in workforce skills between these agencies. The authors conclude that “independent agencies are not particularly expert, influential or independent” (p. 4). The scores allow us to test the conventional wisdom against the claims of Devins and Lewis (Reference Devins and Lewis2023) using measures derived from observational sources of data.

The dependent variable is an agency's Capacity in a given year. Using measures developed by Selin (2015), I estimate the effect of two types of structural independence: decision-maker independence and political-review independence. Decision-Maker Independence measures how much influence the president has over the selection and removal of key agency officials (mean: − 0.12, SD: 0.87). Political-Review Independence measures the ability of the White House to review the agency's policy proposals (mean: 0.10, SD: 0.95).

I control for several other structural variables to reduce the possibility that agency independence correlates with some other cause of capacity building. I control for Agency Ideology on the grounds that liberal politicians may be more likely to invest in bureaucratic capacity and have stronger incentives to do so in agencies that align with their policy preferences (Benn, Reference Benn2019; Bednar and Lewis, Reference Bednar and Lewis2023) (mean: −0.07, SD: 1.04). I use Richardson et al. (Reference Richardson, Clinton and Lewis2017) measure of agency ideology, which measures the agency's stable ideological leaning in both Democratic and Republican administrations. I also control for the agency's policy domain. I code each agency's main policy area by reading its mission statement and assigning it a topic area according to the Comparative Agendas Project's major-topic codes. The inclusion of topic fixed effects controls for the possibility that Congress and the president invest heavily in agencies concerned with certain issues and simultaneously give these same agencies greater autonomy (Binder and Spindel, Reference Binder and Spindel2017; Miller and Whitford, Reference Miller and Whitford2016). I control for the Age of the agency (in hundreds of years) because newer agencies have had less time to build capacity than older agencies (mean: 0.57, SD: 0.50). Finally, I include year fixed effects to account for time trends that influence policymaking capacity in all agencies. I estimate the model using an OLS regression with standard errors clustered at the agency level.

Table 2 reports the results. Figure 2 plots the predicted effect, highlighting the 2021 scores of notable agencies. A one standard deviation increase in either decision-maker or political-review independence increases bureaucratic capacity by more than a quarter of a standard deviation. The effect is even stronger when Selin's measures are substituted for an indicator of whether the agency is an independent commission (O.A.5). Consistent with conventional wisdom, agencies with higher levels of structural independence exhibit higher levels of policymaking capacity.

Table 2. Model estimates of effect of independence on capacity

Note: Standard errors clustered at agency level.

*p < 0.05; **p < 0.01; ***p < 0.001.

Figure 2. Predicted effects of structural independence on policymaking capacity.

The results suggest that insulating structures intended to protect bureaucratic autonomy promote capacity building. These results stand in stark contrast to those found by Devins and Lewis (Reference Devins and Lewis2023). While agencies like the Securities and Exchange Commission (SEC) and the Federal Energy Regulatory Commission (FERC) have moderate skills according to survey-based measures, these agencies rise above executive agencies in the policymaking-capacity scores. Reassuringly, there is some overlap between the two measures. For example, both the scores and the survey measures show that the Federal Trade Commission has comparatively high levels of capacity and the Transportation Security Administration has comparatively low levels of capacity.

The difference in the results of this study and Devins and Lewis (Reference Devins and Lewis2023) underscores the need to better understand the concept of bureaucratic capacity and how to measure it. The results of Devins and Lewis (Reference Devins and Lewis2023) suggest that federal executives perceive no benefit to these insulating structures. Yet the policymaking-capacity scores suggest that these agencies still do a better job recruiting and retaining higher educated and more experienced civil servants—even when controlling for the agency's policy domain. In addition, the policymaking-capacity scores tell us specifically about the agency's ability to engage in policymaking whereas the survey measures do not ask respondents to consider a particular task performed by these agencies. However, the survey measures may capture hard-to-measure components of human capital, such as teamwork and management, better than the policymaking-capacity scores. In the end, deciding whether these structures adequately promote capacity building requires us to further engage with what we mean by capacity and by what standard we evaluate an agency's capacity.