The central claim of this book is that understanding the moral salience of algorithmic systems requires understanding how they bear upon the autonomy of persons. In Parts II through IV of the book, we explain in detail several different ways in which algorithmic systems are important in relation to autonomy and agency. But before we can do that, we have the basic ground-clearing task of providing an account of autonomy and its value. The discussion in this chapter will set out the basics of our view. However, autonomy is a foundation for many of the concepts and arguments we develop in later chapters, such as responsibility, liberty, paternalism, and democratic legitimacy. We will explain these concepts in more detail in the chapters that draw on them.
We begin by setting out some basics of autonomy, including some key distinctions. Then we explain two broad categories of competing views, focusing on exemplars of each. We offer an ecumenical account of autonomy that incorporates features of both psychological autonomy and personal autonomy, and which requires both procedural and substantive independence. The chapter concludes with an explanation of some of the ways that considerations of autonomy bear upon our arguments in later chapters.
2.1 Autonomy Basics
Autonomy is at root self-government. Individuals can (to some degree and with more or less success) develop their own sense of value, make decisions about what matters most to them, and act accordingly. But just below the surface of that general statement is a rich, complex, and deeply contested set of questions about the scope, nature, and moral importance of autonomy. Our task in this chapter is to offer an account that shows our philosophical commitments, positions itself within some of the weightiest philosophical debates about autonomy, and explains where (and why) we decide not to make stands. The account we offer here is lightweight and ecumenical. It is lightweight because it takes on minimal commitments. It is ecumenical in that it is compatible with a broad range of views. This approach can go a long way in helping understand the moral importance of algorithmic systems without having to resolve some of the deepest and most vexing disputes about the nature and scope of autonomy.
We can get a sense of the basic contours of autonomy by considering some archetypical ways that autonomy can be undermined.Footnote 1 For one, a person’s actions and preferences are not autonomous to the extent that they are the result of coercion or deceit. Where an employer gives an employee the option of working after they have clocked out or being fired, the employee’s choice to keep their job is of course their genuine preference. However, that choice is not autonomous because it is constrained in an illegitimate and coercive way. Suppose instead that the employer constrains the employee’s options through deceit. The employer lies to the employee that the company will fail if the employee does not put in extra, unpaid hours after they have clocked out. Again, the employee’s choice to do unpaid work is based on their genuine desire to keep the company afloat and retain their job. It is not, however, an autonomous choice because they have been deliberately deceived.
A more subtle type of case concerns adaptive preferences. Suppose that the employer continuously asks the employee to work extra, unpaid hours in order for their business to thrive. The employee believes that other employment options are limited and fears losing their job if they do not do the unpaid work. Although it is a burden, the employee does indeed want the business to thrive. They may consciously embrace the practice to make the burden seem manageable. This would seem to conflict with their autonomy in that they formed their preference to work extra, unpaid hours in response to illegitimate pressure from the employer and their belief that other alternatives were bad. Finally, suppose the employer comes to rely on the employee in a close working relationship, provides appropriate encouragement, and genuinely values the employee’s skill and professionalism. Indeed, the employer values it so much that they regularly promote other, similarly qualified employees to better-paid, management positions so that they can keep drawing on the employee’s talents directly. The employee may come to think that the management positions are not really attractive anyway, as they involve a steep learning curve, increased time demands, and working with people they do not know. These are, in the employee’s mind, grapes that are out of reach and probably sour anyway. What the employee prefers conflicts with their autonomy in that they formed their preferences in response to their limited opportunity.Footnote 2
The key to each of these examples is that just what it means for individuals to develop their own sense of value, the conditions under which individuals’ decisions are their own, and precisely when people act according to their own preferences is itself a nuanced question. Hence, it is not enough to say that autonomy is simply a matter of people being able to do what they want. Rather, the deeper question is this: Under what conditions are people and their choices, preferences, and values properly understood as autonomous?
The view that we will advocate here is that although autonomy of preference and choice is important, this sort of autonomy is limited. A fuller understanding of autonomy will focus on autonomy of persons, which is to say the social conditions under which a person is autonomous. Our view, though, is ecumenical in that it incorporates both psychological views and personal autonomy views. The reason is that the normative considerations substantially overlap, as we explain in Section 2.4.
2.2 Some Distinctions
With our first gloss in mind and having described several ways in which persons’ autonomy may be impinged, it will be useful to make a few distinctions. After doing so, we can fill out our conception.
2.2.1 Global versus Local
Autonomy can apply to a relatively narrower or wider range of circumstances. For example, a person may be autonomous with respect to local decisions. That is, they may be able to make decisions about actions with immediate effect and may be able to ensure that those decisions comport with their values. Suppose, for example, that Ali exercises substantial control over most aspects of her life and is able to do more or less as she pleases. However, suppose that her employer routinely assigns her to projects that she does not like. When Ali asks for different responsibilities, the employer ignores her. This is a case in which Ali lacks local autonomy over the circumstances of her employment, and that is true regardless of whether she could find another job relatively quickly and regardless of whether she can effectively govern other facets of her life. In other words, her global autonomy is consistent with her lack of local autonomy.Footnote 3
It is also true that a person may be locally autonomous but lack global autonomy. Suppose that Bari lacks financial resources, lives in an isolated community, and has had little opportunity to develop her talents. And suppose that her family and community expect her to fulfill a strict set of social obligations: care for her siblings and older relatives, cook for the family, and obey her husband. However, she has a great deal of latitude in how she fulfills those expectations. Bari, in this case, lacks global autonomy even while being able to exercise local autonomy. Notice that Bari may lack global autonomy either because the social expectations placed upon her are strictly enforced or because she has internalized those expectations and formed adaptive preferences favoring her actual circumstances.
Distinguishing local and global autonomy matters in two ways. First, particular cases of coercion and manipulation are wrong (when they are wrong) at least in part because they involve failures to treat people as autonomous, but they need not be so substantial as to undermine a person’s overall ability to govern their life. Second, the fact (if it is a fact) that some groups of people are not currently the subject of manipulation and coercion is not enough to ensure that their autonomy is respected globally. They may have developed values and preferences under oppressive or otherwise limiting circumstances, or their circumstances may be constrained overall.
2.2.2 Capacity, Exercise, Successful Self-government
Closely related to the distinction between global and local autonomy are a number of different meanings for autonomy that are relevant in moral and political philosophy. Joel Feinberg, for one, distinguishes the capacity to govern oneself, successful self-government, the personal ideal of self-government, and moral claims that one might assert that reflect a person’s sovereignty over theirself. Footnote 4
The first meaning of autonomy is the capacity for (global) self-government. There are a number of rock-bottom, baseline capacities that a person must have to self-govern. They must be able to consider the world and make rational assessments and decisions. “Rational” here means only that there is a close connection between facts and inferences, and a person’s decisions either line up with well-enough ordered assessments of facts and reasons, or they make decisions fully aware that they do not line up. The second issue of capacity is that a person must not be so profoundly damaged, and must not have had their world so dramatically circumscribed, that their understanding of it prevents well-enough ordered assessments of facts and reasons. Capacity autonomy does not require much; it is a bar low enough that adults can generally clear it. Note that “capacity” is distinct from “potential.” Infants and young children lack the (present) capacity to be autonomous, but they have the potential to develop that capacity. So the “capacity for autonomy,” we might say, refers to the current state of a person and whether they can exercise self-government at this moment so long as the situation in which they are placed is conducive to that exercise. A person held hostage has the capacity for autonomy, should they escape or be released. A disenfranchised person in an apartheid state has the capacity for autonomy, should the governing regime change.
Contrast capacity autonomy with the successful exercise of autonomy, which Feinberg calls “autonomy as condition.”Footnote 5 A person with the capacity to govern themself may be constrained by circumstances, hindered by other people, or fail to use their capacity. Autonomy in this sense involves several things. It includes subjecting one’s values to scrutiny, that is, exercising the capacity to self-scrutinize. It also involves engaging in some degree of self-determination, which is to say one must act on their values, principles, beliefs, and so forth to steer their course. Feinberg suggests that it also involves a degree of self-legislation by adopting moral principles and holding oneself to those principles. This way of conceptualizing autonomy owes a great deal to Kant. The third (“autonomy”) formulation of the Categorical Imperative is “the idea of the will of every rational being as a will giving universal law” and requires a person act according to a maxim of one’s will and such that their will “could at the same time have as its object itself as giving universal law.”Footnote 6 Maxims are compelling, and one acts autonomously in following universal maxims, in that one both sets and follows them.
Successful exercise of autonomy does not require that one make up this self-legislation whole cloth; one will invariably adopt principles that are based on one’s community. The capacity to self-govern, the values an agent develops, and the ways in which they incorporate those values into their life are socially situated.Footnote 7 Moreover, developing one’s sense of what is important depends on social conditions that nurture the ability to do so.Footnote 8 Social structures may delimit the conceptions of value that are available for a person to draw upon in developing their own principles and their own sense of value. Persons’ abilities to incorporate their values into their important decisions will depend on the prevailing “social forms” and the opportunities that exist in the broader social context.Footnote 9 Nonetheless, holding oneself to their principles autonomously requires that one adhere to them because they are principles, not merely because others happen to adhere to those principles. This in turn requires authenticity, self-control, and taking moral responsibility for one’s actions.
Such conditions form the basis of a third conception: autonomy as ideal. The successful-exercise conception outlines important component parts of autonomy. However, people are not isolated, wholly self-legislating individuals. Rather, they are parts of families, communities, and other social groups, and exercising autonomy must be compatible with being parts of social groups. Moreover, the elements of successfully exercising autonomy (self-determination, authenticity, self-control, integrity) can be used to bad ends. Autonomy is not the only relevant value, and it is a mistake to elevate it above the social and historical context of human life. Abstracting autonomy away from humans’ social nature altogether, Feinberg explains, ignores the fact that no one selects “his country, his language, his social community and traditions. No individual invents afresh his tools, his technology, his public institutions and procedures [sic].”Footnote 10 Moreover, all of those things (country, language, tradition, etc.) are key parts of being human, and we all become self-aware within those contexts and “as part of ongoing social processes.”Footnote 11 Hence, a person’s successful exercise of their capacity to self-govern according to their individual preferences may not be ideal, in part because it risks ignoring one’s social circumstances and it may conflict with one’s responsibilities to other community members.
Lastly, we may understand autonomy as a right. Individuals with the capacity for autonomy have certain valid claims and others have correlative obligations. One is that individuals have a claim to be recognized as having the capacity to govern themselves. Related to recognition is that individuals have decisional prerogatives; where others interfere with a person’s decisions, that interference is a limitation on autonomy (though it may well be justifiable). The extent of those claims is a further question we return to throughout the book. What matters here, though, is that the notion of autonomy as a right is distinct from the notions of autonomy as a capacity, a condition, or an ideal.
2.3 The Key Split
These distinctions are important and useful, but we still owe our own account. The view we advance in this book incorporates two different conceptions of autonomy. The first focuses on the relationship between a person and their motives, intentions, values, and preferences; that is, it holds that autonomy is primarily understood in terms of how an individual’s desires, preferences, and actions relate to their history and psychology. For lack of a better term, we will call this “psychological autonomy.” A different way of understanding autonomy focuses on an individual’s social conditions, relationships to other people, status within a community and polity, and the range of options and opportunities in which one develops a sense of self and others. Following Marina Oshana, we will call this “autonomy of persons” or “personal autonomy.”Footnote 12 The boundaries between these two ways of understanding autonomy are blurry, as a person’s social circumstances are closely related to their desires, values, and preferences. Nonetheless, both facets are important in understanding the relationship between algorithmic decision systems and autonomy. We will argue that both components are morally important, and the conceptual differences between them matter less than the fact that both conceptions can underwrite similar social, moral, political, and legal claims.
To explain the two different ways of understanding autonomy, we will examine an exemplar of each.
2.3.1 Psychological Autonomy
There are different ways to characterize psychological conceptions of autonomy. Oshana emphasizes that such views are fundamentally concerned with psychological authenticity.Footnote 13 Perhaps the best, most succinct characterization of this set of views is that they are “accounts of the autonomous agent’s special relation to her own motives.”Footnote 14 Another fruitful way to understand such views is that they are procedural accounts, which is to say that autonomy does not turn on a person’s particular desires, preferences, and beliefs.Footnote 15 Rather, what matters is whether their motivations and actions stand in the right relation to their psychology.Footnote 16
An important objection to procedural views is that focusing on the individual and their psychology may not adequately account for the ways individuals are fundamentally social, embedded within relationships, embodied, and historical. Those social and relational facts are inextricable from how a person develops values, preferences, and desires. A view of autonomy whereby a person is autonomous just in case their actions comport with their higher-order preferencesFootnote 17 will miss the fact that a person’s highest-order preferences may be formed in oppressive (or otherwise severely delimited) circumstances and hence are themselves suspect.Footnote 18 We address this concern in our discussion of personal autonomy in Section 2.3.2.
Nonetheless, autonomy must have at least some procedural, psychological component, and such views need not be so narrowly constructed that they recognize only lone individuals, independent of their social relations and historical selves. Consider John Christman’s account, which places individuals’ social and historical contexts at the center of autonomy.
Christman argues that individual autonomy has two key requirements: competence and authenticity.Footnote 19 Each is necessary and both are jointly sufficient for a person to be autonomous. The competence conditions Christman describes are similar to baseline autonomy requirements in other accounts. First, a person must have some set of fundamental values and commitments and the ability to “effectively form intentions” to act on the basis of those values and commitments.Footnote 20 Second, they must have the capacity to critically reflect on their basic values and commitments, their motivations, and other facets of their decision-making abilities. In other words, autonomy requires that a person have the capacities to form, develop, and critically reflect on their values and to intentionally and effectively act in accord with those values. That’s the easy part.
More controversial, and more difficult to reckon, is the authenticity requirement. This is a conditional requirement that a person would not be alienated from their basic values and commitments were they to “engage in sustained critical reflection” on them.Footnote 21 By “alienated,” Christman just means that the person would believe that their values and commitments were incompatible with their sense of themself and their practical identity over time. As Christman puts it, those values and commitments could “not be sustained as part of an acceptable autobiographical narrative organized by her diachronic [i.e., existing over time] practical identity.”Footnote 22 For the critical reflection to be adequate to ensure autonomy, it must be sustained over time, occur in a range of conditions, consider processes that affected how the person came to form their values and commitments, and not be distorted by other factors.Footnote 23
There are three key features of Christman’s account that will help as we develop our own view. The first is that the view is proceduralist; it is based on the mechanisms and processes by which persons come to have values and commitments and how they incorporate those into beliefs and actions. Procedural accounts do not depend on the content of persons’ values. An advantage to procedural views is that they do not presume that any particular values, commitments, and beliefs are inconsistent with autonomy. Procedural views contrast with perfectionist views, which build into their conceptions of autonomy at least some requirements for the content of persons’ values and commitments.
Second, Christman develops his account with an eye to addressing important lines of criticism of psychological autonomy views. These criticisms are (1) that selves are decentered and historical (i.e., there is no isolated, asocial self that is cut off from the circumstances that form one’s sense of self) and (2) that values and commitments form in social contexts and in relation to other people and communities.Footnote 24 Third, Christman’s view is internalist, which is to say that the criteria for autonomy are indexed only to a person’s own psychology.
2.3.2 Personal Autonomy
Christman’s understanding, like those of Frankfurt, Dworkin, Fischer and Ravizza, and others, is that the relation between a person and their intentions (in Christman’s case, whether the competence conditions and the authenticity conditions obtain) forms sufficient conditions for a person to be autonomous. Such accounts have received substantial criticism in recent years because of their focus on individuals rather than their social connections, communities, structures, physical embodiment, emotion, and so forth. We return to some of these critiques in Section 2.6.
To capture this separate family of conceptions of autonomy, consider Marina Oshana’s thoroughly social and relational account. Oshana’s starting point is that views like Christman’s, which center on an agent’s relation to his or her values and preferences, are at root about persons’ psychologies. However, Oshana argues, people are not reducible to their psychological states, and an adequate account of personal autonomy must involve more than a person’s psychological history, competency, and authenticity.Footnote 25 Autonomy should instead be understood primarily as a characteristic of persons, and personal autonomy is inherently a social phenomenon: “Autonomy is not a phenomenon merely enhanced or lessened by [social relations]. Social relations do not just causally facilitate or impair the exercise of autonomy. Rather, appropriate social relations form an inherent part of what it means to be self-directed.”Footnote 26
Oshana’s argument draws on several cases in which agents fully accept and internalize values that subordinate their own interests: a person raised in oppressive circumstances who embraces the subservience their community demands, a person who voluntarily becomes a part of a total institution in which they commit to serving the institution and its hierarchy, and so on. Among Oshana’s examples is a woman who chooses and values being subservient in a marriage and being the “angel of the house.” She has no say in important facets of family financial and life decisions, and she develops none of the professional and educational skill and social capital that would allow her to change her situation. In Oshana’s conception, the woman’s reasons for her choice are consistent with her values, she is reflective about those values, she finds the life wholly gratifying, and her values are not based on social conditions that established or reinforced a belief in her inferiority. In other words, Oshana’s conception is of a person who meets both a competence condition and authenticity condition. Nonetheless, Oshana argues that the “angel of the house” lacks autonomy precisely because of her social status and her personal relationships, regardless of whether she is the architect of both. Similarly, Oshana argues, a person who surrenders to the strictures of a monastic religious order such that the order controls all facets of their life lacks autonomy regardless of whether doing so was a choice that comports with their deepest values.
The point of Oshana’s examples is that there are plausibly individuals who do not have the power to manage important aspects of their lives because of restrictive or oppressive social circumstances, and yet their situations may be consistent with their authentic values. Nonetheless, persons’ de facto ability to manage important facets of their lives is “tantamount to governance over their selves.”Footnote 27 And because self-governance just is autonomy, the lack of de facto power is incompatible with autonomy.
To sharpen her argument, Oshana contrasts the angel of the house case with the “would-be surrendered woman,” who has a great deal of financial, social, and educational independence, is professionally very successful, and can exercise a great deal of global and local control over the course of her life.Footnote 28 However, her values and self-conception are to live like the angel of the house, deferential to a controlling partner. In other words, her actions and motivations in navigating life are inauthentic (at least in Christman’s sense). Oshana maintains that she is nonetheless autonomous precisely because her social and relational circumstances allow her to govern herself, even if she has not been able to do so in a way that satisfies her deeper value commitments.
There are several important consequences of Oshana’s conception. One is that it de-emphasizes autonomy with respect to preferences and values and instead emphasizes autonomy of persons. On Oshana’s view a person can be autonomous with respect to their desires, preferences, and values, but still not be autonomous in a morally important sense.
Another feature is that Oshana’s view is weakly perfectionist. Her view is that autonomy requires that a person recognize themself as the person with primary authority over their life; that is a substantive, non-proceduralist value and her view is hence perfectionist. It is only weakly perfectionist because recognizing oneself as having authority over their life is a relatively nondemanding requirement. Many different conceptions of value will be compatible with it.Footnote 29
Third, which is related to the first consequence and is perhaps most controversial, is that authenticity (in Christman’s sense, in Frankfurt’s sense, or anyone else’s) is neither necessary nor sufficient for a person to be autonomous. On Oshana’s view a person can be autonomous even if they are, upon reflection, alienated from their desires, motivations, and aspects of their character. However, if they have latitude to change that desire and to act on an alienating desire, Oshana’s view is that they are still autonomous.
Fourth is that social conditions are key for autonomy in multiple ways. They may be causally important. Christman agrees on this point – indeed, it is hard to see how it could be otherwise. Social conditions are also at least partially constitutive of autonomy. Regardless of the connection between a person’s values and preferences and one’s psychology, one cannot be autonomous on Oshana’s view if others fail to foster, support, and respect their self-governance.
The fundamental difference between Christman’s and Oshana’s accounts concerns whether autonomy properly applies to one’s psychology (including its contents over time and the social and historic processes affecting it) or to one’s person (including one’s de facto power to determine their affairs). This difference entails that the conditions of autonomy are different on the two accounts. Whereas Christman’s view demands only competence and authenticity, which are procedural requirements, Oshana’s view also includes additional requirements that are associated with appropriately conducive social conditions and choice architectures.
The conditions for personal autonomy on Oshana’s account do have some points of contact with Christman’s. In particular, she argues that personal autonomy requires a kind of procedural independence, which in turn includes a number of competence conditions. Foremost among these is epistemic competence. A person must be self-reflective and self-aware, and they must conceive of themself as a person “who can affect the world in light of a perspective and plan for life that is of her making.”Footnote 30 Related is that a person must be rational. Again, this is not the caricature of “rational” in the sense of being coldly calculating. Rather, one must be attuned to their environment and understand the world around them. They must be able to develop and be disposed to follow through with plans based on their own ends and their sense of value. Furthermore, they must be able to distinguish choices, actions, and relationships that are conducive and not conducive to self-governance.
Another condition that is compatible with Christman’s view is that agents must have self-respect. Respect involves recognizing the inherent worth of a person and treating them accordingly. To respect another person, one cannot understand that person’s value as deriving solely from their usefulness to oneself. That, in turn, demands recognizing them as autonomous and not “treating [them] in a manner that makes light of [their] autonomy.”Footnote 31 Likewise, respecting oneself demands that one understand one’s value (and the value of one’s commitments) as inherently valuable.Footnote 32 Christman’s view would account for self-respect under non-alienation.Footnote 33
What is most distinctive of Oshana’s account is that it posits conditions that go beyond an agent’s relationship to their own intentions and preferences. One of these involves an agent’s control. The view that autonomy requires an agent have control over their actions is familiar in accounts of psychological autonomy. For example, Fischer and Ravizza argue that autonomy demands that agents exercise a kind of “guidance control,” such that the source of actions is the agent themself.Footnote 34 However, guidance control can be understood as local guidance: Is this action something over which the agent exercises guidance? Oshana’s view is that personal autonomy requires a more global control over his or her ability to determine how they live. Can one, in other words, effectively act to advance their interests and satisfy their commitments, or do their social circumstances allow others to severely limit their ability to exercise control (if they choose to do so). Oshana writes:
Autonomy necessitates a fairly robust variety of control of a sort that must be effective within a person’s social situation … . We cannot claim a person is autonomous if she is party to social relations or institutions that would enfeeble her ability to determine how she will live if it were the will of others that they do so.Footnote 35
It is neither possible nor necessary for autonomy that an agent be able to exercise control over all aspects of their life. Just how much control autonomy demands, though, is a vexing question. Here Oshana’s conception follows closely Joseph Raz’s view that mere choice and control over that choice is insufficient. Rather, autonomy requires an agent have access to a variety of relevant, attractive options. A person with the choice of whether to eat bland food now or eat it later does not have autonomy over their diet, and a person having to constantly make decisions that affect their very survival may have a variety of options, but they are not attractive, and the person is therefore not autonomous with respect to their life’s course.Footnote 36
The crux of Oshana’s account, and another reason it is distinctive, is her argument that autonomy demands substantive independence. For a person to be substantively independent they must have sufficient social and relational support, and they must not have such substantial social and relational impediments that prevent them from enjoying de facto ability to determine their life course.
Substantive independence is itself multifaceted. First it involves social conditions that afford a person some baseline level of social and psychological security. A person with very little such security is vulnerable to arbitrary actions by others and hence will lack the de facto power necessary for global autonomy. Second, substantive independence requires that a person be able to have values and pursue interests that are different from those of people with relatively greater power and influence, and to do so without risk of reprisal. The idea here is that if others use one’s values and interests as a reason to exact a toll on the person, then they are dependent on others’ forbearance of their values and interests. But one cannot be self-governing when they depend on that forbearance.
Substantive independence also requires a degree of financial self-sufficiency. That simply means that one is not subject to the control of others through financial means. And if, for example, one is dependent on a state or employer for financial support, the terms of that support or employment cannot be contingent on inappropriate conditions or subject to arbitrary termination.Footnote 37 Further, substantive independence requires that a person not be subject to misinformation that curtails their ability to exercise their agency over facets of their life.
Oshana summarizes her understanding of substantive independence by drawing on Philip Pettit’s work on republican freedom. Specifically, she argues (following Pettit) that understanding freedom as merely freedom of choice misses the mark. The idea, which we address at length in Chapter 5, is that many accounts of freedom (or liberty) focus on negative liberty or whether a person is subject to external constraints, imposed by others, which prevent them from engaging in the activities that they wish to pursue. Other accounts address positive liberty, which is to say persons’ de facto ability to engage in the activities they wish. A person might be free of others’ constraints to, for example, produce a movie yet might not have the financial or social wherewithal to actually do so. In that case they would have negative liberty but lack positive liberty. Both of those facets of liberty focus on freedom of choice.
Oshana thinks these conceptions leave out the fundamental importance of freedom of the chooser. A person may be fortunate enough that others do not interfere with their actions, and they may have resources to act more or less according to their values. However, if other people have social power to interfere with them and constrain their ability to function, or if they have to negotiate obstacles and order their life to make such interference less likely, their freedom is nonetheless constrained. Specifically, their social freedom is diminished.
Consider, for example, a series of lawsuits concerning “stop-and-frisk” policies in New York City.Footnote 38 From 2004 through 2012, New York City police conducted over 4.4 million “Terry” stops of people in the streets. Terry stops are short, informational police stops that do not rise to the level of a full search. Because they are limited in scope, the legal requirement for conducting a Terry stop is lower than the “probable cause” standard required under the Fourth Amendment of the U.S. Constitution. Specifically, police may stop people under the Terry standard if they have “specific and articulable facts which, taken together with rational inferences from those facts, reasonably warrant that intrusion.”Footnote 39 Hence, police may stop persons if they have reasonable suspicion that “criminal activity may be afoot” and may conduct a brief frisk for weapons so long as it is based on a reasonable suspicion that the person is armed and dangerous. Fifty-two percent of the Terry stops conducted during this period included a frisk for weapons, though 98.5 percent of the frisks turned up no weapon. A very small percentage of the stops resulted in either arrest (6 percent) or summons (6 percent). The overwhelming majority of the persons stopped were Black (52 percent) or Hispanic (31 percent); 10 percent of the people stopped were White (the population of New York City at the time was 23 percent Black, 29 percent Hispanic, and 33 percent White). Police used some kind of physical force in 23 percent of stops of Black people, 24 percent of stops of Hispanic people, and 17 percent of stops of White people. Finally, despite the fact that the Terry standard is a low hurdle, police conducting stop-and-frisks often did not state a specific crime they suspected to be afoot (in 2004, police failed to state a specific crime for only 1 percent of stops, but by 2009 police failed to state a specific crime for 36 percent of stops).Footnote 40 Moreover, data collected from the forms that police filled out after stops indicated that many more stops were “apparently unjustified,” as officers often simply checked boxes to justify stops post hoc (e.g., checking boxes indicating persons stopped had made “furtive movements” or made “furtive movements” along with having a “suspicious bulge” in their clothing).Footnote 41 This pattern led to a series of lawsuits, and the U.S. District Court determined that the city had violated persons’ Fourth and Fourteenth Amendment rights by acting with deliberate indifference to unconstitutional police stops.Footnote 42
What is important for our purposes is how they illustrate Oshana’s conception of substantive independence and its relation to republican freedom. The idea is this: Considered in isolation, any particular stop-and-frisk event constitutes a relatively small imposition of freedom. It does not take a long time and hence in most cases will not prevent one from going about their business shortly. And, hence, the mere fact of being subject to a stop-and-frisk does not undermine one’s substantive independence. However, stopping the analysis there omits the overweening and arbitrary nature of the New York City stop-and-frisk program in practice at scale. Because the stops were so frequent, often failed to meet even the low Terry hurdle, and were so divorced from actual criminal conduct and from actual weapons possession, they infected the daily lives of residents (and in particular Black and Hispanic residents) with persistent exposure to arbitrary power. The ability to go about one’s life free from being stopped was, in effect, at the whim of the police. It is in that way that people’s freedom was impinged, and it is in that way that their substantive independence was reduced. And, hence, their personal autonomy was diminished.
2.4 Reconciling Psychological and Personal Autonomy
In the previous sections we described two categories of autonomy views. The first, which focuses on an individual’s special relationship to their values, intentions, and motivations, is exemplified by John Christman’s account. Christman’s view is procedural in that it does not index autonomy to the content of an individual’s values and motivations. The view is relational in that it understands the importance of a person’s history and social circumstances in supporting values and preferences from which a person would not be alienated. As noted in Section 2.3.1, there are numerous competing views of autonomy in this category, articulating different kinds of procedural conditions necessary for persons to be autonomous. The second category, exemplified by Marina Oshana’s view, understands psychological autonomy as an insufficient account of what matters morally, which is personal autonomy. Oshana’s view is that while elements of psychological autonomy are important, they are neither necessary nor sufficient for personal autonomy. Personal autonomy demands more, including substantive independence.
This split seems wide at first blush, one focusing on a person’s psychology and its history, the other focusing instead on facts about a person’s place in the social, material world (and in a weakly perfectionist way at that). However, for several reasons, we need not draw a conclusion about which family of views is better supported. To begin, our project is different from the projects of Christman, Oshana, Buss, Frankfurt, Dworkin, and others. We are not aiming at a new, comprehensive account of autonomy and its importance. Rather, we are beginning with the premise that autonomy is important, adopting a minimal, ecumenical approach to the existing accounts, and developing an understanding of the moral salience of automated decision systems from the account built from convergence among the other accounts. Hence, differences in the families of views matter here only to the extent that they affect an analysis of autonomy-based moral claims.
And when we look at that set of issues, the categories of accounts of autonomy (including Christman’s and Oshana’s) converge. Both types of account can agree about many of the conditions that are important for individuals’ autonomy. Consider Oshana’s example of the woman raised in oppressive circumstances. She argues that the woman in that case lacks autonomy because the social conditions necessary for her to act with substantive independence are lacking, regardless of whether her (current, local) values and beliefs are procedurally independent. Christman agrees that the woman lacks autonomy, but he argues that this is because, in all likelihood, she would experience alienation if she were to reflect on her values and preferences in light of the oppressive processes affecting their formation.Footnote 43 What’s important for our account, though, is that Oshana and Christman agree that she lacks autonomy, it is the result of a moral wrong, and the lack of autonomy is a key part of the explanation for why her treatment is morally wrong.
Oshana’s would-be surrendered woman case is a bit more difficult. Oshana, recall, argues that this person is autonomous because she is procedurally and substantively independent. She has the social wherewithal to act differently than she does, regardless of whether her current actions are ones from which she would be alienated (if she were to critically reflect upon them). Christman’s view is that the would-be surrendered woman lacks autonomy precisely because her actions do not meet the authenticity conditions. However, as Oshana has constructed the case, Christman would not be able to point to a moral wrong undermining her autonomy. Instead, Christman could argue that a more thoroughgoing examination of a person’s history and the social structures in which a person’s values are formed are likely to uncover limitations rooted in competence conditions or authenticity conditions. However, if there are no such limitations, Christman would allow that she lacks autonomy.Footnote 44
What is important, though, is that Christman and Oshana’s positions about what is morally relevant are compatible. What matters to each is that the would-be surrendered woman’s social and relational circumstances are not particularly constrained and that she does experience alienation. The difference is whether that alienation is incompatible with the best-supported conception of autonomy, not whether there is a moral infirmity, and not what the source of such an infirmity might be.
Moreover, even in the relatively narrow range of cases in which Oshana and Christman would disagree about whether a person is autonomous, they can agree that there are autonomy-based wrongs. Consider instead a case in which a person is raised in oppressive circumstances, internalizes facets of that oppression, and in which they come to have values, motivations, and preferences that reflect that oppression. And suppose that even with sufficient opportunity to critically reflect on those values, motivations, and preferences and their genealogy, they would experience no alienation. Christman would have to conclude that the person is autonomous (after all, they meet his competence and authenticity conditions). Oshana would conclude that they lack autonomy only if they lack sufficient social opportunity to change their views and act accordingly. She would disagree that the person is autonomous only if their current social circumstances reinforce the values, motivations, and preferences formed under limited conditions. But Christman and Oshana could agree that there is an autonomy-based wrong in their limited circumstances. Christman could argue that those circumstances are unjust because they tend to be wrongs that lead to alienation. Oshana’s view is that the wrong is based on the fact of social limitation per se. In other words, the precise explanation of the wrong will vary. However, the views overlap in the cases of moral wrongs and the type of wrongs, while differing in the explanation of them.
There is also the possibility of an account of autonomy that bridges the psychological authenticity, procedural accounts with social-relational accounts. Recently, Zi Lin has argued that instead of self-rule, autonomy should be understood as independence from other-rule.Footnote 45 Her idea is that psychological authenticity requires something other than non-alienation or confluence with higher-order values. Those accounts index authenticity solely to a person’s psychology (and in Christman’s view, their diachronic psychology). Such views have some well-known conceptual problems, including the problem of regress and problem of problematic influencers. Both of these, Lin argues, can be resolved by building autonomy around independence from other-rule. Hence, on Lin’s view, psychological autonomy has a relational component that is constitutive of autonomy, rather than merely being causally relevant.
2.5 An Ecumenical View
There are plenty of disputes about the nature of autonomy: whether it describes a person’s relation to his or her values, motivations, and intentions or describes a person’s social and relational circumstances; whether competence and authenticity conditions are necessary or sufficient for a person to be autonomous; whether social conditions are constitutive of autonomy or merely causally relevant to whether one is autonomous. But, as we explain in the previous section, the different views can agree that certain things matter because of autonomy. That is the basis of our ecumenical view of autonomy in which we distill a number of key points about autonomy and its value from different conceptions.
To begin, any plausible view of autonomy will recognize the importance of procedural independence (though they may disagree about whether procedural independence is necessary for autonomy, sufficient for autonomy, or merely important for autonomy). Procedural independence requires several things. One is epistemic competence, which is to say one must be to some degree self-reflective, self-aware, and understand themself as able to actualize a life plan. Respecting people as autonomous demands fostering epistemic competence.
Notice, though, that affording people the circumstances in which they can exercise their epistemic competence is a facet of personal autonomy. Hence, severely or deliberately constrained information environments are ways in which an epistemically competent person will fail to have personal autonomy because they lack substantive independence.
Another component of psychological autonomy is rationality, or the ability to understand the world around oneself, to make close connections between facts and inferences, and to make decisions that line up with well-enough ordered assessments of facts and reasons. As noted, this is a low bar. Rationality is also related to personal autonomy. Constraints on a person’s ability to reason clearly can come from poor informational environments or from psychological stressors that thwart their ability to think clearly, hence inducing poor decisions.
Procedural independence also includes some degree of authenticity. Oshana argues that authenticity is not required, and this is a key difference between her and Christman. Our view, however, is not that authenticity is necessary or sufficient for autonomy. Rather, authenticity is important for two reasons. One is that authenticity is evidence of personal autonomy. The ability to reflect on one’s values and preferences and to recognize them as compatible with one’s sense of self and practical identity over time is an important test of the degree to which one’s values and preferences are one’s own, and (hence) that one self-governs. Likewise, lack of authenticity – that is, where a person would be alienated from their preferences and values upon reflection – is defeasible evidence that one’s personal or social autonomy is compromised. And fostering authenticity is itself morally important in that individuals are morally valuable in part because individuals are capable of determining for themselves what is of value in their lives, and they are (hence) the source of that value.Footnote 46 In other words, structures that make it likely that individuals would experience alienation from their desires and values (in Christman’s conception, where structures make it likely that a person will have inauthentic preferences and motivations) are in all likelihood antithetical to personal autonomy. Those structures may be morally justifiable overall, but that would be in spite of their relationship to autonomy.
Beyond procedural independence and its competence and authenticity components, autonomy requires substantive independence. Whether one is personally autonomous turns on their circumstances and their environment.Footnote 47 This includes social conditions such as choice among a range of attractive options (per Raz), control over meaningful facets of life, conditions of self-respect, relative financial independence, and so forth. In this we follow Oshana’s conception closely. There are, however, a few ways in which substantive independence is insufficient in ensuring autonomy overall. A person can be substantively independent (in terms of having financial wherewithal, not subject to others’ arbitrary power) but can have competence conditions undermined. Likewise, one can have local substantive independence but have been subject to conditions that make it difficult to have authentic preferences and desires (we will return to this in Chapters 4, 5, and 6). Of course, those limitations may be understood in terms of more global substantive independence.
We can summarize our view and provide a foundation for the arguments in the remainder of the book with a few primary propositions.
The first set of these concerns Feinberg’s distinctions between the various meanings of “autonomy.” First, autonomy as a capacity predominantly grafts onto psychological autonomy; it is entirely possible for a person to have a capacity for autonomy, but be prevented from exercising it (i.e., being denied personal autonomy). Second, successful exercise of autonomy in Feinberg’s sense demands substantive independence; however, the autonomy one exercises in that sense is psychological. Third, autonomy as ideal (which is to say that individual autonomy properly integrated into a larger matrix of values, including one’s responsibilities and community concerns) reflects persons’ obligations to respect others’ psychological autonomy and ensure the conditions of personal autonomy. Fourth, the idea that autonomy is a right should be understood in the sense that persons have claim-rights grounded in their autonomy.
The next propositions reflect the fact that both psychological and personal autonomy are important and (hence) that both procedural and substantive independence are valuable.
To begin, procedural independence is morally important, but limitations on a person’s procedural independence are important in large part because of conditions that impose such limitations (including limitations placed by other people, by organizations, by social structures, by natural causes, and so forth). Fostering conditions that promote procedural independence is the crucial part. It is similarly important that others respect the actions of those who lack procedural independence. They may have capacity autonomy, after all, even if it is unsuccessfully exercised.
Substantive independence is important beyond its role in fostering psychological autonomy. It does not require that one can literally do without others – business partners may need each other’s expertise and efforts, but that does not undermine their substantive independence. Rather, what matters is whether others make demands that are arbitrary, demand inauthentic or epistemically unjustifiable actions, or undermine one’s ability to act for one’s own reasons. That does not mean any kind of compromise is antithetical to respecting autonomy. What matters is whether agreements and dependence are such that one could agree to them as part of valuable social relationships and other goods. It also matters how global those contradictions are.
2.6 Objections
So far, we have considered a few different conceptions of autonomy and offered the account we will use to ground the arguments in the remainder of the book. There are, however, important critiques of autonomy, both as a concept and as a basic value. One family of criticism is that autonomy-based moral theories (and deontological theories generally) are simply mistaken about what matters. Consequentialists and virtue ethicists (among others) might argue that other values are the proper measure of moral value. As important as those criticisms are, we won’t offer a defense here. Rather, we will simply confirm that a rock-bottom assumption of this project is that autonomy is morally valuable, and it is an important enough (and rich enough) value that it can ground the arguments we offer throughout. If one disagrees with that assumption, this project probably won’t be persuasive.
Much more important in our view are criticisms levied against the very concept of autonomy and its value. Feminists, for instance, have critiqued autonomy-based theories as atomistic, divorced from social responsibilities, unmindful of the importance of relationships in identity formation, hyperrational, and disembodied. These criticisms have provided a compelling corrective to major strains of autonomy scholarship. The principal views that we have drawn on for our account are sensitive to these concerns and explicitly incorporate the critiques into their accounts. Indeed, the critiques are so clearly correct that any plausible contemporary account of autonomy will be explicitly relational. Nonetheless, it is worth canvassing a few of the critiques here.Footnote 48
One set of critiques Mackenzie and Stoljar characterize as “metaphysical.” Specifically, such critiques reject autonomy views on the grounds that autonomy attributions assume that agents are atomistic. That, in turn, can mean several things. One possibility is that individuals are causally independent or isolated from others; that is, individuals are self-creating sources of values and desires. Of course, that is false; one can discern autonomy in some sense even while recognizing that people’s understandings are caused by their social milieu.Footnote 49 A different possibility is that agents are atomistic in the sense that they are independent of social and family relationships or that persons have intrinsic properties that do not depend on how they relate to others. Again, individuals are not like this at all. First, as a matter of empirical fact humans are hyper-social. Second, people’s identities and values do not cohere with absent relationships with others. Nonetheless, autonomy remains conceptually coherent so long as we allow that there are important ways in which an individual can form their values under competence and authenticity conditions and has some claim to avoiding arbitrary interference.Footnote 50 A final way of understanding atomistic individualism is that persons are metaphysically distinct. Mackenzie and Stoljar point out that this is not so much a critique as an obvious fact.Footnote 51
A related set of critiques are based on conceptions of care. The idea is that traditional understandings of autonomy have under- or devalued women’s perspectives and traditional, gendered social roles. Hence, relationships of care have been systematically excluded from autonomy conceptions. But those roles are of fundamental human value, and ignoring their centrality is a mark against any view. Mackenzie and Stoljar point out, though, that a number of feminist scholars, such as Jennifer Nedelsky and Evelyn Fox Keller, incorporate understandings of care into conceptions of autonomy. Acting as an agent in the world – a constitutive part of autonomy – should involve interdependence, nurturing, and care. And such capacities are indeed types of competence conditions for autonomy.Footnote 52
The upshot of these critiques is that while traditional conceptions of autonomy are cramped and implausible, more capacious understandings can address some of those shortcomings. Specifically, any reasonable account of autonomy will recognize that the sources of self, meaning, and value will be intimately bound up with one’s social, community, and family relationships. We are not self-executing sources of value; rather, we are sources of value within social, relational contexts. Moreover, the competencies of autonomy will involve one’s ability to enter and foster those relations. And lastly, the procedural and substantive independence conditions (e.g., those articulated by Christman and Oshana, and upon which we draw heavily) provide an explanation for why oppressive conditions (based on gender, or race, or ethnicity, or class, or happenstance) conflict with autonomy.Footnote 53
2.7 Conclusion: Related Concepts and Moral Salience of Autonomy
In this chapter we have made some basic distinctions about autonomy and canvassed two important, representative views (each exemplifying a different family of conceptions of autonomy). We have argued that those groups of views have substantial normative overlap, and we have used that overlap to advance an ecumenical view of autonomy. That conception provides a foundation for the arguments in the rest of this book. Before turning to those arguments, though, it is worth briefly describing why autonomy matters and how it grounds other concepts that will figure into the chapters that follow.
One way that autonomy matters morally is built into our account. People are capable of determining their values, desires, and preferences, and they can use those values, desires, and preferences to guide their decisions and steer their lives. They are, in other words, a source of value. To the extent that others severely constrain individuals’ ability to form their own sense of value (e.g., by limiting their procedural or substantive independence), they stifle the degree to which individuals’ values are their own. Moreover, thwarting persons’ abilities to act on their own values by coercion, deception, or severely constrained choice architecture is a way of circumventing autonomy and an affront to persons as choosers and self-governors.
However, the moral salience of autonomy goes well beyond limitations on persons’ abilities to act on their own desires and values. Consider again the distinctions we set out at the beginning of this chapter. The ability to act on one’s own desires and values is a matter of successfully exercising one’s capacity for autonomy. That assumes that people have the capacity to autonomous. It may not be enough, though, to simply assume people have that capacity. Rather, it is plausible that there is a social responsibility to promote that capacity. Indeed, that is an important justification for education.Footnote 54
Throughout the book, we consider different kinds of moral concerns about automated systems, all of which are rooted in autonomy. In Chapter 3, we address broader social claims regarding use of algorithmic systems. The systems we discuss in that chapter – K-12 teacher evaluation systems and criminal justice risk assessment tools – do not directly limit autonomous individuals’ choices via deception, coercion, or manipulation, and they do not obviously undermine substantive independence. Rather, we argue that autonomy is a foundation for a claim that persons be subject only to social systems that they could reasonably endorse. In Chapter 4, we make the case that autonomy includes more than the ability to act on one’s values and preferences. It also has an important informational component. That informational component is vital regardless of whether a person is able to put information into practice. Autonomy demands, in our view, the ability to exercise both practical agency (the ability to function effectively in important arenas) and cognitive agency (the ability to exercise a kind of evaluative control and understand one’s place in the world, regardless of one’s ability to affect it).
Later, we will expand our discussion of the social conditions of freedom. We have already discussed how Oshana’s account incorporates facets of Philip Pettit’s understanding of republican freedom into her understanding of personal autonomy. Our task in Chapter 5 is to draw out the autonomy- and agency-based conditions of freedom. In Chapter 6, we argue that successful exercise of autonomy may in many cases demand a degree of epistemic paternalism. That is, given the competence and authenticity requirements for autonomy, certain kinds of media entities may be permitted (or even obligated) to exert a degree of editorial control over the content posted on their platforms. In Chapter 7 we explain the moral requirements of autonomous agents. That is, autonomous persons are capable of legislating and following moral principles. That capacity creates obligations to act responsibly. Obfuscating that responsibility will in many cases be a distinct kind of wrong. Finally, in Chapter 8, we will consider how autonomy plays a crucial role in underwriting political legitimacy.
Having introduced our polestar cases, situated concerns about algorithmic systems in a broader discourse, and explained our strategy in Chapter 1, and having provided our catholic conception of autonomy here, we can begin directly addressing the autonomy-grounded moral concerns in algorithmic systems. Next stop: Chapter 3.
Open access