Artificial intelligence and automation are finding their way into the healthcare sector – some systems even claim to deliver better results than human physicians. However, the increasing automation of medical decision-making is also accompanied by problems such as the maintenance of the relationship of trust between physicians and patients or the verification of decisions. This is where the patient’s right to explanation, which is enshrined in the General Data Protection Regulation (GDPR), comes into play. This chapter explains how the right is derived from the GDPR and how it should be established.

One important criticism of algorithmic systems is that they lack transparency, either because they are complex, protected by intellectual property, or deliberately obscure. There is a debate about whether the EU’s General Data Protection Regulation (GDPR) contains a “right to explanation” This chapter addresses the informational component of algorithmic systems. We argue that information access is integral for respecting autonomy, and transparency policies should be tailored to advance autonomy. We distinguish two facets of agency (i.e., capacity to act). The first is “practical agency,” or the ability to act effectively according to one’s values. The second is “cognitive agency,” which is the ability to exercise what Pamela Hieronymi calls “evaluative control”. We argue that respecting autonomy requires providing persons sufficient information to exercise evaluative control and properly interpret the world and one’s place in it. We draw this distinction out by considering algorithmic systems used in background checks, and we apply the view to key cases involving risk assessment in criminal justice decisions and K-12 teacher evaluation.

Algorithmic accountability has emerged as a package of legal ideas that, on one hand, attempt to impose administrative law mechanisms such as transparency and due process on automated decision-making systems, and on the other hand, has developed computational approaches to constraining machine learning. In particular, by ensuring the complex computational analysis of individuals through machine learning models occurs more ‘fairly’, and is more explainable. As well as describing the necessity for computational legal implementations that actively constrain how data processing occurs, the chapter argues that there are risks that these mechanisms may involve ceding to data science and its corporate stakeholders the epistemological terrain as to what types of calculations are ‘fair’ and what type of information is an ‘explanation’.