3.1 Introduction
On 10 September 2020, Pace Gallery in London held an exhibition by the artist Trevor Paglen examining the visual products from artificial intelligence and digital data systems.Footnote 1 Titled ‘Bloom’, the exhibition featured an over-sized sculpture of a human head. Bald, white, and possibly male, this eerily symmetrical ‘standard head’ had been modelled on measurements from canonical experiments in facial recognition history by Woody Wilson Bledsoe, Charles Bisson, and Helen Chan Wolf occuring at Panoramic Research Laboratory in 1964.Footnote 2
Centring this ‘standard head’ in the space, Paglen surrounded it with photographic prints of leaves and flowers re-composed from RAW camera files by computer vision algorithms. These machine visualisations of nature encircled the ‘standard head’ illustrating how digital imaging using autonomous toolsets can achieve significantly different graphical outcomes. The exhibit foregrounded face recognition technology yet provoked viewers to consider the cross-practice connections between computing and data classification, humans and nature, and how image-making is becoming technically autonomous.Footnote 3 Another take-away is how these systems require multi-faceted elements to work and the ‘mushrooming and blossoming from all kinds of datasets’.Footnote 4
As a form of networked visual surveillance, facial recognition technology (FRT) works from the extent to which it operates in larger information infrastructures, FRT ‘is not a single technology but an umbrella term for a set of technologies’.Footnote 5 These digitally networked systems allow imaging data to transform from one state to another, and transfer from one site to another. Recent improvements in FRT, as a remote identification system, has reached a point to be technically possible to capture biometric images and data from subjects in public, private, and personal spaces, or interactions online, without their consent or awareness, or adequate regulatory oversight. This includes a distribution of sensitive and personal user information between state and private-sector organisations, while contributing to training machine learning tools using honeypots of data, and enabling ‘ever more sophisticated and effective forms of social control’.Footnote 6
Unlike the suggestion of Paglen’s exhibition, the origins of FRT cannot be reduced to the experiments in 1964. We need to widen the lens as the technical operations Stakeholders inside these systems are globally distributed and as Chair of Electronic Frontiers Australia’s Policy Committee, Angus Murray iterated require ‘bargains of trust’.Footnote 7 For example, Domestic and federal police agencies use systems that rely on huge amounts of data aggregation in private cloud servers and proprietary hardware that store and transmit data from online platforms, smart devices, foreign owned closed-circuit television (CCTV) companies and creators of wearable body cameras.Footnote 8 In Australia, retail outlets such as Bunnings use FRT and identity data to extract information from social media, where most people have images of themselves uploaded. They perform analysis based on the specific visits and transactions for certain shoppers.Footnote 9 Similarly images captured in public spaces, of crowds or of protesters, can be matched to social media posts or online forums managed by global technology firms, such as Facebook and Google, or transnational intelligence agencies such as the NSA and GCHQ. In the United Kingdom, Daragh Murray witnessed FRT software draw rectangles around the faces of people in public streets from a live CCTV feed. The system then extracted key features and compared these with stored features of criminal suspects in a watch list.Footnote 10 Matching an image to a watchlist is not the only function to consider here, but a need to query the distribution and ownership of data in the system being collectively assembled by the Tokyo-based technology giant NEC, in the example provided above.Footnote 11 Other examples of this diffuse and operational data flow include how China’s Zhejiang Dahua Technology Co. Ltd sold thermal imaging cameras, armed with facial recognition software, to scan workers entering Amazon factories during COVID-19, that is despite them being black-trade listed in the United States.Footnote 12
FRT and its computer procedures are therefore systems and ‘technologies in the making’, not artefacts with singularly defined origins and easy to regulate outcomes.Footnote 13 While an abundance of research looks at the use of FRT in border security and biometric surveillance,Footnote 14 retail shopping or school aged education,Footnote 15 and the gendering and racial divide between datasets with calls to ban these systems,Footnote 16 other elements also require scholarly, legislative, and regulatory attention.
This chapter considers how large-scale technical systems such as FRT have bloomed yet build on the echnical roots of multiple systems and the provenance of data sources that remain under considered. Tracing the genealogical origins and provenance of such datasets and statistical toolsets plays an important role in framing current uses for regulatory challenges. In this regard, this chapter presents empirical findings from research on early Indian statistical measures, the convergence of Chinese and Western technology companies, and the increase in computer vision experiments including those conducted on animals for bio security identification purposes. This chapter argues these diverse material innovations and information domains not only act as testbeds for FRT systems, but encompass some of the globalised products contained in FRT infrastructure.Footnote 17
3.2 FRT Does Not Have a Singular Origin, They Are ‘Systems in Motion’
Bledsoe’s ‘standard head’ algorithm didn’t remain at the University of Texas nor in the domain of artificial intelligence history. Owing to funding by the RAND Corporation, the algorithm worked its way into informational models for law enforcement purposes. In the development of the New York State Intelligence and Identification System (NYSIIS), Bledsoe was recruited to develop his algorithm to computationally solve ‘the mug-file problem’.Footnote 18 By contributing to the world’s first computerised criminal-justice information-sharing system,Footnote 19 as Stephanie Dick posits, Bledsoe’s algorithm and its ideas travelled with his over-simplifications and data assumptions in tow.Footnote 20 This influenced not only law enforcement databases and decisions on criminal targets in the United States, but also FRT developments that followed.Footnote 21 In its final state the algorithm was not used to automatically detect faces – as FRT does now – but contributed to a standardisation of ‘mug shot’ photos for computer filing systems. Bledsoe, who was later the president of the Association for the Advancement of Artificial Intelligence, used 2,000 images of police mug shots as his ‘database’ for making comparisons with a new set of photographs to detect any similarity. This American National Standards Institute Database, whose archives of mug shots featured convicted criminals (and those just accused), was the predominant source of visual information for Bledsoe’s facial-recognition technology (a role now filled by social media).Footnote 22 To this end, Bledsoe and his Panoramic Research collaborators manually drew over human facial features with a device that resembled an iPad called a GRAFACON or RAND tablet. By using a stylus, images were rotated and re-drawn onto the tablet and recorded as coordinates on a grid. This produced a relatively high-resolution computer readable image. A list of distances were calculated and recorded as a person’s identification code for locations such as the mouth, nose, or eyes.Footnote 23 Facial recognition (at this time) was a mathematical code of distances between features, drastically reducing individual and social nuances between them, and largely informed by Bayesian decision theory to use ‘22 measurements to make an educated guess about the whole’.Footnote 24
In essence, Bledsoe had computerised the mug shot into a ‘fully automated Bertillon system for the face’.Footnote 25 This system, invented by French criminologists Cesare Lombroso and Alphonse Bertillon in 1879, gained wide acceptance as a reliable and scientific method for criminal investigation, despite problematic eighteenth-century anthropometric experiments. The mug shot was invented to recognise criminal suspects who were repeatedly arrested: portraits were drawn and statistically labelled on common morphological characteristics.Footnote 26 The resulted ‘mug shots’ were standardised and collected by police departments and accepted as evidence in courts. Photo IDs modelled on the mug shot not only became an official format for policing, but have become standard issue in nation-state passports presented at airports and for driver’s licence photographs. The first ever US photo driver’s licence, issued in 1958, was created by French security company IDEMIA – a world leader in biometric security. Founded in 1922 as the defence contractor SAGEM, it then became SAGEM-Morpho in the 1980s, and parts of IDEMIA go back even further, and they have effectively led to every shift in the photo identity issuance and credentialling in the US since.Footnote 27
Bledsoe’s 1960s laboratory experiments thus relied on two separate building blocks invented in France. Hampered by the technology of his era, Bledsoe’s ideas for FRT were not truly operationalised until the 1990s – driven by a technological wave of mobile phone and personal computer sales, online networked wireless video systems, and digital cameras.Footnote 28 Yet the experimental use of FRT is still being conducted in a way largely never done before.Footnote 29 Clare Garvie contends that forms of automated imaging for policing actions remain unregulated and represent a ‘forensic science without rules’:
[T]here are no rules when it comes to what images police can submit to facial recognition [databases] and algorithms to help generate investigative leads. As a consequence, agencies across the country can, and do, submit all manner of probe photos – low-quality surveillance camera stills, social media photos with filtering, and scanned photo album pictures. Records from police departments show they may also include computer-generated 3D facial features, or composite and artistic sketches.Footnote 30
In the next section, I explore how the automation of FRT relies not only on a diverse manufacturing of ‘images’ – products of reduction, appropriation, transformation, or digital manipulation – and situated instances of exploitation conducted in South America, the United States, France, Russia, Japan, and China to name a few different jurisdictions, but also how modern FRT resurrects a century old vision of ‘statistical surveillance’.Footnote 31 To do so, I consider how a 100 year old mathematical experiment in British India has aided the probabilistic functionality of autonomous FRT systems.
3.3 The ‘Mind Boggling Systems’ Where Everyone Only Ever Has One ID
In 1991 Turk and Pentland produced the first real-time automated face recognition.Footnote 32 Famously, this was deployed at the crowded USA Super Bowl in 2001. This experimental trial was called ‘Facefinder’. The system captured surveillance images of the crowd and compared them with a database of digital mug shots held by Tampa police, the Florida Department of Law Enforcement and the FBI.Footnote 33 The experiment not only demonstrated the potential for remote surveillance of crowds, but also led to the National Institute of Standards creating a Face Recognition Vendor Test (FRVT) to evaluate this emerging FRT market.
A quick look at the ongoing FRVT of 1: N facial algorithms reveals a globalised picture: ‘The report lists accuracy results alongside developer names as a useful comparison of facial recognition algorithms and assessment of absolute capability. The developer totals constitute a substantial majority of the face recognition industry.’Footnote 34 This includes performance figures for 203 prototype algorithms from the research laboratories of over fifty commercial developers and one university. Similar to Beldsoe’s 1960s experiments for NYSIIS, this evaluative test scenario also uses frontal mug shots and profile view mug shots alongside desktop webcam photos, visa application photos, immigration lane photos, and traveller kiosk photos.
A brief survey of this report illustrates the scale and scope of a global FRT market. To name a few vendors, the developers and their places of origin include NEC (Tokyo); Microsoft (United States); Veritas (Spain); Herta Security (Spain); AnyVision (Israel); IDEMIA (France), utilised in Kenya and in Turkey; Daon (Ireland); Dahua (China); Moonwalk (China); Sensetime (China); Hyperverge (California); Cognitec (Germany); QNAP (Taiwan); Tevian (Russia); VisionLabs (Russia/Netherlands); Clearview AI (United States); DeepGlint (China) and finally Neurotechnology (Lithuania), which is a provider of deep-learning-based solutions for high-precision biometric identification and object recognition technology.
Importantly, the Lithuania based Neurotechnology recently partnered with Tata Consultancy Services as one of three biometric service providers for the largest biometric ID system in the world, Aadhaar.Footnote 35 Co-ordinated by The Unique Identification Authority of India, the system registers people and compares their facial biometric with the existing records of 1.3 billion people to verify applicants have not registered under a different name. Aadhaar is ‘a mind-boggling system’, says Anil Jain, a computer scientist who consulted on the scheme, ‘and the beauty is that it ensures one person has only one ID’.Footnote 36
India has a rich history of producing material and statistical innovations to identify individuals based on their physical characteristics.Footnote 37 In 2020, Google posted an online tribute to Professor Prasanta Chandra Mahalanobis (1893–1972) as part of its ‘Arts and Culture’ series.Footnote 38 Mahalanobis is famous for creating new statistical and biometric functions as key technologies he advocated to the world through his Indian Statistical Institute.Footnote 39 The global celebration of his work was recognised in part after his creation of a similarity distance metric in 1936. This was produced from his specific interest in racial classification.Footnote 40 He developed a biometric function to analyse and identify people based on physical and racial similarity. To do so he compared data collected from the Chittagong Hill Tract area (modern Bangladesh) with international race data sets collected from Swedish and Chinese records.Footnote 41 He then set about learning how to create an identification of race based on statistical measurements of facial features and their similarity, which he could apply in India. The aim was to help identify exotic and ethnic caste groups to be classified in the British colonial administration.Footnote 42
Significantly, he also innovated by using facial photographs of living subjects to compare the accuracy of his biometric measurements, compared with analysing skulls in the era’s practice of phrenology.Footnote 43 By testing his distance function with the invention of an experimental imaging device in 1937, Mahalanobis was a central figure in pushing ‘part of a biometric nationalism in which the face provided a form of data’.Footnote 44 His metric, commonly known as a Mahalanobis Distance Function, despite being created eighty-six years ago, is consistently used in modern FRT.
Even the most sophisticated and large-scale FRT systems necessitate this basic approach of comparing images on facial features by using scores that compare a match of the similarity.Footnote 45
In technical terms, the selection of a decision metric – such as the Mahalanobis Distance Function – ‘[h]elps to measure distances between specific facial features and generate a unique representation (as a ‘facial signature’) for each human face.Footnote 46 Similar to Bledsoe’s code, this is then compared with a database of stored images in order to match a face to similar images.
In this regard, similarity measure functions operationalise the matching process as a critical decision-making module. Selection of the proper similarity measure is thus an important determination for the accuracy of the matching result. Such measures include Minkowski distances, Mahalanobis distances, Hansdorff distances, Euclidean, and cosine-based distances.Footnote 47 Yet the Mahalanobis distance is the best at structuring data for unknown targets. This is critical to criminal subject investigations for matching suspects from surveillance images in supermarkets, stadiums, or of protest crowds. The similarity measure enables high-speed cluster analysis – critical to a speed of decision-making – especially for faces with a high-number of variables and in relation to fitting an unknown person into a known database. FRT can then determine if an unknown image (taken from a web profile or a surveillance camera) matches a person in the database (compared with drivers’ licences or mug shots). This approach is also suitable for machine learning and is a prominent approach for training systems on person re-identification by ‘improving classification through exploiting structures in the data’.Footnote 48
As Adriana Dongus suggests, ‘[t]he large datasets produced by science and law enforcement at the turn of the nineteenth century continue to form the material backbone and precedent to current machine learning.’Footnote 49 By examining the critical and ubiquitous distribution and embedment of early decision classifiers, we establish the importance of selecting certain rule functions in ‘a statistical layer’ of FRT systems.
When applied to machine learning, this includes assigning weights to autonomously identify the importance in probable matches. This is used in image-labelled data sets,Footnote 50 to estimating facial position poses from video,Footnote 51 to automatically locating an unproductive worker on a factory floor,Footnote 52 or identifying ethnic minority faces in a crowd, as is occurring in China with the Uyghur (Uighur) population. While much important work on facial recognition is salient to the United States,Footnote 53 there is a need to examine how FRT is conditioned on a globalised supply chain. This includes the ‘production, schematization, maintenance, inflection, and reproduction of certain [decision] rules’ and how they replicate use of problematic standards in public surveillance.Footnote 54
Indeed, there has been a ‘tendency to gloss over the amount of effort that goes into developing and integrating new technologies and systems with older technologies’.Footnote 55 Computation moves fast – yet many lessons remain and are yet to be learned.
From legislative, ethical, and regulatory standpoints, it is worth noting that biometric systems and data (including use of statistical functions and facial images) are constructed on complex and interoperable supply chains involving third-party vendors needed to make these systems work. Yet there is potential incentives built within these globalised computing systems to exploit regulatory gaps and vulnerabilities that could be used against various human populations at a later date.Footnote 56 The final section examines how Mahalanobis’s 100 year old experiment is relevant not only to our digital identity systems today, such as the United Nations High Commission for Refugees (UNHCR) Population Registration and Identity Management Eco-SystemFootnote 57 but builds on different use-cases. These include not only nation-state surveillance, such as the identification and detection of ethnic minorities in China, but the increasing datafication of animals and computerisation of biosecurity measures in agriculture that can be transferrable to human populations.Footnote 58
3.4 Dynamic Matching Strategies in FRT Extend Beyond Recognising Human Beings
To securely identify forcibly displaced persons seeking UNHCR repatriation assistance at refugee processing centres the UNHCR records biometrics such as iris, fingerprints, and facial metrics.Footnote 59 Driven in part by a Biometric Matching Engine developed by Accenture, this Population Registration and Identity Management Eco-System (PRIMES) employs a patented ‘dynamic matching strategy’ comprising at least two sets of biometric modalities.Footnote 60 With the advent of new, technologically advanced modes of biometric data gathering and analysis, some of the current ‘international legal thought, doctrine, and practice are, in the main, poorly equipped to deal with them’, especially in situations of forced migration.Footnote 61 One reason is the lack of manual processing options and how the introduction of machine learning can lift the collection of sensitive and personally identifiable information outside the scope of pre-existing legal methods. In grappling with new forms of quantification and statistics these systems do not just contain hundred-year old statistical decision functions but the pairing of imaging, data aggregation, and machine learning at scale. The autonomy granted to machine learning may remove abilities to interrogate the validity of the earlier datasets and matching results a system relies on to achieve a result. Such logic clusters ever increasing data collections into new ‘probabilistic dependencies’.Footnote 62 Yet what this curtails are reasonable efforts to disentangle bias from standardised classifications, and how the natural divergences that occur between humans, different social groups, and their situated actions, are erased in deference to the calculative inferences instead. In the use of FRT there is always ‘politics attached’. Avi Marciano illustrated this in the context of Israel where biometric standards establish hierarchies for decision making by defining particular bodies as ‘ineligible’ to access.Footnote 63
Some FRTs are directly complicit in human rights abuses, including a reported detention of up to 1.5 million Uyghur Muslims in Xinjiang.Footnote 64 Owing to the increasing scale of an inescapable surveillance that the Chinese Communist Party has funded, ubiquitous CCTV systems and facial recognition are operationalised in public spaces alongside the monitoring of online communications and patterns-of-life data from mobile phones. Idealised as an all-seeing pervasive surveillance network enabled by a state manufacturing of computer vision technology, digital platforms, and data aggregation centres,Footnote 65 the simplified idea that Chinese technology and its authoritarian state surveillance system are indigenous is significantly flawed. Before China started using CCTV systems and facial pattern-matching techniques to identify ethnic minorities in Xinjiang Province, Bledsoe proposed to the Defence Department Advanced Research Projects Agency (then known as ARPA) that it should support Panoramic Research Laboratory in studying the feasibility of using facial characteristics to determine a person’s racial background.Footnote 66 This is another instance of the politics and the power of FRT recurring and returning and re-playing into new uses, new places, and new eras, yet with similar purposes.
Western companies were involved in the creation of these systems at the start. The export of surveillance technologies from the Global North to China started in the 1970s. It is only now that Chinese technology companies are found competing with and replacing those suppliers in a globalised market.Footnote 67 The current status of FRT developed in China with known human rights and privacy violations is not adequately restricted by regulatory frameworks in Europe and the United States.Footnote 68 To better disentangle use-cases requires not only a more through mapping of globally entangled and technical supply-chains, whether through critical research or in the building of oversight capabilities such as independent risk assessments, compliance audits, or technical red-teaming in the light of such swiftly evolving material properties.
A contemporary focus on understanding FRT must therefore be concerned not only with the implementation and implications for nation and state-bound privacy law, but to make transparent infrastructural supply chains and situated origins of datasets and technical domains they were created in. This should not simply be restricted to law enforcement and public organisations being required to undertake better procurement strategies – often limited to purchasing orders or responses to requests for information – but to identify the exact sources of the FRT hardware, software, decision functions, and datasets.Footnote 69
Indeed, there are circumstances in which we may need to look further afield. This includes so-called dual-use systems that are adopted not just from domains in nation state and military operations but those trained on animals within precision agriculture.Footnote 70 In the shift from classical identification methods to computer vision tools, the future of farming lies in the paddock-to-plate digital identification of each product. Whether for cross-border bio-security purposes or the optimisation of meat traceability FRT is seen as a viable investment to remotely track animals. These systems commonly utilize open-source software architectures, machine learning and modular camera systems.Footnote 71 Yet in the computational transference between animal bodies, digital and data visualisation, and informational materials, we collapse into the heart of Trevor Paglen’s art project titled in ‘Bloom’. The visualisation and classification of all images and all bodies helps to establish the adoption of autonomous methods. This includes initiatives from the global accounting firm KPMG and Meat and Livestock Australia to collect data that translate into efforts to strengthen computer vision market positions. Agribusinesses are not yet treated as handling any sensitive data or training bodily surveillance systems nor are they subjected to regulatory approaches that can throw their data practices into question.Footnote 72
As Mark Maguire suggests, a genealogical and infrastructural approach to FRT ‘demands we consider how technologies are an assemblage of different elements delivered from specific contexts’ yet re-made, aggregated, customised, adapted, and re-purposed for newly defined, profit-driven, and yet often speculative objectives.Footnote 73
3.5 Conclusion
At the time of Bledsoe’s experiments there was a meeting between the administrative management of the NYSIIS law enforcement data bases and the computer design company Systems Development Corporation (SDC) of Santa Monica, California, in September 1964.Footnote 74 The aim was to decide in what manner to proceed with the implementation of the system, and what techniques to commission for deployment. In summary, the critical inflexion point centred on: ‘First buy the computer and decide what to put on it; (2) Or do an extensive feasibility analysis and, as a result of that study, decide on the computer (how large and powerful) and the functions to be performed.’Footnote 75
As the technical capacity of computing systems in the 1960s was nascent, SDC lacked capability to deliver the required system at scale. Yet this allowed a pause for discussion, consideration, and to recognise that computing capabilities must be defined for a particular purpose, and there should be a thorough vetting of the modular building blocks the system would contain.Footnote 76 The title of that report was ‘A System in Motion’, and it recognised that multiple capabilities – from query and search functions onto image recognition – could not be adequately managed and regulated when developed at once. The NYSIIS report stated the application of computers to solve recognition problems for law enforcement was a foregone conclusion. Yet the question remained whether social institutions and organisations should allow for deploying use of complete automation, especially as they function as a sum of moving, and largely unknown ‘experimental parts’?Footnote 77
Although most state departments and law enforcement undertake basic steps to adhere to industry best practices, such as compliance, testing, and legal obligations to avoid public scrutiny, these approaches often lack consistency. FRT is an experimental practice constituted by practices and elements that can be hidden from view, trialed and tested in domains unsuitable to be deemed fit-for-purpose. Whether being trained on exploitative data captured from refugees, prisoners, or operationalised on farm animals, this is called ‘the deploy and comply problem’ and requires public consultation and impact considerations before being put into action.Footnote 78 A prime example is the use of Clearview AI facial algorithms by New Zealand Police in 2020 without consulting the Privacy Commissioner or considering the impacts to vulnerable Indigenous groups.Footnote 79 This is indicative of multiple instances of harm, error, oppression, and inequality that have been caused by autonomous decision and surveillance systems.Footnote 80 What is needed are efforts to trace, assess, and determine if the modular ‘elements’ of an FRT system are legitimate, credible, feasible, and reasonable. This challenge seeks to ringfence the ‘lineage of intent’ – yet can FRT systems be restricted by ethical, legal and technical guardrails to specific, deliberate, and predefined purposes?Footnote 81 This is what this book is seeking to address.