Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-22T01:37:22.482Z Has data issue: false hasContentIssue false
  • English
  • Français

Emerging Vulnerabilities and Risks in Therapy Manufacture: Cell and Gene Therapy Production as an Emerging Domain of Critical Infrastructure

Published online by Cambridge University Press:  10 October 2024

Edison Bicudo Open the ORCID record for Edison Bicudo [Opens in a new window]  and
Irina Brass
Edison Bicudo*
Affiliation:
Department of Sociology and Policy, Aston University, Birmingham, UK
Irina Brass
Affiliation:
Department of Science, Technology, Engineering, and Public Policy, University College London, London, UK
*
Corresponding author: Edison Bicudo; Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Cell and gene therapies derive from a substantial manipulation of cells or the application of gene editing techniques. They are promising products because they enable therapy personalisation, are potentially helpful for treating rare or resistant diseases, and may become useful in future epidemics. Because of this strategic worth, the infrastructure needed for manufacturing these therapies is turning into a subdomain within the domain of critical infrastructures (CIs), understood as the structures whose operation is key for the integrity and security of a nation. This paper analyses why cell and gene therapy infrastructure can be considered as an emergent CI domain, stressing three aspects: automated manufacturing equipment; software solutions (including the growing adoption of artificial intelligence and cloud technology); and human expertise. These complex manufacturing systems, which are becoming increasingly automated and digitalised, may be surrounded by new risks and vulnerability points, which requires adequate regulatory solutions and governance initiatives. A comprehensive approach is therefore advanced here, where therapy manufacture has medical and technological relevance, but is equally crucial from the viewpoint of nations’ public health and internal stability.

Keywords

cell and gene therapiesdigitalisation and automationrisks
Type
Articles
Information
European Journal of Risk Regulation , First View , pp. 1 - 21
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2024. Published by Cambridge University Press

I. Introduction

As social issues evolve and technologies are developed, the range of activities and resources on which people, nations, and governments decisively depend is expanded. This paper deals with this process, by focusing on how new technologies for therapy production are created, and then become vital for healthcare delivery in societies formed by ageing populations and threatened with chronic diseases and impending epidemics. Thus, the paper analyses cutting-edge approaches for the manufacture of cell and gene therapies.

The latter are medicines deriving from a substantive manipulation of cells or the application of gene editing techniques.Footnote 1 They have been hailed as promising clinical resources, as they enable, for example, the development of therapies with cells taken from the patient, thus promoting therapy personalisation.Footnote 2 This can be crucial in ageing societies where chronic conditions become increasingly prevalent. Gene editing techniques have enabled, for example, the development of therapies aimed to fight resistant types of cancer, with some outcomes that have impressed scientists and clinicians.Footnote 3 Furthermore, cell and gene therapies may prove necessary to cope with future epidemics or pandemics. Indeed, the Covid-19 pandemic intensified the development of these therapies in various countries.Footnote 4

It is argued in this paper that the manufacture of cell and gene therapy is gradually constituting a subfield within the broad field of critical infrastructures (CIs). There are different, but only slightly divergent, definitions for CI. A sufficiently comprehensive one comes from the European UnionFootnote 5: “Critical infrastructures consist of those physical and information technology facilities, networks, services and assets which, if disrupted or destroyed, would have a serious impact on the health, safety, security or economic well-being of citizens or the effective functioning of governments […].” Therefore, health has been covered by the definition of CI in the EU,Footnote 6 as well as in the United States,Footnote 7 Canada,Footnote 8 and other countries.

The definition of CI, as given above, is considerably broad, and perhaps even somewhat vague, for two reasons. On the one hand, one can expect to see, in different countries, some variations in the list of CIs. For example, hydroelectric energy sources and the related infrastructure, including the transmission grid, would be considered as CI in a country where 60% of the energy has this origin, but not in a country where this proportion reaches only 5%. On the other hand, different infrastructures are considered as critical for slightly different reasons. For example, information infrastructures are critical because countries can no longer function without speedy and precise exchange of various sorts of information involving people and organisations, whereas transportation infrastructures are critical because countries’ normal live would collapse if people and material goods could not circulate. Because of all these variations, the definition of CI is underpinned by the broad idea of disruption of essential services or activities.

The phrase “critical infrastructure” was initially used in the United States, when the White House drew attention to the escalation of both physical and cyber threats,Footnote 9 a concern that was greatly reinforced after the 9/11 terrorist attacks.Footnote 10 In this way, some might claim that the globalisation of the CI concept has been a product of American ideological hegemony. However, other incidents have triggered a concern with CIs in other countries, such as the 2004 terrorist attacks in Madrid, with a subsequent discussion culminating in the 2006 European Programme for the Protection of Critical Infrastructure.Footnote 11 More recently, the European Union updated its regulation and terminology regarding critical infrastructure. The 2557 Directive,Footnote 12 approved in 2022, speaks of resilience, understood as infrastructures’ capacity to continue to operate even under strain or attack. This Directive focuses on “critical entities,” which encompasses some new areas such as “digital infrastructure” and “banking,” as well as more traditional areas like “health.” As argued in the literature on CIs, the disruption of vital infrastructures for the nation – including health-related infrastructures – is a threat concerning all countries, not only developed ones.

As explained by Markopoulou and colleagues,Footnote 13 the concept of CI is always being expanded to account for emerging technologies and social concerns, including concerns with digital technologies and cyberattacks. These issues begin to reach the domain of cell and gene therapies, due to recent trends which include: the development of a growing set of products, covering new disease areas; the urgency and medical severity of the conditions generally treated with cell and gene therapies;Footnote 14 and the gradual consolidation of gene-edited therapies for cancer, which “could replace standard cancer treatments in the future”Footnote 15 and thus be deployed to considerably large patient populations.

Furthermore, new vulnerability points may be introduced to cell and gene manufacture as a result of its growing use of digital and automated technologies. It is frequently pointed out that automation and digitalisation reduce errors in therapy manufacture, as more precision is achieved, minimising the space for contamination, optimising processes, and streamlining quality control.Footnote 16 Thus, by automating and digitalising their systems, manufacturers are believed to be putting in place robust and safe CIs. However, in this paper we argue that digitalisation and automation can also introduce new vulnerability points in therapy manufacture if such implementation is not performed in a safe manner. Because of these changes, current risk management systems, largely based on the conduct of clinical trials and the adoption of Good Manufacturing Practices (GMPs), may prove insufficient in the years to come.

Our analysis is guided by lessons and insights from different strands of the CI literature, including the one focusing on the networked nature of CI,Footnote 17 the one stressing vulnerabilities,Footnote 18 and the one interested in the establishment of resilient systems.Footnote 19 If as claimed by some analysts,Footnote 20 the identification of CI domains is key to ensure the protection of these structures, then it is now important to acknowledge the gradual emergence of a new CI domain in the field of cell and gene therapy manufacture.

To demonstrated how this process has evolved, this paper is organised as follows. The first section outlines the research methods we have mobilised. Subsequently, we analyse the emergence of vulnerability points in automated and digitalised cell and gene therapy manufacture, highlighting new risk areas associated with hardware, software, and human expertise. After a brief discussion, we present some concluding remarks.

II. Research methods

This study was conducted, from 2021 to 2023, at the Department of Science, Technology, Engineering and Public Policy, University College London (UCL), in the framework of the Future Targeted Healthcare Manufacturing Hub. UCL’s Research Ethics Committee reviewed and approved the following three research methods.

First, we conducted a literature review, involving books, papers, pieces of legislation and regulatory guidance, and grey literature publications. This revealed the main issues and challenges currently being discussed by professionals involved, or interested, in cell and gene therapy manufacture.

Second, from May to August 2023, we conducted an online survey using the Qualtrics platform,Footnote 21 exploring expectations and impressions about automation and digitalisation in cell and gene therapy manufacture, as well as possible associated risks. In total, forty-five participants completed the online questionnaire presented in Appendix 1. For data processing, data visualisation, and statistical tests, the R platformFootnote 22 was used.

Third, from 2021 to 2023, 64 qualitative, in-depth interviews were conducted. Table 1 outlines interviewees’ affiliations, as well as those of survey participants.

Table 1. Summary of qualitative interviews and the online survey

The online survey covered a narrow group of organisations, those more directly engaged with the development of solutions and technologies to automate and digitalise cell and gene therapy manufacture.

The world regions/countries of interviewees and survey participants are shown in Chart 1.

Chart 1. Regions/countries in which research participants are based.

The interviews explored the technical and regulatory challenges of cell and gene therapy manufacture, including possible vulnerabilities deriving from its growing automation and digitalisation. This included issues such as: availability/dearth of software packages useful for therapy manufacture; connectivity between devices and software packages; concerns with the use of cloud technology; availability/dearth of standards; and regulatory oversight of innovative manufacturing systems. Interview analysis was guided by the principles of content analysis whereby emergent themes are identified in interviewees’ answers.

Interviewees signed an informed consent form, including the selection of the way in which they would be identified in our publications. Some interviewees accepted to disclose the names of their organisations whereas others preferred to remain completely anonymous. In this paper, we follow the type of identification chosen by each interviewee.

Some of our interviewees were also invited to participate in the online survey, but it is not possible to know how many did so, as the online questionnaire was completely anonymous.

In the next section, we begin to analyse the emergence of vulnerability points in the newest therapy manufacturing systems.

III. Therapy manufacture as an emerging CI domain

Identifying domains to be considered as critical infrastructure (CI) is not always uncontroversial and straightforward. According to LautaFootnote 23: “[…] governments and public officials have worked diligently to finitely capture which parts of the society’s infrastructure should be considered critical, and which should not.”

It is argued here that infrastructures associated with manufacturing cell and gene therapies are rapidly gaining the status of CI, especially in those countries where such manufacture is at more advanced stages. These therapies have potential to become a pivotal strand of both common healthcare and the response to public health emergencies and epidemics. In this regard, at least three aspects should be taken into account. First, by 2030 over 1 billion people in the world will be aged over 65 years.Footnote 24 Cell and gene therapies are scientifically considered as promising for the treatment of conditions associated with aging, such as cancer and cardiac diseases.Footnote 25 Second, cell and gene therapies are commonly associated with rare diseases, but recent studies have targeted conditions of large patient populations, such as diabetesFootnote 26 and eye diseases,Footnote 27 which can eventually accord more central positions to these therapies in healthcare systems. Third, cell and gene therapies may prove essential in future disease outbreaks, in a world where chaotic urbanisation and advanced globalisation heighten the likelihood of epidemics and pandemics. For these reasons, the effective provision of healthcare, and hence the preservation of countries’ normal operations, can become, in the decades to come, increasingly dependent on the infrastructures required for cell and gene therapy manufacture. If attacks, failures or errors come to disturb the operation of these infrastructures, an important section of healthcare provision will be paralysed, harming patients, straining clinical settings, and perhaps compromising a quick response to disease outbreaks.

To demonstrate how the structures associated with cell and gene therapy manufacture are gradually acquiring this status of CIs, we analyse below three of its dimensions: hardware, software, and human expertise.

1. Hardware

Like other CI-related activities, cell and gene therapy manufacture must happen in highly protected spaces. Furthermore, access needs to be ensured to basic resources, as explained by Interviewee 31 (Fraunhofer Institute, Germany):

[…] the hardware part, so, what we need is, we need electricity, obviously; we need gases, so O2, CO2, pressured air. This is something that is available in all clean room suites […] Or at least we would hope that it’s not a problem in clean room suites.

Those specialised, clean spaces are found in manufacturing units of companies but also in GMP facilities installed in some hospitals, as we showed elsewhere.Footnote 28 Shortages in basic resources such as electricity and CO2 would then constitute a vulnerability point in therapy manufacture. Companies and hospitals have contingency plans in place, as these shortages represent a threat to healthcare in general.

The vulnerabilities of material infrastructures are sometimes pointed out by those stressing the advantages of digital systems. For example, Interviewee 43, based in a software company (MyCellHub, Belgium) declared: “If you work on paper and your facility is flooded, the paper is gone. If you work on digital and your facility is flooded, no worries, everything is still in the cloud.” At the same time, however, new kinds of vulnerabilities may be introduced by the growing interdependence between material equipment and digital systems. As claimed by Alosert and colleagues,Footnote 29 depending on how manufacturing systems are installed and managed, the reliance on “IT infrastructure” sometimes “[…] poses risks to the continuity of a manufacturing process and ultimately the success of a product if a system breaks down.”

On this material infrastructure side, cell and gene therapy manufacture is absorbing the practices of Industry 4.0,Footnote 30 with more automation (that is, the use of machines to replace manual operations) and with increasingly high levels of device–device connectivity. Equipment manufacturers are indeed striving to create closed systems, combining different devices and robotic platforms operating with as little human intervention as possible. It is then key that each device has interfaces to communicate with software and other devices. This point was made by Interviewee 1 (Adva Biotechnology, Israel), based in a company offering automated manufacturing equipment for cell and gene therapy: “People won’t buy a stand-alone device if it doesn’t talk with anything else. If you want a continuous manufacturing […], you have to connect between other devices.”

Employees of companies offering manufacturing equipment do tend to consider automation as a relatively safe endeavour, as seen in Chart 2, with data from our online survey.

Chart 2. Perception of risk in automated cell and gene therapy manufacture.

Therefore, none of the survey participants consider that automation makes therapy manufacture “riskier” or “much riskier.” Participants based in companies offering manufacturing equipment (hardware companies) are the most confident, frequently declaring that manufacture becomes “much safer.”

However, our literature review and qualitative interviews provided a more nuanced picture. When manufacturing devices are connected, starting materials, cells, and final products may be moved from device to device by means of sterilised plastic tubes. Such tubing connections are generally not specifically designed for cell and gene therapies,Footnote 31 and their viability has therefore been “[…] carefully based on the safety and reliability considerations of other substances such as protein solutions.”Footnote 32 This adaptation constitutes a weak point of current systems because: “Using those connectors in cell therapy development can incur risks related to introducing contamination, damaging cell products, and leaking.”Footnote 33

These issues are compounded by the current lack of standardisation, with a variety of technical approaches being adopted by equipment providers, such as robotic systems, liquid pumps, single-use tubing connections, imaging technologies, and others. In addition, equipment evolves quickly, which is also true for systems that have been widely used in the field. One example is the CliniMACS Prodigy system, which has been offered, since 2013, by Miltenyi Biotec, a German company. As explained by Interviewee 47 (Miltenyi Biotec, Germany):

At the moment you are only able to do […] therapy manufacturing, but in the future we are planning to connect this to respective devices of the Prodigy portfolio in order to have the full automated service. So we’re speaking about lab automation devices which will interact with the Prodigy and communicate between the machines in order to transfer information but also probably samples in the future.

As different companies create and modify systems by applying their own solutions, the field can eventually become complex, diverse, and difficult to oversee from a regulatory point of view. This is coupled with a series of open questions pertaining, for example, to component validation when therapies are manufactured with cells taken from the patient, or to how single-use components – such as connecting tubes – should be managed and tested.Footnote 34 Furthermore, cellular and gene-edited therapies are nowadays produced in small batches, for relatively small patient populations. If patient populations increase, requiring larger production scales, it is not sure that systems will continue to be robust and safe.Footnote 35

In this way, safety considerations begin to emerge, with the seriousness and complexity typical of CIs. They are combined with considerations from a burgeoning digital side of cell and gene therapy manufacture, as reviewed below.

2. Software

The relevance acquired by software in the operation of CIs has been pointed outFootnote 36 and is also valid for advanced strands of healthcare and therapy manufacture, because: “Data science and informatics methods provide critical infrastructure for precision health.”Footnote 37 Manufacturing units tend to gradually resemble so-called cyber-physical systems, where physical equipment is controlled and monitored by digital algorithms.Footnote 38

The adoption of digital solutions in therapy manufacture has been praised in both the literature and specialists’ discourses. Interviewee 43 (MyCellHub, Belgium), for instance, stressed the benefits of digital documentation over manual documentation in manufacturing units:

The most critical benefit is that you can prevent errors. So especially for, let’s say, cell and gene therapies […], making an error in the paperwork could mean that the patient doesn’t get his product or his medicine […].

In our online survey, participants based in software companies did express the highest enthusiasm about digitalisation. Of the fourteen questions we asked, nine provided options organised in five steps, from the most enthusiastic to the most suspicious stance. For analysis, we attributed a score of 5 to the most open attitude whereas the most suspicious one received 1 as score.Footnote 39 We then added up the scores, creating an index for each participant. The higher the index, the more enthusiastic the participant is in relation to automation and digitalisation in cell and gene therapy manufacture. The outcome is presented in Chart 3.

Chart 3. Index of survey participants’ optimism regarding digitalisation and automation in cell and gene therapy manufacture.

Even though each participant group has a small size, there is remarkable coherence within them, with participants in software companies displaying the highest index. Performing a statistical test,Footnote 40 we find a significant difference between these boxplots. A post-hoc test showed that the relevant difference lies between people in consulting companies and those in software companies.Footnote 41 The high index for software companies reflects a commercial bias which was seen above for hardware equipment (Chart 2). In turn, professionals based in consulting firms help biotech companies go digital, which make them aware of the manifold intricacies of this process, generating a more cautious attitude.

In spite of these differences, the survey generally revealed a rather positive attitude towards digitalisation. Yet the digital side of therapy manufacture, due to its recency, is surrounded with uncertainties and vulnerability points. As we showed elsewhere,Footnote 42 computer code is an inherently unstable product, which can have medical implications when software is used in healthcare-related activities. Vulnerability points include “[…] software updates, risk of computer viruses, internal operation system errors and unexpected interruptions […].”Footnote 43 When software is applied in therapy manufacture, bugs need to be carefully monitored, because as claimed by Interviewee 31 (Fraunhofer Institute, Germany): “[…] an automated system will do exactly what you tell it to do […] if you program your process wrong, then of course you will have an error every single time. And if you start up a new system, you will always have bugs, initially.”

A relevant area is that of artificial intelligence (AI), which begins to become a reality in therapy manufacture. AI algorithms can be most useful for identifying patterns in manufacturing data, determining critical points in bioprocessing, and performing quality control based on data from the production process.Footnote 44 So far the application of AI in therapy manufacture has been restricted to academic laboratories, as well as basic tasks performed by commercial manufacturing equipment. When such solutions are eventually applied at larger scales, much care will be necessary because: “AI-based algorithms are prone to behaving in unpredictable ways when applied in the real world.”Footnote 45 In the domains of medical devices and clinical diagnostics, for example, a persistent challenge has been the variable operation of AI algorithms, which tend to display different performances in different sites.Footnote 46 Such variations would be disturbing in cell and gene therapies, whose safe production requires consistency and reproducibility across manufacturing sites.Footnote 47

Another point deserving attention is the increasing use of cloud technology in software applications for cell and gene therapies. As laboratories and companies are processing expanding amounts of data, it is useful to access products on a software-as-a-service basis, whereby data are processed and stored in the cloud. According to Interviewee 20 (Digital Lab Consulting, UK), “that’s probably the norm now.” This solution is considered as safe and robust by some specialists such as Interviewee 59, based in a consulting firm:

If you think about somebody trying to steal the data, it’s probably easier to break in physically into your laboratory and actually steal the hardware than breaking into a cloud system […] Nowadays, it’s really pretty easy to secure your cloud systems from such attacks, by using encryption mechanisms and things like that.

However, this certainty is not widespread. In our survey, cloud technology was the point inspiring the most intense concerns, as seen in Chart 4.

Chart 4. Concerns about the use of cloud technology in cell and gene therapy manufacture.

Most survey respondents, particularly academics and consultants, were “very concerned” about cloud technology. Cybersecurity preoccupations have been present in other health-related areas, such as medical devices, as we studied before.Footnote 48 This is a crucial point from the perspective of CI protection too. “The health sector has been identified as one of the most important Critical Infrastructures, that has also been the target of a continuously increasing number of cyber incidents.”Footnote 49

Previously,Footnote 50 we showed that some cell and gene therapy manufacturers are considering point-of-care manufacture, whereby the therapy is produced in the hospital or close to the hospital – as opposed to using centralised units. If point-of-care manufacture and the use of cloud technology are combined, then the potential for cyberattacks increases, because, as shown by Jalali and Kaiser,Footnote 51 some healthcare institutions struggle to ensure basic cybersecurity.

These cyber threats are not hypothetical and are beginning to be dealt with by some organisations. For example, the Fraunhofer IPT Institute (Germany) is leading the development of an automated system named AIDPAPTH, which combines various manufacturing devices controlled by software incorporating artificial intelligence approaches. Aimed to manufacture gene-edited therapies to treat resistant cancers, the system will be initially tested in three hospitals in the city of Wurzburg. In order to gather and process the data generated by the manufacturing sites, cloud technology will be used, feeding the controlling software. Interviewee 32 (Fraunhofer Institute, Germany) explained:

[…] this is a really sensitive topic, but we do have one company in the project developing such a […] patient management software. And we have security experts. We are currently looking on how […] patient data can be stored in the cloud and can then be connected to the control software.

Ideally, in this kind of manufacturing system, patient data would be fully anonymised, that is, slightly altered to prevent personal identification. However, Interviewee 9, who participated in BigData@Heart, an international consortium for medical data sharing, explains: “[…] in order not to be able to identify anyone, you need to destroy a lot of data […] You need to round up some numbers, for instance blood pressure numbers, and so on […].” If this is done, personal identification is prevented but manufacturers may forfeit the quantitative refinement necessary for processing data in cell and gene therapies.

To deal with all these possible vulnerabilities, software developers have several methods, including computer code auditingFootnote 52 and security testing.Footnote 53 Blockchain technology can also become a tool ensuring data security and consistency in health-related applications.Footnote 54 More recently, automated tests have emerged, with machine learning approaches being used to identify software vulnerabilities.Footnote 55 This is, for example, the approach adopted in Interviewee 61’s company (Stemsoft, Canada):

[…] we use a test-driven development methodology with 100 per cent test automation […] Every check-in of code is retested automatically. That means, if a developer breaks something [in the code base], we know about it that day. We don’t know about it six months from now when it goes through a manual […] release phase.

Therefore, the further digitalisation of highly regulated therapy production will require that systems be consistently validated. For “it’s the process of validation that ensures that that piece of software is suitable for use in a regulated environment,” as Interviewee 20 (Digital Lab Consulting, UK) put it.

Considering the national and strategic relevance that cell and gene therapy manufacture can acquire, it will be key to reinforce these software security strategies. This demands the presence of professionals with the necessary expertise, as explained below.

3. Human expertise

In addition to hardware and software issues, it is important to consider a third aspect which is not always duly accounted for in the CI literature: human expertise. Indeed, technical factors may be insufficient for setting up robust and secure CIs. Sometimes, “[…] the organizational dynamics like security culture and human factors may be more effective for the improvement of security.”Footnote 56

Interviewee 25 (National University of Ireland Galway) declared that in GMP manufacturing facilities, “most of the errors are human errors.” Equally, the literature frequently points to risks entailed by human fatigue, imprecision, and person-to-person variability.Footnote 57 “The more times a cell therapy batch is touched, the more chances for human error.”Footnote 58 In this sense, automation would in theory bring about increased safety.

Therapy manufacture possesses a characteristic which according to Aaken and Wildhaber,Footnote 59 is common in CIs: the field is populated by various organisations adopting divergent procedures. When such organisational variability occurs, the overall technical system may become a source of errors, but analysts still tend to be primarily wary of human errors.Footnote 60 In this way, it is important to remember that tasks performed by individuals are not always problematic and risky. Even when therapy manufacture becomes highly automated, there are domains in which human expertise is rather necessary to ensure accuracy and safety.

When migrating from old ways of working to more digitalised approaches, therapy manufacturers need professionals capable of implementing the transition. For example, in their study on cloud technology in healthcare settings, Cresswell and colleagues witnessed the difficulties faced by some hospitals. “Here, a lack of existing knowledge and skills in organizations to deploy and exploit cloud functionality was an important rate-limiting step. For instance, organizations frequently lacked implementation and migration skills […].”Footnote 61

Some of our interviewees, particularly those in consulting firms, also pointed out this issue of technology migration. For example, therapy manufacturers may need to update their Laboratory Information Management System (LIMS) software, used to control and document procedures and workflows. This may involve what Interviewee 20 (Digital Lab Consulting, UK) named “regulated data”: those which must be submitted to the regulatory agency. If technology update or migration is not skilfully performed, data can be lost or corrupted, as this interviewee explained: “That’s the whole point of doing things like data migration plans and data migration testing. It is to […] minimise the risk of that happening, or if it does happen, to make it detectable.” One of the reasons why manufacturers hire services from consulting firms is precisely the need for accessing expertise for technology migration.

Hardware management is another area demanding human expertise. Even though automated manufacturing systems require less human intervention, and might be much less demanding in terms of operators’ skills, their initial installation can be quite complex, sometimes demanding engineering knowledge. For example, in the AIDPATH initiative led by the Fraunhofer IPT institute (described in the previous sub-section), little human intervention will be needed when the system is eventually fully operational, but setting it up is a considerably complex task. According to Interviewee 31 (IPT Institute, Germany), “[…] the installation of the system right now needs to be carried out by IPT, honestly.”

Additionally, if software has become so central, as explained above, the question is where professionals will be pooled from to write code bases and develop packages. Certainly, many products will be found, ready-made, on the market, such as Laboratory Information Management Systems, as well as Quality Management Systems. However, when it comes to combining various devices in a single manufacturing system – like in the AIDPATH case – it may be necessary to develop new drivers or packages enabling data sharing and promoting device-computer communication. Furthermore, professionals must be present to make sure that all the virtual products and connections continue to work safely over time.

According to van Asselt and colleagues,Footnote 62 attention should be paid to “how expertise can be valuable in the governance of risk.” In cell and gene therapy manufacture, this is also crucial, as the points briefly made above suggest. The growing automation and digitalisation of this field might evoke concerns about countless jobs being replaced by technology.Footnote 63 However, two factors should be considered. On the one hand, automation does not always eliminate the need for high-skilled workers. For example, Interviewee 29 (University College London) has a collaboration with a biotech company producing a gene-edited therapy. Even though this company has used automated systems, high-skilled employees are necessary to perform and oversee a range of manufacturing tasks. On the other hand, depending on how the transition from manual to digital is realised, professionals, instead of losing their jobs, could be moved from basic manufacturing positions to higher-skilled research and development positions.Footnote 64 And artificial intelligence solutions, instead of necessarily being a threat to workers in cell and gene manufacture, can be a new resource in future workers’ hands.Footnote 65

However, availability and maintenance of human skills seem to be turning into a rather serious issue in this field. Because of a dearth of skilled professionals,Footnote 66 “[…] companies have a limited pool of talent for manufacturing staff with very high turnover rates.”Footnote 67 If this situation seems problematic from a market point of view, it is even more concerning from a CI point of view. Countries might have difficulties for installing and running those manufacturing infrastructures if they cannot train and mobilise sufficient numbers of professionals. Emerging manufacturing systems do ask for new sets of expertise, with new responsibilities, without which therapies may fail to be produced, and public health programs may fail to progress.

IV. Discussion

Given the issues addressed in the previous section, cell and gene therapy manufacture infrastructure, considered as an emerging domain of CI, is of public interest, with an important role to be played by regulators and public inspectors. It is then important to avoid a gap that has surprisingly marked the CI literature, namely, the little attention given to regulatory issues. “The number of academic studies that are about regulatory approaches on critical infrastructures is limited.”Footnote 68

In cell and gene therapies, it is generally considered that automation and digitalisation enhance regulatory compliance by standardising data collection and processing, as well as minimising human errors in data handling.Footnote 69 Furthermore, it is assumed that a reliable documentation of processes is made possible by manufacturing devices that generate log files with data regarding the parameters and specifications of therapy production. Indeed, equipment for cell and gene therapies, such as Prodigy (offered by Miltenyi Biotec, Germany) and Cocoon (by Lonza, Switzerland), do generate a log file “[…] which forms the basis of the QM [quality management] documentation required for every GMP manufacturing process.”Footnote 70

However, at least three issues come to blur this promising picture. First, automation has not been accompanied by standardisation. In cell and gene therapies, much variability is to be expected in manufacturing processes, depending on the product’s attributes and risk profile. Generally, manufacturers expect measurements to vary within certain safe ranges, but these ranges will vary from manufacturer to manufacturer, which is compounded by the variety of devices, connections, and protocols now being used.Footnote 71 “Those differences create the potential for significant product/process failures because there are no agreed expectations about the minimum failure rate. Such variability ultimately leaves patients vulnerable to unintentional harm.”Footnote 72 Second, quality control is based on the identification of certain biomarkers in final medicinal products. However, it is sometimes doubtful whether such biomarkers are sufficient to guarantee that a certain medicine will always have the expected clinical effect. This uncertainty can be even larger when it comes to personalised medicines produced with cells taken from the patient.Footnote 73 Finally, cell and gene therapy manufacture has displayed a feature typical of CIs: the field is populated by “a multiplicity of actors”, making it difficult to manage and distribute liabilities.Footnote 74

Therefore, rising levels of automation and digitalisation simplify some regulatory tasks but also require new regulatory solutions. For example, the use of cloud technology tends to concern some players, as seen above, making regulators and companies engage in a dialogue to manage possible vulnerabilities. According to Interviewee 59 (Consulting firm), this dialogue has been frequent.

[…] we validate loads of systems in the cloud in regulatory spaces. And it’s not the cloud or the technology which is the problem. The problem is you have to convince or prove that you did a proper risk assessment, and you have to do this risk assessment in a way that it conforms to the regulatory bodies […] and prove to an auditor or an inspector that you know what you are doing.

Regulators are also aware of other technical approaches, including artificial intelligence. Interviewee 26 (Hardian Health, UK), for example, is based in a consulting firm and is collaborating with the UK regulator as “[…] a specialist advisor on upcoming regulatory frameworks for AI and machine learning devices.”

In overseeing CIs, regulators have relied on some standardsFootnote 75 – a phenomenon that can repeat itself in cell and gene therapy manufacture. In terms of physical connections, providers of connecting tubes have sought certification to ISO 9001, and sometimes also standards for related areas, such as ISO 13485, valid for medical devices.Footnote 76 As for digital technologies, the most remarkable initiative has been the publication of the Good Automated Manufacturing Practices, an initiative led by the International Society for Pharmaceutical Engineering.Footnote 77 However, and once again, this standardisation and regulatory clarity is in its infancy, and certainly much less advanced than the guidelines found in related fields such as medical devices.Footnote 78

Obviously, cell and gene therapy manufacture, framed as an emerging CI domain, should be steered not only by standards but also by mandatory regulations. In this sense, rules to be created can be strategically combined with existing rules, especially those related to the proper operation of manufacturing systems. There are the basic regulations pertaining to cell and gene products, such as the ATMP Regulation of the European Union,Footnote 79 which pertains to products deriving from gene editing, cell manipulation, tissue engineering or a combination of these techniques. Most manufacturing equipment will be subjected to procedures ensuring the safety of mechanical systems, such as the UKCA mark. And considering the growing digitalisation of manufacturing systems, new guidelines and directives on the matter will apply, such as the European Union’s Artificial Intelligence Act.Footnote 80 As is common for CIs, the protection of cell and gene manufacturing infrastructure can then be underpinned by both existing regulations and other frameworks yet to be designed.Footnote 81

Elsewhere we stressed that connected devices and software evolve in time.Footnote 82 For this reason, both the EU regulator (EMA) and the American regulator (FDA), in their oversight of medical-related software, require post-market follow-up plans to account for changes made to software after commercialisation. In cell and gene therapy manufacture, with the growing relevance of software, such care will be more and more important, so that supporting software can be assessed in its historic and spatial dimensions. On the historic side, the main question is: “How does the algorithm perform over time, both in terms of maintaining and improving performance, with periodic updates as appropriate?”Footnote 83 As for the geographical dimension, it is worth asking: “How does the algorithm perform at every local site […]?”Footnote 84 In the longer term, regulators might even rely on software auditing practices, including those performed by third parties – something that has been seen in areas such as clinical trials.Footnote 85

In the same way that software evolves, regulations evolve and sometimes trigger changes in software design. For example, the collection and processing of some kinds of cells follow the guidelines designed by the Joint Accreditation Committee of the International Society for Cellular Therapy (JACIE). In England and other European countries, these guidelines have been made mandatory for clinical sites and manufacturers processing some types of cells. In Interviewee 61’s company (Stemsoft, Canada), software has undergone modifications to comply with these guidelines:

One of the more recent editions [of these guidelines] added a label reconciliation requirement, and that prompted us to make a change to our software […] we exposed the audit trail in a way that made it very easy to see this label has been printed once and only once, there were no reprints, there was no regeneration […], all that for traceability. That’s something we added […] because JACIE added it.

It has been noted that: “Critical infrastructures are mostly owned and operated by private entities in developed countries.”Footnote 86 In the case of cell and gene therapy manufacture infrastructure, in its conversion into a CI area, we are dealing with pharma and biotech companies whose operations may involve several countries and whose supply chains have become highly international. In this way, governments and regulators will need to adjust to the field’s emerging features, being ready to “[…] request information from the CI providers or incentivize them to reveal information.”Footnote 87 In parallel, efforts towards international regulatory convergence may also be beneficial, even though this sort of convergence has been limited in the field of cell and gene therapies.

V. Conclusion

This paper has analysed emerging vulnerability points and risks in cell and gene therapy manufacture systems, which are becoming increasingly automated and digitalised. Cell and gene therapies enable medicine personalisation and can potentially help tackle chronic and resistant conditions such as cancer and rare motor diseases, in addition to being potentially useful in future epidemics. In this way, the infrastructure necessary for their production tends to constitute an area of critical infrastructures (CIs), understood as infrastructures whose operation guarantees services and activities which are vital for countries.

As confirmed by the recent EU Directive on critical entities,Footnote 88 health-related infrastructures are part of the CI landscape. More specifically, the Directive mentions “entities carrying out research and development activities of medicinal products.” With recent advances in cell and gene therapies, which can potentially tackle unmet medical needs, one can expect that such research and development tasks will lead to curative activities broadening the scope of health-related CIs. Moreover, the digitalisation of therapy development and deployment, as analysed above, reinforces these considerations, with updated regulations tending to frame digital infrastructures as another domain of CIs, as is the case of the new EU Directive.

With the proliferation and refinement of automated manufacturing equipment and software solutions, the technical side of therapy manufacture is progressing consistently. However, as argued by Stone,Footnote 89 it is often “[…] easy to fall into the trap of creating highly functional solutions that are not fit for the final purpose, thus neglecting to consider appropriately the risks we face.” To make these possible risks stand out, this paper has focused on three domains. First, manufacturing equipment (hardware) has been an expanding area – with devices being sometimes combined – but this creates doubts regarding the use of connecting tubes not specifically designed for cell and gene therapies, and a lack of technical standardisation frequently related to commercial lock-ins. Second, there has been a proliferation of software companies and products, but doubts seem to be even more intense here, due to the recent incorporation of approaches such as artificial intelligence. Moreover, the growing use of cloud technology has diffused concerns related to cybersecurity, especially for manufacturing systems that may be deployed in hospitals. Finally, the human side is also relevant, as it is not sure that qualified experts will be available in sufficient numbers for tasks such as equipment installation, technology updates, and software quality and safety control.

Our proposal that cell and gene therapy manufacture infrastructure is becoming an area of CI may provoke some reservation or critique. The CI literature has pointed out the political and contested nature of decisions aimed to decide what should be considered as critical or not.Footnote 90 Furthermore, the list of resources deserving critical infrastructure protection (CIP) has grown to seemingly unreasonable lengths in some countries. For instance, “[…] the American authorities had in 2009 included 77.000 assets in their CIP-inventory.”Footnote 91 Nevertheless, it seems to us that pointing to the critical nature of therapy manufacture is important for highlighting the strategic worth of such activities, in addition to stressing the need for streamlined regulatory frameworks. Indeed, it was seen above how regulators and therapy manufacturers have already sought new kinds of dialogue and regulatory guidance, so that the newest manufacturing systems continue to evolve in safe ways.

Some risks are characterised by their having “high-impact-low-likelihood,” to use words from van Asselt and colleagues.Footnote 92 These are the risks found in cell and gene therapies, which are frequently delivered to patients who are in small numbers but are terminally sick. It is key to consider the high-impact of these medical risks, even if they have low-likelihood from a technical viewpoint. Moreover, it is key to go beyond this technical viewpoint to reach the realm of national policies and public health, which this paper has attempted to realise by means of a dialogue with the CI literature. In this way, it is possible to circumvent the managerial approach to risk that has nowadays become widespread, emphasising technicalities to the detriment of governance and political views.Footnote 93 Strengthening manufacturing systems for cell and gene therapy is surely optimal for patients’ safety and technology development, but is also critical from the viewpoint of nations’ integrity and security.

Acknowledgements

This study has been carried out with funding from the UK Engineering & Physical Sciences Research Council (EPSRC), under the grant reference EP/P006485/1, for the Future Targeted Healthcare Manufacturing Hub based at University College London (UCL). We would like to thank our colleagues within the Hub and all our interviewees for sharing their time and expertise with great generosity.

Competing interest

The authors declare that they do not have any conflicts of interest.

Appendix 1: Online survey questionnaire

  • I – In your view, In the next 5 years, the automation and digitalisation of life sciences laboratories, pharma companies, and biotech companies will:

    1. 1. Expand rapidly

    2. 2. Expand slowly

    3. 3. Remain on the level it is today

    4. 4. Be slowly reduced

    5. 5. Be rapidly reduced

  • II - What is the life sciences/biotech domain where software and data processing can have the biggest impact in the next 5 years?

    1. 1. Supply chain management

    2. 2. Laboratory workflow management

    3. 3. Manufacture

    4. 4. Process analytics and products’ quality control

    5. 5. Quality management and regulatory compliance

    6. 6. Data mining for product development

    7. 7. Overall business management

    8. 8. Other: _______________

  • III – In terms of risk management, the risks faced by a highly digitalised laboratory are:

    1. 1. Much fewer than the risks faced by a laboratory reliant on paperwork and manual processes

    2. 2. Fewer than the risks faced by a laboratory reliant on paperwork and manual processes

    3. 3. As numerous as the risks faced by a laboratory reliant on paperwork and manual processes – even though the risks have a different nature

    4. 4. More numerous than the risks faced by a laboratory reliant on paperwork and manual processes

    5. 5. Much more numerous than the risks faced by a laboratory reliant on paperwork and manual processes

  • IV – In terms of cutting-edge automated solutions for cell and gene manufacture (such as robotic systems and automated cell processing devices), life sciences laboratories and companies should ideally:

    1. 1. Implement them immediately, because their robustness and safety are very clear

    2. 2. Implement them in the short term, because their robustness and safety are clear

    3. 3. Wait and see what happens in the field, including data about these technologies’ robustness and safety

    4. 4. Implement then in the long term, because their robustness and safety will not be evaluated soon

    5. 5. Not implement them, because they are neither robust nor safe

  • V – In terms of cutting-edge software solutions for cell and gene manufacture (such as artificial intelligence, cloud technology, digital twins, and process analytical technology), life sciences laboratories and companies should ideally:

    1. 1. Implement them immediately, because their robustness and safety are very clear

    2. 2. Implement them in the short term, because their robustness and safety are clear

    3. 3. Wait and see what happens in the field, including data about these technologies’ robustness and safety

    4. 4. Implement then in the long term, because their robustness and safety will not be evaluated soon

    5. 5. Not implement them, because they are neither robust nor safe

  • VI - Nowadays, academic laboratories and biotech companies are using a variety of devices (cell processing devices, centrifuges, bioreactors, microscopes, and others). Sometimes, these devices are combined and connected, in spite of the sometimes-difficult interfaces between them (such as the existence of various Communication Protocols). This practice brings about:

    1. 1. No risks for therapy development and manufacture

    2. 2. Some manageable risks for therapy development and manufacture

    3. 3. No change in terms of risks for therapy development and manufacture

    4. 4. Risks that are hard to manage in therapy development and manufacture

    5. 5. Unmanageable risks for therapy development and manufacture

  • VII - Nowadays, academic laboratories and biotech companies are using a variety of software packages (Laboratory Information Management Systems, Electronic Laboratory Notebooks, Manufacturing Execution Systems, and others). Sometimes, these software packages are combined and connected, in spite of the sometimes-difficult interfaces between them (such as lack of Application Programming Interfaces). This practice brings about:

    1. 1. No risks for therapy development and manufacture

    2. 2. Some manageable risks for therapy development and manufacture

    3. 3. No change in terms of risks for therapy development and manufacture

    4. 4. Risks that are hard to manage in therapy development and manufacture

    5. 5. Unmanageable risks for therapy development and manufacture

  • VIII – Considering the growing digitalisation and automation of cell and gene therapy manufacture, the regulatory framework for quality control and batch release should be:

    1. 1. Much less strict than it is today

    2. 2. Less strict than it is today

    3. 3. As it is today

    4. 4. Stricter than it is today

    5. 5. Much stricter than it is today

  • IX – Considering the increasing use of cloud technology and Software as a Service in life sciences laboratory and companies, how concerned are you with patient data leak and cyberattacks in this field?

    1. 1. Not concerned at all

    2. 2. Slightly concerned

    3. 3. Neutral

    4. 4. Very concerned

    5. 5. Extremely concerned

  • X – With more automated systems and less manual work in life sciences labs and companies, therapy manufacture becomes:

    1. 1. Much safer

    2. 2 Safer

    3. 3 Neither safer nor riskier than it is today

    4. 4 Riskier

    5. 5 Much riskier

References

1 European Parliament and the Council, Regulation (EC) No 1394/2007 of the European Parliament and of the Council (2007).

2 Sofieke de Wilde and others, “Clinical Development of Gene- and Cell-Based Therapies: Overview of The European Landscape” (2016) 3 Molecular Therapy – Methods & Clinical Development 1; Nicholas Medcalf, “Centralized or Decentralized Manufacturing? Key Business Model Considerations for Cell Therapies’ (2016) 2 Cell & Gene Therapy Insights 95.

3 Muhammad Zaheer Abbas, “Strategic Use of Patent Opposition Safeguard to Improve Equitable Access to Innovative Health Technologies: A Case Study of Car T-Cell Therapy Kymriah” (2020) Early Access Global Public Health 1; Vicki Brower, “The CAR-T Cell Race” (2015) 29. Available at: https://www.the-scientist.com/bio-business/the-car-t-cell-race-35701 TheScientist; Adrian P. Gee, “GMP CAR-T Cell Production” (2018) 31 Best Practice & Research Clinical Haematology 126; Bruce L. Levine and others, “Global Manufacturing of CAR T Cell Therapy” (2017) 4 Molecular Therapy: Methods & Clinical Development 92; Xiuyan Wang and Isabelle Rivière, “Clinical Manufacturing of CAR T Cells: Foundation of a Promising Therapy” (2016) 3 Molecular Therapy – Oncolytics 1; Ming Liu and others, “Global Hotspots and Future Prospects of Chimeric Antigen Receptor T-Cell Therapy in Cancer Research: A Bibliometric Analysis” (2020) 16 Future Oncology 597; Andrew Fesnak and Una O’Doherty, “Clinical Development and Manufacture of Chimeric Antigen Receptor T Cells and the Role of Leukapheresis” (2017) 13 European Oncology and Haematology 28.

4 Kevin Doxzen, “Gene therapy can make a real impact on global health but we need equitable access, say experts” World Economic Forum <https://www.weforum.org/agenda/2022/10/experts-gene-therapy-impact-on-global-health/>.

5 Commission of the European Communities, Critical Infrastructure Protection in the fight against terrorism (European Parliament 2004), p. 3.

6 Kristian Cedervall Lauta, “Regulating a Moving Nerve: On Legally Defining Critical Infrastructure” (2015) 6 European Journal of Risk Regulation 176; Dimitra Markopoulou and Vagelis Papakonstantinou, “The Regulatory Framework for the Protection of Critical Infrastructures Against Cyberthreats: Identifying Shortcomings and Addressing Future Challenges: The Case of The Health Sector in Particular” (2021) 41 Computer Law & Security Review 1.

7 Sumit Ghosh, “Evolutionary History of Critical Infrastructure Protection in the USA” in Sumit Ghosh and Elliot Turrini (eds), Cybercrimes: A Multidisciplinary Analysis (Springer 2010).

8 Kristian Cedervall Lauta, supra, n 6.

9 Bilge Karabacak, Sevgi Ozkan Yildirim and Nazife Baykal, “Regulatory Approaches for Cyber Security of Critical Infrastructures: The Case of Turkey” (2016) 32 Computer Law & Security Review 526.

10 Sumit Ghosh, supra, n 7; Dimitra Markopoulou and Vagelis Papakonstantinou, supra, n 6.

11 Dimitra Markopoulou and Vagelis Papakonstantinou, supra, n 6.

12 European Parliament and the Council, Directive (EU) 2022/2557 of the European Parliament and of the Council of 14 December 2022 on the resilience of critical entities (European Parliament and the Council 2022).

13 Dimitra Markopoulou and Vagelis Papakonstantinou, supra, n 6.

14 Anne Black, Sumantha Gabriel and David Caulfield, “Implementing chimeric antigen receptor T-cell therapy in practice” <https://pharmaceutical-journal.com/article/ld/implementing-chimeric-antigen-receptor-t-cell-therapy-in-practice>; Julia Thornton Snider and others, “The Potential Impact of CAR T-Cell Treatment Delays on Society” (2019) 25 The American Journal of Managed Care 379.

15 Vicki Brower, supra, n 3, p. 1.

16 Meletios-Nikolaos Doulgkeroglou and others, “Automation, Monitoring, and Standardization of Cell Product Manufacturing” (2020) 8 Frontiers in Bioengineering and Biotechnology 1; Fraiser Kansteiner, “Robots, Automation and Pod Factories: How Cell and Gene Therapy Makers Are Catching Production Up to Speed” Fierce Pharma <https://www.fiercepharma.com/manufacturing/robots-automation-and-pod-factories-future-proofing-production-eve-cell-and-gene> accessed March 2023; Panagiota Moutsatsou and others, “Automation in Cell and Gene Therapy Manufacturing: From Past to Future’ (2019) 41 Biotechnology Letters 1245.

17 Ted G. Lewis, Critical Infrastructure Protection in Homeland Security: Defending a Networked Nation (Wiley 2006).

18 Alan T. Murray and Tony H. Grubesic, “Overview of Reliability and Vulnerability in Critical Infrastructure” in Alan T. Murray and Tony H. Grubesic (eds), Critical Infrastructure: Reliability and Vulnerability (Springer 2007).

19 Laurie Anne Schintler and others, “Moving from Protection to Resiliency: A Path to Securing Critical Infrastructure’ in Alan T. Murray and Tony H. Grubesic (eds), Critical Infrastructure: Reliability and Vulnerability (Springer 2007); Arjen Boin and Allan McConnell, “Preparing for Critical Infrastructure Breakdowns: The Limits of Crisis Management and the Need for Resilience” (2007) 15 Journal of Contingencies and Crisis Management 50.

20 Anne van Aaken and Isabelle Wildhaber, “State Liability and Critical Infrastructure: A Comparative and Functional Analysis” (2015) 6 European Journal of Risk Regulation 244; Sumit Ghosh, supra, n 7; Dimitra Markopoulou and Vagelis Papakonstantinou, supra, n 6.

21 www.qualtrics.com

22 R Core Team, “R: a Language and Environment for Statistical Computing” <https://www.R-project.org/>.

23 Kristian Cedervall Lauta, supra, n 7, p. 178.

24 Chen L-K, “Urbanization and Population Aging: Converging Trends Of Demographic Transitions In Modern World” (2022) 101 Archives of Gerontology and Geriatrics 1.

25 Phillips MI, “gene, Stem Cell, and Future Therapies for Orphan Diseases” (2012) 92 Clinical Pharmacology & Therapeutics 182; Vicki Brower, supra, n 3, p. 1.

26 Calne RY, Uin GS and Lee KO, “Stem Cell and Gene Therapies for Diabetes Mellitus” (2010) 6 Nature Reviews Endocrinology 173.

27 Barnstable CJ, Jonas JB and Zhang K, “Regenerative Medicine, Advanced Stem Cell, and Gene Therapies for Eye Diseases” (2022) 11 Asia-Pacific Journal of Ophthalmology 299.

28 Edison Bicudo and Irina Brass, “Institutional and Infrastructure Challenges for Hospitals Producing Advanced Therapies in the UK: The Concept Of ‘Point-Of-Care Manufacturing Readiness’” (2022) 17 Regenerative Medicine 719.

29 Haneen Alosert and others, ’Data Integrity Within the Biopharmaceutical Sector in the Era of Industry 4.0’ (2022) 17 Biotechnology Journal 1, p. 3.

30 Ibid.; Jelena Ochs and others, “Needle to Needle Robot-Assisted Manufacture of Cell Therapy Products” (2022) 7 Bioengineering & Translational Medicine 1.

31 Derek Pendlebury, “Cell & gene therapies: a guide to single-use connections, 10 transferable lessons From the bioprocessing industry” CPC Worldwide <https://www.cpcworldwide.com/Portals/0/Library/Resources/Literature/WhitePapers/Documents/CPC-cell-gene-therapies_white-paper.pdf>; Edwin Stone, “No more sleepwalking: new mindsets for manufacturing cell and gene therapies at commercial scale” BioProcess International <https://bioprocessintl.com/manufacturing/cell-therapies/cell-therapy-as-a-medical-device-new-mindsets-for-manufacturing-cell-and-gene-therapies-at-commercial-scale/> accessed March 2023.

32 Stone, supra, n 31, p. 1.

33 Ibid. p. 1.

34 Derek Pendlebury, supra, n 31.

35 Stone, supra, n 31.

36 Alessandro Cantelli-Forti and others, ’Critical infrastructure protection system design based on SCOUT multitech seCurity system for intercOnnected space control groUnd staTions’ (2021) 32 International Journal of Critical Infrastructure Protection 1; Gülsüm Eksi, Bedir Tekinerdogan and Cagatay Catal, “Software Security Management in Critical Infrastructures: A Systematic Literature Review” (2022) 30 Turkish Journal of Electrical Engineering & Computer Sciences 1142.

37 Suzanne Bakken and others, “Data Science and Informatics Provide Critical Infrastructure for Precision Health” (2018) 67 Nursing Research E35, p. E36.

38 M. Imran Malik and others, ’Developing resilient cyber-physical systems: a review of state-of-the-art malware detection approaches, gaps, and future directions’ (2023) 12 Computers 1.

39 In Appendix 1, the questions marked with a star (*) are those included in this quantitative analysis. The score of each option is also displayed.

40 One-way ANOVA, with p value = 0.008.

41 The test used here was Tukey HSD, with a p value = 0.004 when “software company” and “consulting” are compared.

42 Edison Bicudo, Neuroimaging, software, and communication: the social code of computer code (Palgrave McMillan 2019).

43 Michael Kulik and others, ’Automation in the context of stem cell production: where are we heading with Industry 4.0?’ (2016) 2 Cell & Gene Therapy Insights 499.

44 Panagiota Moutsatsou and others, supra, n 16.

45 David B. Larson and others, “Regulatory Frameworks for Development and Evaluation of Artificial Intelligence-Based Diagnostic Imaging Algorithms: Summary and Recommendations” (2020) 18 Journal of the American College of Radiology 413 415, p. 415.

46 Ibid.; Vegard Antun and others, “On Instabilities of Deep Learning in Image Reconstruction and the Potential Costs of AI” (2020) 117 PNAS 30088; John R. Zech and others, “Variable Generalization Performance of a Deep Learning Model to Detect Pneumonia in Chest Radiographs: A Cross-Sectional Study’ (2018) 15 Plos Medicine 1.

47 Edison Bicudo and Irina Brass, supra, note 28; Edison Bicudo and others, ’The UK’s emerging regulatory framework for point-of-care manufacture: insights from a workshop on advanced therapies’ (2021) 7 Cell & Gene Therapy Insights 1005.

48 Irina Brass and Andrew Mkwashi, “Risk Assessment and Classification of Medical Device Software for The Internet of Medical Things” (STAR-IoT Conference).

49 Dimitra Markopoulou and Vagelis Papakonstantinou, supra, n 7, p. 2.

50 Edison Bicudo and Irina Brass, supra, n 28; Edison Bicudo and others, supra, n 47.

51 Mohammad S. Jalali and Jessica P. Kaiser, “Cybersecurity in Hospitals: A Systematic, Organizational Perspective” (2018) 20 Journal of Medical Internet Research 1

52 Ruchi Sharma, Avinash K. Shrivastava and Hoang Pham, “Software Security Evaluation Using Multilevel Vulnerability Discovery Modeling” (2023) 35 Quality Engineering 341.

53 Saikath Bhattacharya, Munindar P. Singh and Laurie Williams, “Software Security Readiness and Deployment” (2021 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)); Wentao Wang and Nan Niu, “Detecting Software Security Vulnerabilities Via Requirements Dependency Analysis” (2022) 48 IEEE Transactions on Software Engineering 1665.

54 Haneen Alosert and others, supra, note 29; Thomas A. Hemphill, ’Biologics regulation, second-to-market competition, and the use of blockchain technology: an opportunity for the FDA to support responsible biotechnology innovation’ (2020) 7 Journal of Responsible Innovation 689

55 Bayan O. Al-Amri, Hatim Alsuwat and Emad Alsuwat, “Human Factor & Artificial Intelligence: For Future Software Security to be Invincible, a Confronting Comprehensive Survey” (2021) 21 International Journal of Computer Science and Network Security 245; Raghavendra Rao Althar and others, “Automated Risk Management Based Software Security Vulnerabilities Management” (2022) 10 IEEE Access 90597.

56 Bilge Karabacak, Sevgi Ozkan Yildirim and Nazife Baykal, supra, n 10, p. 532.

57 Jelena Ochs and others, supra, n 30; Panagiota Moutsatsou and others, supra, n 15; Michael Kulik and others, supra, p. 43; John Tomtishen, “Automated manufacturing will deliver more – and better – cell therapies” Genetic Engineering and Biotechnology News <https://www.genengnews.com/magazine-issues/automated-manufacturing-will-deliver-more-and-better-cell-therapies/> accessed March 2023; David J. Williams and others, “Precision Manufacturing for Clinical-Quality Regenerative Medicines” (2012) 370 Philosophical Transactions of the Royal Society A 3924.

58 John Tomtishen, supra, n 57, p. 1.

59 Anne van Aaken and Isabelle Wildhaber, supra, n 20.

60 Alissa L. Russ and others, “The Science of Human Factors: Separating Fact from Fiction” (2013) 22 BMJ Quality & Safety 802.

61 Kathrin Cresswell and others, “Key Challenges and Opportunities for Cloud Technology in Health Care: Semistructured Interview Study” (2022) 9 JMIR Human Factors 1, p. 6.

62 Marjolein B. A. van Asselt, Ellen Vos and Isabelle Wildhaber, “Some Reflections on EU Governance of Critical Infrastructure Risks” (2015) 6 European Journal of Risk Regulation 185, p. 190.

63 Associated Press, “Over 30 million U.S. workers will lose their jobs because of AI” MarketWatch <∼https://www.marketwatch.com/story/ai-is-set-to-replace-36-million-us-workers-2019-01-24#::text=The%20report%2C%20published%20Thursday%2C%20says,by%20machines%20using%20current%20technology.>; Mohammad Bashayreh, Fadi N. Sibai and Amer Tabbara, “Artificial Intelligence and Legal Liability: Towards an International Approach of Proportional Liability Based on Risk Sharing” (2021) 30 Information & Communications Technology Law 169.

64 John Tomtishen, supra, n 57, p. 1.

65 Yoonyoung Park and others, “Evaluating Artificial Intelligence in Medicine: Phases of Clinical Research” (2020) 3 JAMA Open 326.

66 Dalip Sethi, “Less than 5 percent human effort: thoughts on the role of automation on cell and gene therapy” The Medicine Maker <https://themedicinemaker.com/manufacture/the-role-of-automation-in-cell-and-gene-therapy> accessed February 2023; Andrew Stone, “Automation is key in cell and gene therapy manufacturing” Reuters Events <https://www.reutersevents.com/pharma/clinical/automation-key-cell-and-gene-therapy-manufacturing> accessed March 2023.

67 John Tomtishen, supra, n 57, p. 1.

68 Bilge Karabacak, Sevgi Ozkan Yildirim and Nazife Baykal, supra, n 10, p. 527.

69 Meletios-Nikolaos Doulgkeroglou and others, supra, n 16; Michael Kulik and others, supra, n 43.

70 Panagiota Moutsatsou and others, supra, n 16, p. 1250.

71 Jelena Ochs and others, supra, n 30; Stone, supra, n 31; David J. Williams and others, supra, n 57.

72 Stone, supra, n 31, p. 1.

73 Robert James Thomas and others, “Cell culture automation and quality engineering: a necessary partnership to develop optimized manufacturing processes for cell-based therapies” (2008) 13 SLAS Technology 152.

74 Anne van Aaken and Isabelle Wildhaber, supra, n 20, p. 245.

75 Bilge Karabacak, Sevgi Ozkan Yildirim and Nazife Baykal, supra, n 10.

76 Derek Pendlebury, supra, n 31.

77 Haneen Alosert and others, supra, n 29.

78 IMDRF Software as a Medical Device (SaMD) Working Group, Software as a medical device (SaMD): application of quality management system (International Medical Device Regulators Forum 2015).

79 European Parliament and the Council, Regulation (EC) No 1394/2007 of the European Parliament and of the Council (2007).

80 European Parliament and the Council, Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts (European Parliament and the Council 2021)

81 We are grateful to an anonymous reviewer who made this crucial point during the peer review process.

82 Edison Bicudo, supra, n 42; Edison Bicudo, Alex Faulkner and Phoebe Li, “Software, Risks, and Liabilities: Ongoing and Emergent Issues in 3D Bioprinting” (2020) 24 Journal of Risk Research 1319.

83 David B. Larson and others, supra, n 45, p. 419.

84 Ibid., p. 419.

85 Ibid.; Megan Beckett and others, “Bridging the Gap Between Basic Science and Clinical Practice: The Role of Organizations in Addressing Clinician Barriers” (2011) 6 Implementation Science 1; Sandra J. Hecker, Christopher Preston and MaryAnn Foote, “Production of High-Quality Marketing Applications: Strategies For Biotechnology Companies Working With Contract Research Organizations” (2003) 9 Biotechnology Annual Review 269.

86 Bilge Karabacak, Sevgi Ozkan Yildirim and Nazife Baykal, supra, n 10, p. 527.

87 Anne van Aaken and Isabelle Wildhaber, supra, n 20, p. 253.

88 European Parliament and the Council, supra, n 6.

89 Stone, supra, n 31, p. 1.

90 Kristian Cedervall Lauta, supra, n 7; Marjolein B. A. van Asselt, Ellen Vos and Isabelle Wildhaber, supra, n 62.

91 Kristian Cedervall Lauta, supra, n 7, p. 183.

92 Marjolein B. A. van Asselt, Ellen Vos and Isabelle Wildhaber, supra, n 62, p. 185.

93 Kristian Cedervall Lauta, supra, note 6; Sheila Jasanoff, “Beyond Calculation: A Democratic Response to Risk” in Andrew Lakoff (ed), Disaster and the Politics of Intervention (Columbia University Press 2010).

Figure 0

Table 1. Summary of qualitative interviews and the online survey

Figure 1

Chart 1. Regions/countries in which research participants are based.

Figure 2

Chart 2. Perception of risk in automated cell and gene therapy manufacture.

Figure 3

Chart 3. Index of survey participants’ optimism regarding digitalisation and automation in cell and gene therapy manufacture.

Figure 4

Chart 4. Concerns about the use of cloud technology in cell and gene therapy manufacture.