a. How should the defectiveness of a product that incorporates artificial intelligence be assessed?

In order for a producer to be exempt from liability, he must first have placed a product on the market and the product must be found to be defective according to the circumstances of Art. 6 of the Proposal for a Directive. In other words, the defectiveness of the product and consequent occurrence of the damage is a precondition for the producer to be exempt from liability if any of the causes foreseen for this purpose occur.

In order to assess the defectiveness of the product, the Proposal for a Directive adopts the criteria of the “consumer expectation test,” which refers to the product safety that the general public is entitled to expect. However, reasonable expectations are not to be assessed individually with regard to a specific person, rather Art 6 Proposal for a Directive refers to the public at large, which seems to lead to a more objective analysis of product defectiveness.Footnote ²⁴ In order to interpret this article, Recital 22 of the Proposal for a Directive establishes some guidelines: (1) Consumer expectations refer to the safety of the product expected by the general public; (2) The safety expected of a product by the general public must be measured objectively; and (3) Safety must be judged taking into account the purpose and properties of the product, its objective characteristics and the specific requirements of the group of users. Recital 22 of the Proposal for a Directive then contains a provision which I consider to be extremely relevant and which serves to introduce the following section: “Some products, such as life-sustaining medical devices, entail an especially high risk of damage to people and therefore give rise to particularly high safety expectations.” On the basis of such a provision, it could be asked whether the higher the implicit risk of the product is, the higher the expectations of the general public are, to the point of not admitting any error on the part of the algorithm or AI system (expert level). Thus, if the risk implicit in the product is lower, the expectations of the general public should not be so high (medium level).

In my view, in the area of products with digital elements and AI systems, the safety expectations of the general public are particularly high, given the risks to safety and life in general. The criterion adopted by the Proposal for a Directive is therefore not new and, as things stand at present, some suggest the adoption of the “risk-utility test” to determine the defectiveness of the design of an AI system, instead of the consumer expectation test adopted in the Proposal for a Directive. According to this criterion, a product is defective when the harm could have been reduced or avoided with an alternative design at a reasonable cost, thus disregarding the reasonable expectations of users, which may be illusory in the case of AI systems. Applying this criterion to an AI system such as that integrated in an autonomous vehicle, there will be a defect in the system if there is an AI system on the market capable of reducing or avoiding the harm at a lower cost. ^{Footnote 25}

If Art. 6 of the Proposal for a Directive is analysed, there are new circumstances for assessing the defectiveness of a product that essentially refer to products with digital elements that incorporate algorithms or AI systems.

b. Cases that could eventually lead to the defence of development risks in the digital age

- Ability for self-learning and the unpredictability of artificial intelligence. First of all, Art 6.c of the Proposal for a Directive refers to the “effect on the product of any ability to continue to learn after deployment.” For its part, the Council proposed replace the reference to “deployment” for “it is placed on the market or put into service”. Focusing, therefore, on the circumstance of Art. 6.c) of the Proposal for a Directive, the first conclusion is that the European legislator has foreseen, as a circumstance revealing the defectiveness of a product, the effect of the self-learning of the AI system.

For a better understanding of how the self-learning capacity of products works, I will give an explanation based on autonomous vehicles. These are made up of two main elements: firstly, the tangible movable good itself and, secondly, the software, which, in turn, consists of AI technology (machine learning) and programming for the execution of rules, tasks and symbol recognition. Furthermore, the AI system integrated in an autonomous vehicle is capable of performing four main functions. Firstly, it observes and analyses its own data (machine learning). Secondly, as a result of prior data analysis, these systems are able to advise the user on the best possible alternative. Thirdly, the system itself can decide between all possible alternatives and, finally, it is able to execute one of them. Therefore, the autonomous vehicle is a product endowed with limited artificial intelligence. The vehicle must make a decision, based on the machine learning it integrates, which evolves with the experience of the vehicle itself and the other autonomous vehicles in the fleet (Art. 6.1.c) Proposal for a Directive).

Due to the fact that the autonomous vehicle’s AI system is based on data that is generated while the vehicle is driven and data that has been generated by other vehicles in the fleet with which it is in contact, it is difficult for a producer to invoke the defence for development risks. Faced with a new and critical situation, the autonomous vehicle will make the decision that is most reasonable according to the experience of all autonomous vehicles. Therefore, as the autonomous vehicle’s decision is based on a data storage system, the predictability of the vehicle’s decision is increased, so that the exemption of the autonomous vehicle manufacturer from liability ^{Footnote 26} is not justified. Consequently, the artificial intelligence of the autonomous vehicle leaves little room for unpredictability and, in any case, unpredictability will be limited by the user’s experience and by the driving conditions. This makes it more difficult for the autonomous vehicle producer to try to exempt itself from liability on the grounds of the defectiveness of the decision taken by the autonomous vehicle, leaving it liable for the damage caused. This is true for any product. I consider that, although the product can make its own decisions, these decisions will be based on the data available to the product and on the basis of which it acts autonomously. Furthermore, another element limiting the unpredictability of the decision taken by the algorithm is the user’s experience and the reasonable and predictable use of the AI-equipped product, as well as its ability to connect to other products with digital elements that can provide it with information on which it bases its decisions. These circumstances therefore limit the unpredictability of the decision taken by the algorithm and limit the unpredictability resulting from the progressive increase in self-learning capability. This means that unpredictability cannot be considered an unknown risk, since even though it is limited, it is an implicit risk in any product that incorporates artificial intelligence.Footnote ²⁷

Likewise, the reasonableness ^{Footnote 28} of the decision taken by the autonomous vehicle to avoid dangerous behaviour, based on the algorithm and machine learning, should not be a ground for invoking the protection afforded by development risks. The defence for development risks is based on the impossibility of recognising a defect in a product due to the state of the art of scientific and technical knowledge at a given time, but not on the reasonableness of the decision taken by a product with artificial intelligence because, as has been explained, its decision is predictable, although not necessarily reasonable.

- Upgrading, modification and acquisition of new features after the product has been released onto the market. Another aspect that must be taken into account is that the Proposal for a Directive continues to envisage that a product cannot be considered defective if it is updated or improved after having been placed on the market, provided that the upgrade or enhancement is performed within the scope of the manufacturer’s control. Recitals 37 and 38 of the Proposal for a Directive refer to this issue, which is also included in Art 6.2 of the Proposal for a Directive. Therefore, if the autonomous vehicle’s software is updated once it has been placed on the market and it is able to improve its functions, the vehicle will not be defective. In other words, it is expected and desirable that the technology will continue to evolve in the future, meaning that, in my opinion, it is extremely difficult for the producer to excuse himself from development risks, given that the product remains under his control and can be updated in the event of new risks presented by digitalisation.

The concept “within the control of the manufacturer” is essential in order to be able to impute liability on the original manufacturer rather than a third party. According to Recital 37 of the Proposal for a Directive, it is considered that “Such software or related services should be considered within the manufacturer’s control where they are supplied by that manufacturer or where that manufacturer authorises them or otherwise influences their supply by a third party.” Therefore, if a third party modifies the product and, as a result, the product causes damage to third parties, liability could be attributed to the person who modified the product, who is considered the manufacturer by virtue of the fact that he modified the product (Art 7.4 Proposal for a Directive), in which case the original manufacturer would be exonerated by the unauthorised intervention of a third party.

Consequently, it is a matter of imputing the damage to the person who has modified the product, rather than claiming exoneration from liability for development risks. In this case, the person who has made the substantial modification to the product can claim the cause of exoneration provided for in Art 10.1.g) Proposal for a Directive, which allows him to impute liability on the original manufacturer if he can prove that the damage is not caused by the modification of the product. On the contrary, if the modification is carried out by the manufacturer himself, who, even after introducing the product on the market, modifies it or introduces improvements while the product is within his control, the product will not be considered defective, and the damage caused after having made the improvement or upgrade cannot be covered by the claim of development risks.

As well as being upgraded and modified after being introduced on the market, the product may have new features as a result of increased autonomy. This would be a qualitative improvement in the product’s performance as a result of the development of its AI system. This possibility is introduced by the Council in Art 6.1c) Proposal for a Directive. In my opinion, this scenario is not too different from that envisaged in Art 6.2 Proposal for a Directive, concerning the upgrading of the product once it has been placed on the market. As long as the product remains within the manufacturer’s control, the manufacturer is in control of the product, so that a qualitative improvement of the product that gives it new features could be equated to a substantial modificationFootnote ²⁹ carried out by the manufacturer himself and, therefore, liability arising from the damage could not be imputed to a third party (Recital 29 of the Proposal for a Directive). The Council’s review of the Proposal for a Directive makes it clear that: “Where a substantial modification is carried out by the original manufacturer, or within its control, and where such a substantial modification makes the product defective, that manufacturer should not be able to avoid liability by arguing that the defect came into being after it originally placed the product on the market or put it into service. […] Since products also allow for modifications through changes to software, including upgrades, the same principles of substantial modification should apply.”

Consequently, the key idea that emerges from the Proposal for a Directive is that in the case of any product upgrade, any modification or any substantial modification that results in new features, if these operations are carried out by the manufacturer himself, he remains responsible for the new risks posed by the product and for any damage it may cause to third parties, meaning he cannot subsequently claim development risks.

- Safety. In order to determine whether a product is defective in terms of safety, Recital 22 adopts the criterion of the reasonable expectations of the public at large, ie the consumer expectations test. According to this test, when a consumer purchases a product, he expects it to have certain functions, characteristics or properties and, furthermore, he does not expect the purchased product to give rise to other inappropriate or undesirable circumstances. This analysis must be carried out objectively and with the general public in mind, thus disregarding the legitimate expectations of any particular consumer. This is not new, as it is the criterion adopted by Art 6 of Directive 85/374/EEC to determine the defectiveness of a product.

The novelty lies in Recital 24 of the Proposal for a Directive, which makes it compulsory to take into account the safety and cybersecurity requirements of products when analysing the defectiveness of the product, a circumstance that is not expressly included in Art 6 of Directive 85/374/EC. This circumstance for determining the defectiveness of the product appears in Article 6.1.f) of the Proposal for a Directive. According to this provision, compliance with the product’s safety and cybersecurity requirements has a bearing on its consideration as a defective product.

We shall now analyse this circumstance for determining whether a product is defective, in the light of the provisions of the new Regulation (EU) 2023/988 of the European Parliament and of the Council of 10 May 2023 on general product safety (GPSR) ^{Footnote 30} and the Proposal for a Regulation on Artificial Intelligence. ^{Footnote 31}

As things stand at present, reference should first be made to Directive 2001/95/EC of the European Parliament and of the Council of 3 December 2001 on general product safety, hereafter the GPSD, which was repealed by the GPSR. ^{Footnote 32} Article 2(b) GPSD contained the definition of a safe product: “safe product” shall mean any product which, under normal or reasonably foreseeable conditions of use including duration and, where applicable, putting into service, installation and maintenance requirements, does not present any risk or only the minimum risks compatible with the product’s use, considered to be acceptable and consistent with a high level of protection for the safety and health of persons, taking into account the following points in particular:

1. (i) the characteristics of the product, including its composition, packaging, instructions for assembly and, where applicable, for installation and maintenance;

2. (ii) the effect on other products, where it is reasonably foreseeable that it will be used with other products;

3. (iii) the presentation of the product, the labelling, any warnings and instructions for its use and disposal and any other indication or information regarding the product;

4. (iv) the categories of consumers at risk when using the product, in particular children and the elderly.”

In view of this definition, there are two concepts for determining that a product is safe. On the one hand, the strict concept of a safe product refers to the fact that the product presents no or only minimal risks. The GPSD contained a definition of “serious risk” which does not clarify what should be understood by risk: “any serious risk, including those the effects of which are not immediate, requiring rapid intervention by the public authorities.” However, recently, Regulation (EU) 2023/988 of the European Parliament and of the Council of 10 May 2023 on general product safety, amending Regulation (EU) No 1025/2012 of the European Parliament and of the Council and Directive (EU) 2020/1828 of the European Parliament and of the Council, and repealing Directive 2001/95/EC of the European Parliament and of the Council and Council Directive 87/357/EEC, ^{Footnote 33} was adopted, which does contain new definitions of the concepts of “safe product,” “risk” and “serious risk”:

• “(2) ‘safe product’ means any product which, under normal or reasonably foreseeable conditions of use, including the actual duration of use, does not present any risk or only the minimum risks compatible with the product’s use, considered acceptable and consistent with a high level of protection of the health and safety of consumers;

• (4) ‘risk’ means the combination of the probability of an occurrence of a hazard causing harm and the degree of severity of that harm;

• (5) ‘serious risk’ means a risk which, based on a risk assessment and taking into account the normal and foreseeable use of the product, is considered to require rapid intervention by the market surveillance authorities, including cases where the effects of the risk are not immediate.”

The new definition of safe product in the GPSR does not introduce major changes compared to the definition of safe product in the GPSD. However, the main changes that do exist are to be found in the definitions of “risk” and “serious risk”. Both concepts hinge on the combination of the probability that a hazard exists or will occur as a consequence of the normal and foreseeable use of the product. Therefore, both the GPSD and the proposed GPSR recognise that non-existent product risk is practically impossible to achieve, and that safety is therefore about tolerating minimal risks.

Furthermore, the broad concept of safe product refers to compliance with minimum requirements for the protection of health and safety. These are general requirements and at a minimum level, in the sense that all products placed on the market must comply with them. Therefore, if a product exceeds the minimum health and safety protection requirements, it can be placed on the market and made available to individuals and consumers.

From all of the above we can draw the conclusion that the fact that a product is safe does not imply that it cannot be defective, either because it has no risk associated with it (a practically impossible assumption) or because, even if it does entail risks, these are minimal and do not prevent the product from exceeding the minimum requirements for the protection of health and safety and it is then found to be defective once on the market. ^{Footnote 34}

The European legislator is aware of the importance that products incorporating AI systems have acquired and how these systems affect product safety in general. However, the GPSR has done away with the idea that products incorporating artificial intelligence do not fall within the scope of the GPSR, which was provided for in the Proposal for a Regulation preceding the GPSR. ^{Footnote 35}

The European legislator therefore wishes to regulate product safety in a manner that is horizontal, uniform and coherent (Recital 6 GPSR), although it is envisaged that the safety of certain products shall be governed by sector-specific rules (Art. 2 GPSR). The GPSR does not contain specific provisions for products incorporating artificial intelligence and, therefore, as such products are not excluded from its scope, they should also be understood to be covered. Even so, one cannot overlook the fact that the Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts, ^{Footnote 36} which may have an impact on safety issues for products integrating artificial intelligence, is currently in the process of being finalised.

According to the definition of “artificial intelligence system” included in the Artificial Intelligence Act, this can be defined as “‘AI system’ means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments;” Artificial intelligence can generate a wide range of economic and social benefits, but it can also generate risks and harm certain public interests (Recitals 3 and 4 of the Artificial Intelligence Act). There is a definition of the concept of “risk” in the Artificial Intelligence Act, which is referred to the: “combination of the probability of an occurrence of harm and the severity of that harm.” At no point is there any reference to development risks applicable to artificial intelligence.

The focus of the Artificial Intelligence Act is on high-risk AI systems, which are those that qualify as such under Annex II to the Artificial Intelligence Act and, for these systems, a risk management system will be implemented which will consist of a continuous iterative process to be run throughout the lifecycle of a high-risk AI system and which will identify “known and foreseeable risks” associated with each high-risk AI system, estimate and evaluate the risks that may emerge when the high-risk AI system is used as intended and under conditions of reasonably foreseeable misuse, and evaluate other risks that may arise from the analysis of data gathered through the post-market monitoring system.

The Artificial Intelligence Act therefore measures the risks of AI systems in the present (known risks) and in the future (foreseeable risks and risks that may arise). This fits poorly with the concept of development risks, which looks at the future state of science and technology to assess whether, when a product was introduced on the market, the state of scientific and technical knowledge existing at that time made it possible to detect the product’s defect. The same can be said of the understanding of risk in the GPSD and the GPSR.

- Cybersecurity. According to Art. 6.1.f) of the Proposal for a Directive, the defectiveness of a product must also be assessed on the basis of compliance with the minimum cybersecurity requirements (Recital 23 of the Proposal for a Directive). In this regard, it is worth highlighting the Proposal for a Regulation of the European Parliament and of the Council on horizontal cybersecurity requirements for products with digital elements, ^{Footnote 37} the so-called Cyber Resilience Act. ^{Footnote 38} The recitals of this Proposal for a Regulation refer to the need to improve the functioning of the internal market by establishing essential cybersecurity requirements for placing products with digital elements on the market, so that these products are put into circulation with fewer vulnerabilities and thus increased security (Recitals 1 and 2 Proposal for a Regulation on cybersecurity). Furthermore, the minimum cybersecurity requirements for digital products set out in the Proposal for a Regulation constitute a horizontal regulatory framework applicable to all digital products (Recitals 4 and 6 of the Proposal for a Regulation on cybersecurity). Therefore, the Proposal for a Regulation on cybersecurity has a general scope of application and the applicable sectoral legislation should be taken into account in order to complete the horizontal regulatory framework that it envisages. ^{Footnote 39} Indeed, Art 3 of the Proposal for a Regulation on cybersecurity provides that it will not apply to products with digital elements in certain areas, including vehicles.

As regards the cybersecurity risks of digital products, Art 3.36) of the Proposal for a Regulation on cybersecurity contains the definition of “significant cybersecurity risk”: “a cybersecurity risk which, based on its technical characteristics, can be assumed to have a high likelihood of an incident that could lead to a severe negative impact, including by causing considerable material or non-material loss or disruption.” Again, the essence of the definition hinges on the more or less high probability that the product with digital elements will suffer an accident leading to material or immaterial damage.

The placing on the market of products with digital elements shall be authorised when these products comply with the essential requirements set out in Annex I of the Proposal for a Regulation, which envisage that products with digital elements shall be designed, developed and produced to ensure an adequate level of cybersecurity in relation to the risks; be delivered without known vulnerabilities that could be exploited by a malicious third party; be delivered with a secure by default configuration; ensure protection against unauthorised access through appropriate control mechanisms, and protect the confidentiality of personal or other processed data through state-of-the art mechanisms.

As regards AI systems that are classified as high-risk under the Proposal for a Regulation on Artificial Intelligence, Art 8 of the Proposal for a Regulation on cybersecurity states that the essential requirements set out in Annex I are applicable to such systems and, therefore, compliance with these requirements will determine their compliance with the cybersecurity requirements set out in Art 15 of the Proposal for a Regulation on Artificial Intelligence.

As mentioned above, it is Annex I of the Proposal for a Regulation on cybersecurity that sets out the essential cybersecurity requirements for products with digital elements. Without attempting an exhaustive examination of these essential requirements, the provision contained in the second paragraph stands out, according to which: “products with digital elements shall: (aa) be placed on the market without any known exploitable vulnerabilities.” Footnote ⁴⁰ The Proposal for a Regulation on cybersecurity does not define what should be understood by “known exploitable vulnerability,” so the future regulation governing this matter should ideally provide a definition of this concept. ^{Footnote 41} However, a definition of “vulnerability” is found in Article 3(38), which refers to the definition of “vulnerability” in Article 6(15) of the so-called NIS2 Directive: “weakness, susceptibility or flaw of ICT products or ICT services that can be exploited by a cyber threat.” ^{Footnote 42} Therefore, that the concept of vulnerability includes the concept of weakness, which refers to the lack of properties that make the product robust against third party intrusions. In conclusion, a product will meet the essential cybersecurity requirements if it is delivered without any known vulnerability that could be exploited. Otherwise, the product will be defective because it does not meet the essential cybersecurity requirements.

At this point, the question that arises is the following: can a vulnerability or defect in the cybersecurity of a product discovered after the product was introduced on the market, and the consequent occurrence of damage associated with this vulnerability, exempt the producer from liability for development risks? The considerations set out in the previous points regarding general product safety can be extrapolated to the field of cybersecurity in order to affirm that, even if a product with digital elements meets the essential requirements in terms of cybersecurity, this does not prevent it from being considered defective if it causes damage to third parties after it has been placed on the market. Thus, a product can become defective after it has been placed on the market because of an unknown or actively exploited vulnerability related to lack of cybersecurity, even though its cybersecurity was certified at the time because it complied with the essential cybersecurity requirements. However, given the known vulnerabilities of products with digital elements, it is difficult to think of a vulnerability that could not have been detected or foreseen.

In addition, as regards vulnerabilities, the NIS2 Directive mandates the European Union Agency for Cyber Security (ENISA) to develop and maintain a European vulnerability database.Footnote ⁴³ This database should focus on the field of product cybersecurity, in order for manufacturers to identify the essential risks related to the properties of products with digital elements at the time they are introduced on the market and thereafter (Recital 32 Proposal for a Regulation on cybersecurity). In other words, manufacturers must ensure that the essential cybersecurity requirements of their products are met throughout their entire lifecycle. ^{Footnote 44} Therefore, to the extent that the product, despite having been placed on the market, remains under the producer’s control and considering that producers will have at their disposal the European database developed by ENISA, it prevents producers from exempting themselves from liability by claiming development risks, in this case, for vulnerabilities that were not known at the time they placed the product on the market and that have been detected subsequently.

Recitals 38 and 41 of the Proposal for a Directive deals with the manufacturer’s liability for the vulnerabilities of its products. In order to ensure consumer protection, economic operators are prevented from avoiding their liability by proving that a defect occurred after they placed the product on the market or put it into service when the defectiveness of the product consists in the exploitation of a vulnerability by a third party, whether known or unknown. The European legislator’s intention is therefore to prevent vulnerabilities in the cybersecurity of a product with digital elements from being used as a pretext for avoiding liability for damage caused. This position can be interpreted in the sense that the manufacturer is liable for product defects deriving from an undiscovered risk (the vulnerability). In other words, it is not the case here of an unknown risk, such as the lesser or greater degree of unpredictability of the AI system as a consequence of its self-learning capacity, but rather that the risk had not been discovered at the time the product was introduced into the market.Footnote ⁴⁵