Introduction
In today’s highly competitive markets, innovation has emerged as a crucial element for industries, as they can no longer rely solely on quality and quantity. Creative problem-solving skills have been recognized as a fundamental competency for driving innovation (Bertoncelli et al., Reference Bertoncelli, Mayer and Lynass2016; Faria, Reference Faria2019; Nakagawa, Reference Nakagawa2011). The Theory of Inventive Problem Solving (TRIZ) is a structured approach to problem-solving that facilitates innovation. This design methodology provides a systematic framework for generating solutions and designing products (Savransky, Reference Savransky2000). In comparison to other problem-solving techniques such as brainstorming, mind mapping, lateral thinking, Axiomatic Design, Environment-Based Design (EBD), and morphological analysis, TRIZ has been identified as a more potent logic for idea generation (Dubois et al., Reference Dubois, Tate, Conrardy, Nguyen, de Guioa and Gaik-Wad2012; Hernandez et al., Reference Hernandez, Schmidt and Okudan2013; Ilevbare et al., Reference Ilevbare, Probert and Phaal2013; Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022). Advocates of TRIZ believe that it allows for the generation of more innovative ideas in a shorter time frame due to its structured approach (Belski and Belski, Reference Belski and Belski2015; Gronauer and Naehler, Reference Gronauer and Naehler2016; Ilevbare et al., Reference Ilevbare, Probert and Phaal2013; Keong et al., Reference Keong, Yip, Swee, Toh and Tai2017; Mohammadi and Forouzanfar, Reference Mohammadi and Forouzanfar2021; Taskin et al., Reference Taskin, Basoglu and Daim2017). According to Poppe and Gras (Poppe and Gras, Reference Poppe and Gras2002), TRIZ is embraced by Western companies as it enables rapid innovation of products, thus enhancing their competitiveness in the current market landscape.
However, learning TRIZ can be challenging and time-consuming, and its application can be complex, potentially leading to misunderstandings and creating barriers for users (Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022). It has been emphasized that designers must undergo thorough and advanced training to effectively utilize TRIZ (Fiorineschi et al., Reference Fiorineschi, Frillici and Rotini2018; Fiorineschi et al., Reference Fiorineschi, Frillici, Rotini, Conti and Rossi2021). Furthermore, TRIZ is primarily applicable in situations where a problem can be defined by its technical parameters and technical or physical contradictions are apparent. However, identifying these contradictions in the design problems is not always straightforward, especially considering that design problems are inherently ambiguous, particularly in the initial stages (Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022). Fundamentally, while TRIZ excels in idea generation, it lacks the ability to systematically explore design problems and offer designers essential insights before commencing effective idea generation. In contrast, EBD offers a comprehensive methodology for processing design problems, asking the design questions, and acquiring proper design knowledge (Dubois et al., Reference Dubois, Tate, Conrardy, Nguyen, de Guioa and Gaik-Wad2012).
This study seeks to introduce a model that integrates EBD into TRIZ to enhance the methodology. Additionally, we utilized the capabilities of a Large Language Model (LLM), specifically ChatGPT, to create pre-coded functions for each stage of the design process, thereby simplifying the application of the design methodology and increasing productivity and efficiency.
The “Related Work” section reviews existing literature on TRIZ, Environment-Based Design (EBD), and Large Language Models (LLMs). This is followed by the “Integrated Support of TRIZ by EBD and LLM” section, which introduces the proposed model and describes the evaluation methods. The “Test Cases and Results” section presents the outcomes of the evaluation, while the “Discussion” section interprets the findings. Finally, the “Conclusion” section summarizes the study’s contributions and implications.
Related work
TRIZ
TRIZ was conceived by Genrich Altshuller and his colleagues in the former USSR. Altshuller first introduced the principles of more effective thinking in inventive engineering in 1956. In the 1980s, the method evolved into a systematic creativity toolset known as the “Theory of Inventive Problem Solving” (TRIZ) and later expanded into a “General Theory of Powerful Thinking” (OTSM) and “Lifetime Strategy for a Creative Person” (ZhSTL) in the 1990s, among other variations. The main approach, known as the “Algorithm for Inventive Problem Solving” (ARIZ), also underwent further development from 1965 to 1985 (Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022; Savransky, Reference Savransky2000). However, the development of TRIZ did not initially follow the conventional scientific development process, as all potential developments had to be approved by the founder instead of being peer-reviewed (Cascini, Reference Cascini2012; Chechurin, Reference Chechurin2016). Moreover, the constraints imposed by the Cold War era hindered the global dissemination of TRIZ (Fiorineschi et al., Reference Fiorineschi, Frillici and Rotini2018). Following Altshuller’s passing in 1998, the International TRIZ Association (MATRIZ) acquired all rights to TRIZ in 1999. Subsequently, various organizations with TRIZ expertise developed their own versions of TRIZ, such as ITRIZ, TRIZ+, xTRIZ, CreaTRIZ, and OTSM-TRIZ. In 2007, the book “Hands-on Systematic Innovation for Business and Management” was published (Darrell, Reference Darrell2007), detailing the application of TRIZ in business and management. Additionally, new TRIZ tools such as Root Conflict Analysis (RCA+) (Souchkov, Reference Souchkov2005) and Problem Networking, Hybridization (Prushinskiy et al., Reference Prushinskiy, Zainiev and Gerasimov2005) were introduced. Furthermore, several efforts were made to integrate TRIZ with modern Quality Management methods such as Quality Function Deployment (Yamashina et al., Reference Yamashina, Ito and Kawada2002), Six Sigma (Zhao, Reference Zhao2005), and Axiomatic Design (Borgianni and Matt, Reference Borgianni and Matt2015).
TRIZ’s primary approach to addressing design problems involves identifying contradictions within the problem and resolving them. TRIZ categorizes contradictions into three types (Savransky, Reference Savransky2000): administrative, technical, and physical, and asserts that there is a contradiction at the root of every design problem (Petrov, Reference Petrov2019). The theory provides tools such as SU-Field analysis and functional analysis to deconstruct the design problem and examine the relationships among its elements. Subsequently, the designer identifies contradictions within harmful relationships and employs TRIZ tools, such as the 40 principles for idea generation, to address them (Rantanen et al., Reference Rantanen, Conley and Domb2017).
TRIZ as a design method has been experienced in different areas ranging from engineering to management and even biology. However, most applications of TRIZ, according to Scopus research result analysis, have been defined in engineering. The method has been applied to various engineering problems such as launching new products (Stratton and Mann, Reference Stratton and Mann2003), upgrading quality processes (Yamashina et al., Reference Yamashina, Ito and Kawada2002), improving the quality of products (Stratton and Mann, Reference Stratton and Mann2003) and systems (Cavallucci and Weill, Reference Cavallucci and Weill2001), reducing the environmental footprint (Spreafico and Russo, Reference Spreafico and Russo2016), designing plastics (Cascini and Rissone, Reference Cascini and Rissone2004), optimizing energy (Jones et al., Reference Jones, Mann, Harrison and Stanton2001), analyzing patents (Sharma et al., Reference Sharma, Khanna and Bhatnagar2016), and developing software (Kluender, Reference Kluender2011).
TRIZ is also applicable in management concepts such as e-commerce (Domb and Mann, Reference Domb and Mann2001), crowd management (Pin et al., Reference Pin, Haron, Sarmady, Talib and Khader2011), innovation management (Livotov, Reference Livotov2008), and marketing (Zouaoua et al., Reference Zouaoua, Crubleau, Mathieu, Thiéblemont and Richir2010). The theory has also been applied to specific problems in areas including agriculture (Liu and Lu, Reference Liu and Lu2008), biology (Vincent et al., Reference Vincent, Bogatyreva, Pahl, Bogatyrev and Bowyer2005), education (Wits et al., Reference Wits, Vaneker and Souchkov2010), story-writing (Mohammadi and Forouzanfar, Reference Mohammadi and Forouzanfar2021), and drug development (Farber et al., Reference Farber, Martynov and Kleyn2018).
Environment-based design (EBD)
EBD (Zeng, Reference Zeng2015) is a design methodology to help designers in the inventive design process. The development of EBD has gone through a few phases. At first, recursive logic was discovered as a new logic for design in 1991 (Zeng and Cheng, Reference Zeng and Cheng1991), which formulates design’s intricate nature of recursion. The second phase was initiated by defining design in mathematical models where the recursive logic was redefined in set theory and a design-governing equation was developed (Zeng, Reference Zeng2002; Zeng and Gu, Reference Zeng and Gu1999a; Zeng and Gu, Reference Zeng and Gu1999b; Zeng and Jing, Reference Zeng and Jing1996). The third phase was marked by definitions of the design problem, design solution, and design knowledge in the structure of the product environment. The new form integrated design requirements, product functions, product behavior, and design solutions into the structure of the environment (Zeng, Reference Zeng2004, Reference Zeng2015; Zeng and Gu, Reference Zeng and Gu2001). Thus, the design process was mathematically formulated into an environment evolution process (Nguyen and Zeng, Reference Nguyen and Zeng2012; Razavi et al., Reference Razavi, Yang, Yang, Akgunduz, Bhuiyan and Zeng2024; Zeng, Reference Zeng2004, Reference Zeng2015).
In EBD, designers are encouraged to start by analyzing the product environment before delving into the product itself. This approach places a strong emphasis on the product’s surroundings and leverages the environment to refine the final product. In this context, the environment encompasses any existing objects excluding the product (Zeng, Reference Zeng2020). To analyze the environment, the methodology employs a design statement in its natural language form. During this phase, EBD utilizes Recursive Object Model (ROM) diagrams to deconstruct problems and specific question-and-answer strategies to gather information (Wang and Zeng, Reference Wang and Zeng2009; Zeng, Reference Zeng2008; Yang et al, Reference Yang, Dou and Zeng2023). Moving to the next stage, EBD processes the information with the assistance of interaction extraction and graph generation. When it comes to interaction processing, the new problem is redefined in the form of active and reactive conflicts (Gutierrez et al., Reference Gutierrez, Zeng, Sun, Tan, Deng and Zhou2014) and interaction dependence networks (Yang and Zeng, Reference Yang and Zeng2020). To generate ideas, the methodology utilizes two different strategies based on the types of conflicts (Zeng, Reference Zeng2015). While EBD is a reliable and effective method for design problem analysis, its solution generation phase is not particularly robust (Dubois et al., Reference Dubois, Tate, Conrardy, Nguyen, de Guioa and Gaik-Wad2012).
Large language models
Significant strides have been made in the adoption of LLMs for natural language processing tasks. These models undergo thorough training using deep learning techniques on a diverse range of textual big data. The initial model introduced for this purpose was word2vec, capable of performing fundamental tasks such as identifying semantic and syntactic relationships between words. Subsequently, more sophisticated models such as GloVe, ELMo, and BERT were developed. Leveraging advanced transformer architecture and unsupervised learning on large data, OpenAI introduced GPT models, which have produced cutting-edge results in various natural language processing tasks, including language modeling, sentiment analysis, and text completion. Other companies have also launched their specific LLMs: Meta unveiled Llama, Amazon introduced Titan, and Google presented Gemini (Devlin et al., Reference Devlin, Chang, Lee and Toutanova2019; Pennington et al., Reference Pennington, Socher and Manning2014; Radford et al., Reference Radford, Narasimhan, Salimans and Sutskever2018; Wang et al., Reference Wang, Zuo, Cai, Yin, Childs, Sun and Chen2023).
Large language models applications in conceptual design
Numerous efforts have been made to explore the capabilities of the LLMs for the advancement of conceptual design and innovation (Binz and Schulz, Reference Binz and Schulz2023; Dortheimer et al., Reference Dortheimer, Martelaro, Sprecher and Schubert2024; Khanolkar et al., Reference Khanolkar, Vrolijk and Olechowski2023; Zhu and Luo, Reference Zhu and Luo2023). It seems LLMs could possess some level of reasoning that could be applied to the design process (Fang et al., Reference Fang, Deng, Zhang, Shi, Chen, Pechenizkiy and Wang2024). Zhu and Luo (Reference Zhu and Luo2023) argued that the limitations in designer knowledge are the primary source of design fixation, and employing LLMs in the design process could enhance knowledge accessibility, thereby fostering the generation of more innovative and effective ideas. Ma et al. (Reference Ma, Grandi, McComb and Goucher-Lambert2023) used GPT 3 to generate solutions for a series of design problems and compared them with solutions generated by humans in a crowdsourced process. The result showed that the LLM could generate feasible and useful solutions. Zhou et al. (Reference Zhou, Li, Zhang, Yu and Duh2024) studied two groups of designers, one using LLM and the other not, and demonstrated that the LLM-Human group generated acceptable solutions in less time. Furthermore, LLMs have been successfully utilized in understanding customer needs, SWOT analysis, manufacturing planning, and brainstorming (Gomez et al., Reference Gomez, Krus, Panarotto and Isaksson2024; Han and Moghaddam, Reference Han and Moghaddam2021; Hu et al., Reference Hu, Li, Pan, Wen and Bao2023; Just, Reference Just2024).
Moreover, it has been claimed that engaging with LLM through a systematic manner and structured prompts can significantly influence the design process (Tian et al., Reference Tian, Liu, Dai, Nagato and Nakao2024). Several studies have used LLMs in the design process when adhering to specific or structured design methodologies. B. Wang et al. (Reference Wang, Zuo, Cai, Yin, Childs, Sun and Chen2023) harnessed LLM to adhere to function behavior structure (FBS) design principles in order to tackle design challenges. Chen et al. (Reference Chen, Tsang, Jing, Sun, Gray, Ciliotta Chehade, Hekkert, Forlano, Ciuccarelli and Lloyd2024c) proposed an LLM-augmented morphological analysis approach to facilitate the efficient generation of innovative design ideas during the conceptual design phase. They used a questioning strategy, which is similar to the question template proposed for eliciting product requirements (M. Wang and Zeng, Reference Wang and Zeng2009), as well as the FBS model and Kansei Engineering to guide LLMs to assist designers in the design process. Additionally, Schmidt et al. (Reference Schmidt, Elagroudy, Draxler, Kreuter and Welsch2024) discussed the potential of LLMs in enhancing human-centered design. In a similar vein, Koh (Reference Koh2023) exploited the ChatGPT API to leverage the design structure matrix (DSM) to efficiently address design problems. A few studies have also applied TRIZ concepts to LLMs to enhance the design process. Trapp and Warschat (Reference Trapp and Warschat2024) demonstrated that GPT-4 could be used to extract contradictions from patents and provide novel insights into TRIZ principles. Zlotin et al. (Reference Zlotin, Zusman and Hohnjec2023) introduced software that utilizes TRIZ concepts to prompt ChatGPT in resolving design issues. Another study by Jiang and Luo (Reference Jiang and Luo2024) proposed AutoTRIZ, a model capable of identifying contradictions and generating design solutions. However, a notable weakness of current studies integrating TRIZ into LLMs is their sole reliance on TRIZ to address design problems, while TRIZ by itself has weaknesses in processing design problems (Ilevbare et al., Reference Ilevbare, Probert and Phaal2013; Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022).
Moreover, the majority of studies that involve LLMs in the design process share another incapacity – they tend to treat LLMs as a black box. This approach involves providing LLMs with a design problem and expecting them to produce accurate solutions. However, design problems are inherently ambiguous, which makes it challenging for designers to fully grasp the precise nature of the problem in the initial stages of design (Zeng and Gu, Reference Zeng and Gu1999a). Therefore, it is unrealistic to anticipate that a machine will offer an exact solution when presented with an incomplete or ambiguous design problem statement. A more effective approach would involve using LLMs as a tool to aid designers at various stages of the design process.
Integrated support of TRIZ by EBD and LLM
In the subsequent sections, we will delve into the methodologies utilized to devise and evaluate the proposed approach. This will encompass an elaborated explanation of the rationale behind utilizing EBD and TRIZ in designing the functions, as well as the evaluation process employed to demonstrate model efficacy. It is our intention that this information will furnish readers with a thorough comprehension of the approach’s functions as well as its range of capabilities.
Reasoning flow and functions
Design is a sophisticated process that begins with defining design requirements and concludes with product descriptions (Zeng and Gu, Reference Zeng and Gu1999a). Many individuals view the conceptual design phase as a stage for generating solutions. While solutions are the primary output of the conceptual design process, it is essential for designers to not solely focus on solution generation during this phase. In their 2022 study, Yang et al. (Reference Yang, Quan and Zeng2022) discussed the decision-making process in design. This process commences with understanding the correct problem, asking the right questions, collecting pertinent information, developing appropriate knowledge, and ultimately making informed decisions (solutions). Easterday et al. conducted various experiments to establish a logical approach to design. They emphasized that design commences with understanding and redefining the design problem, followed by generating solutions. Zeng and Gu (Reference Zeng and Gu1999a,Reference Zeng and Gub) attempted to apply a science-based approach to study and model the design process. Their study simulated the entire design process and introduced a design-governing equation. This equation clearly outlines that the design process consists of three primary stages: problem redefinition (formulating and processing design problems), synthesis (solution generation), and evaluation (assessing the solutions). In this context, an ideal design process can effectively aid designers in processing and redefining the design problem and, subsequently, generating solutions. Designers can then evaluate the generated solutions against the main requirements, ending the design process if they are satisfied (Zeng and Gu, Reference Zeng and Gu1999a).
Several studies have analyzed the strengths and weaknesses of both TRIZ and EBD. The findings indicate that while TRIZ excels in the synthesis stage (solution generation), it lacks systematic tools to assist designers in problem redefinition (Ilevbare et al., Reference Ilevbare, Probert and Phaal2013; Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022). Conversely, EBD is well-equipped with effective tools for problem processing and knowledge gathering (problem redefinition) but lacks strong ideation tools (Dubois et al., Reference Dubois, Tate, Conrardy, Nguyen, de Guioa and Gaik-Wad2012). Therefore, integrating these two design methodologies could establish a comprehensive design framework. Consequently, in this study, we will utilize the modules related to assisting designers in problem redefinition from EBD, as well as the modules related to design synthesis from TRIZ.
In this framework, following EBD’s environment analysis, the designer first inputs the design problem and receives a set of questions and answers from the LLM. Then, EBD’s principles were supported by employing the LLM to process the answers into the necessary interactions, which leads to a functional analysis based on TRIZ theory according to generated interactions. This is where EBD and TRIZ are integrated. Finally, the designer selects the conflict to be resolved for the LLM to generate ideas based on TRIZ principles to resolve the selected conflict. This corresponds to TRIZ’s solution generation. The entire process is reorganized into six LLM-centered stages, as depicted in Figure 1.
Figure 1. Six stages to integrate EBD, TRIZ, and LLM for resolving design problems.
In the following sections, each stage will be elaborated upon using the example of designing a house that can fly to demonstrate how each stage works. This kind of open-ended problem, which is seen in various daily and creative situations, has been utilized in presenting EBD methodology and remains a grand challenge for TRIZ.
It is worth noting that this approach was entirely coded using the LangChain framework in Python. About 18 prompts with the architecture of a single prompt or sequential chain have been employed, all of which followed LangChain instruction (https://python.langchain.com/v0.1/docs/get_started/introduction/, Retrieved on June 15, 2024). This means that each prompt exploits a system template and a human template, with the option to utilize an AI template if necessary. The full text of the 18 prompts is provided in Appendix 1.
One intriguing aspect of LLMs is the “temperature” parameter. The LLM temperature controls the distribution of probabilities assigned to possible next words in a sequence. At low temperatures, the model tends to produce more deterministic outputs, while at higher temperatures, the outputs become more varied, creative, and potentially less coherent (Nakaishi and Hukushima, Reference Nakaishi and Hukushima2022; Suri et al., Reference Suri, Slater, Ziaee and Nguyen2024). The temperature for all of the LLM’s API through all the functions was adjusted to 0 to ensure consistent and fixed output, except that the temperature for Stage 6, idea generation, was set to 1 to enhance the LLM’s creativity during the process of generating ideas.
Stage 1: Analyzing design problem and generating questions
We input a design problem as a sentence. It is common for a design problem to be vague and unclear, especially in the initial design stages (Zeng and Gu, Reference Zeng and Gu1999a). Therefore, it is important to gather information to clarify the design problem for designers. EBD offers a systematic strategy for generating questions in this regard (M. Wang and Zeng, Reference Wang and Zeng2009; Yang et al., Reference Yang, Quan and Zeng2022; Zeng, Reference Zeng2020). This strategy involves breaking down the design problem statement into nouns and verbs. By asking questions starting with “what” related to the noun phrases and “why,” “how,” “who,” “when,” and “where” related to the verbs, we can gather information to clarify the design problem. Additionally, asking similar questions about the entire design problem statement without dissecting the nouns and verbs can provide further insight. Therefore, in the initial stage, we utilize EBD’s strategy to clarify the design problem and generate relevant questions. Figure 2 illustrates the core internal working strategy of this module.
Figure 2. The internal working strategy of the questioning function.
In Figure 3, we can see a detailed illustration of the stage 1 procedure. The question generation function is comprised of six distinct components. The process begins by analyzing the design problem statement and retrieving nouns using prompt 1, and their corresponding types (human or non-human) using prompt 2. Afterward, if the noun is human, the question “who is + noun?” is generated for that noun. If the noun is non-human, “what is + noun?” is generated. For instance, if the design problem is to “design a house that can fly,” the function would extract all the nouns in this sentence using prompt 1. In this case, the only noun is “house.” Prompt 2 would then classify this noun as non-human. Based on this classification, prompt 3 would use “what” to generate the question, “What is a house?” (See Table 1, Question 1).
Figure 3. Question generation function.
Table 1. Generated questions by the proposed prompt
The subsequent stage of the process entails posing questions based on the verbs. To achieve this, the design problem aided by prompt 4 is dissected into its constituent sentences if it comprises of more than one sentence. Prompt 5 then utilizes the interrogatives why, how, when, where, and who to generate questions for each sentence. For instance, in the example of “Design a house that can fly,” the questions listed in Table 1, from 3 to 7 and 8 to 12, are generated at this stage.
The third component of this function is focused on the semantic interpretation of the verb and the generation of a corresponding inquiry. To achieve this, prompt 6 is employed to extract the verbs from the design problem. Subsequently, prompt 7 is utilized to inquire about the meaning of the verb. As depicted in Table 1, questions 2 and 13 were formulated in this section.
In this stage, so far, we have deconstructed the “flying house” design problem and formulated inquiries based on its nouns and verbs within the context of the problem. The subsequent part entails posing a series of five “wh” questions that pertain to the entire design problem. This step involves questioning the design problem using why, where, when, who, and how. Prompt 8 would execute this part. For instance, some questions, such as questions 14–18 in Table 1, have been produced by applying this part to the sample design problem, designing a house that can fly. This aligns well with the 5WH questioning algorithm proposed by Wang and Zeng (Reference Wang and Zeng2009). Chen et al. (Reference Chen, Jing, Tsang, Wang, Sun and Luo2024a) used the similar 5W1H questioning approach with an example of “flying car” in their LLM effort to elicit product requirements.
The final segment of the function, prompted by prompt 9, evaluates all the generated questions and filters out any duplicates. This is necessary because some generated questions may be redundant, and this step ensures that only unique questions are returned.
Stage 2: Answering questions
During the second stage, it is essential to address the questions generated, which can be divided into two types: those that start with “what” and those that begin with “where,” “when,” “how,” “who,” “why.” While questions that begin with “what” aims to explore the object’s meaning, background, and environment, those that begin with “why,” “when,” “where,” and “how” focus on the specific context of the design problem, such as the location, and time, and specific reasons behind the problem. Typically, questions that begin with “what” can be answered through resources such as the internet, books, or papers, while other questions may require specialized data available at a specific time and location (Zeng, Reference Zeng2020).
Although exploring the design problem situation can help answer “why,” “where,” “when,” and “how” questions, processing a large amount of data to answer “what” questions can be challenging. EBD has a specific template to address “what” questions (Wang and Zeng, Reference Wang and Zeng2009; Zeng, Reference Zeng2020). In this context, EBD recommends first determining the meaning of the object. Next, the object’s lifecycle should be identified, along with all its relationships with other objects throughout its entire lifecycle (the object’s environment). This approach provides a structured way to answer questions beginning with ‘what.’ LLM can play a crucial role in defining an object, identifying its lifecycle, and understanding its interactions with other objects throughout the lifecycle. Following the EBD strategy, a module could be developed to effectively handle these types of questions. Figure 4 provides an illustration of the module’s internal operational strategy at this stage.
Figure 4. Internal strategy of the module.
Figure 5 illustrates the details of the approach employed in this stage. The answering function receives the object name as input and leverages prompt 10 to explore the object’s dictionary definition. Next, with the assistance of prompt 11, the object’s lifecycle is extracted and presented in a JSON output format with the aid of the parsed-out chain feature in LangChain. Then, for each stage, the relevant objects and their interactions with the target object are extracted using prompt 12. To illustrate the generated answers, Figure 6 shows the output of the function in response to the “What is a house?” question.
Figure 5. Answering function.
Figure 6. Answering “What is a house” by the model.
Stage 3: Extracting interactions from the answers
In the previous phase, responses were provided in text form. Now, designers have a wealth of textual information related to the design problem. However, not all of this information is useful in the design process. The question is: how can we analyze the text and extract the important activities occurring in the design problem environment? The systematic approach proposed by EBD involves processing answers. EBD encourages designers to recognize action verbs in the text (verbs that denote an action, such as run, eat, do, warm, etc.) and rephrase each sentence containing an action verb in the form of (subject + action verb + object). This approach condenses the extensive text into the most critical actions taking place in the design problem environment. EBD refers to these reworded sentences as “interactions” (Zeng, Reference Zeng2003, Reference Zeng2020). These interactions could be used for generating performance network or could be used to produce functional analysis since they represent design environment components and their relationship together (Rantanen et al., Reference Rantanen, Conley and Domb2017; Zeng and Gu, Reference Zeng and Gu1999b).
Figure 7 illustrates the details of the procedure used at this stage. All the answers in text format will be input into the function. In the first part, the text will be separated into single sentences using the “.split()” function in Python. Next, each separate sentence will be analyzed. With the help of Prompt 13, the LLM will be triggered to recognize all the action verbs contained in the sentence. If there is no action verb present, the LLM will return with a message indicating that there is no action verb. However, if the sentence does contain an action verb, Prompt 14 will ask the LLM to return the sentence in the format of subject + action verb + object. Prompts 13 and 14 will analyze the sentence in the form of a chain. The output of this function will be a list of interactions extracted from the generated answers. As an example, Table 2 demonstrates the text and extracted interactions for the part of the answers that were generated in the previous stage.
Figure 7. Interaction extraction function.
Table 2. Interaction extraction example
Stage 4: Generating functional analysis based on the extracted interactions
Functional analysis, very similar to the performance network proposed by Zeng and Gu (Reference Zeng and Gu1999b), is a crucial technique used in TRIZ to help designers break down a given scenario and identify any inconsistencies that may be causing contradictions. To conduct a functional analysis, designers need to follow specific steps. First, they should create a comprehensive list of all components in the design environment. Next, they should examine the relationships between each component and others. Components that cause changes or maintenance in other components are known as “Tools,” while the affected components are called “Objects.” These relationships can be categorized as either useful or harmful, depending on the designer’s perspective (Rantanen et al., Reference Rantanen, Conley and Domb2017). Therefore, in the third step, the designer should analyze the relationships and determine which ones are useful or harmful from their perspective. According to TRIZ principles, behind each harmful relationship lies a contradiction (Savransky, Reference Savransky2000). Consequently, each harmful relationship can be targeted for the next stage to identify a contradiction and develop a solution. To simplify the process, the designer can also create a diagram with arrows and related components, helping them determine the type of relationship on the diagram (Rantanen et al., Reference Rantanen, Conley and Domb2017).
In Figure 8, a detailed procedure is presented for developing a module that systematically generates functional analysis at this stage. The input to this function is a text containing a list of all interactions generated in the previous stage. The text would initially be split into separate sentences using the “.split()” function. Then, for each sentence, the LLM would analyze the interaction and extract the tool, object, and the impact of the tool on the object, as well as the impact of the object on the tool, with the assistance of prompt 15. The output of this function is a parsed JSON containing functional analysis components and their relationships. Using this information, a designer can generate functional analysis. Figure 9 demonstrates an example that shows generating functional analysis components for an interaction.
Figure 8. Functional analysis generation.
Figure 9. Functional analysis generation example.
Stage 5: Selecting a contradiction
As previously discussed, the relationship between two components in a functional analysis can be categorized as either harmful or useful. While one designer may deem a relationship as harmful, another designer might view it as useful based on their knowledge, experience, and perspective (Orloff, Reference Orloff2016; Rantanen et al., Reference Rantanen, Conley and Domb2017). When a relationship is labeled as harmful, it indicates that there is a contradiction for the designer to address. In other words, behind every harmful classified interaction lies a contradiction for the designer that must be resolved (Petrov, Reference Petrov2019; Savransky, Reference Savransky2000). During this stage, the LLM should not intervene; instead, it should allow the designer to select the contradictions based on their perspective.
Therefore, in this stage, designers should first select a harmful relation and shape it in the contradiction format defined in TRIZ (Rantanen et al., Reference Rantanen, Conley and Domb2017): “I want (condition A) for element X because (the reason), and I want the opposite of condition A for element X because (the reason).” For instance, we can choose this contradiction, “I want the material of the house to be light to reduce the house weight, and I want the material of the house to be not light because the light material is not sturdy,” as a recognized contradiction in the sample design problem.
Stage 6: Generating ideas for a selected contradiction
TRIZ is a powerful methodology that can be used to generate innovative ideas. The primary tool used in TRIZ for idea generation is a set of 40 principles, which have been extracted from a detailed analysis of thousands of patents (Savransky, Reference Savransky2000). These principles serve as the backbone for idea generation in TRIZ. To initiate the idea-generation process in this stage, an LLM-empowered function is employed to consider the design problem and a selected contradiction. This function then generates ideas related to the TRIZ 40 principles.
To generate ideas for each of the TRIZ principles, prompt 16 is utilized, which runs in a for-loop 40 times. During each iteration, the prompt is provided with the design problem as well as the contradiction and requests LLM to generate ideas for a specific TRIZ principle (from principle 1 to principle 40). Once all the ideas have been generated, prompt 17 eliminates any duplicate ideas. Finally, prompt 18 is used to identify experts who can provide valuable insights to resolve the contradiction. Figure 10 shows the overall architecture of the function. Table 3 provides an overview of the contradiction, some of the generated ideas, and the experts who can assist with resolving the exemplified design problem, “Design a house that can fly.”
Figure 10. Idea generation function.
Table 3. Generated ideas by LLM
Evaluation
The model we have developed comprises several distinct functions. As previously explained, the designer inputs a design problem, and LLM attempts to generate questions. It is important to evaluate the effectiveness of the generated questions. Subsequently, the answers are generated, which are then processed, and interactions are extracted. Therefore, the functionality of interaction extraction should also be measured. These interactions are then input into the next function, where functional analysis components are extracted. Hence, the usefulness of the functional analysis function should also be evaluated. Based on the functional analysis, the designer identifies a contradiction, which is then input into the idea generation function to produce new solutions. Consequently, the effectiveness of these solutions should also be considered. In the following section, we will provide detailed explanations of the metrics used to assess each function.
Question-generation evaluation
In the initial phase of the model, the designer utilizes a question generation function to input the design problem and produce inquiries regarding the design problem. Two metrics can be employed to evaluate the effectiveness of the function: correctness, which measures the degree to which the generated questions can provide the designer with valuable knowledge, and comprehensiveness, which assesses whether the function encompasses all the questions that EBD is designed to address (Koh, Reference Koh2023).
As shown in equation (1), Question Generation Accuracy (QGA) in this stage could be calculated as the number of valid questions generated by the LLM divided by the total number of questions generated. Valid questions are those that provide helpful information for the designer regarding the design problem.
$$ \mathrm{QGA}=\frac{\mathrm{Number}\ \mathrm{of}\ \mathrm{valid}\ \mathrm{questions}\ \mathrm{generated}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}\hskip0.4em }{\mathrm{Number}\ \mathrm{of}\hskip0.4em \mathrm{all}\hskip0.4em \mathrm{auestions}\ \mathrm{generated}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}} $$
As shown in equation (2), Question Generation Comprehensiveness (QGC) could be calculated as the number of valid questions generated by an LLM that align with EBD questions for a specific design problem divided by the number of questions generated by a human EBD expert for that specific design problem.
$$ {\displaystyle \begin{array}{c}\hskip-39.3em \mathrm{QGC}=\\ {}\frac{\mathrm{Number}\ \mathrm{of}\ \mathrm{valid}\ \mathrm{questions}\mathrm{in}\mathrm{s}\ \mathrm{generated}\hskip0.35em \mathrm{by}\hskip0.35em \mathrm{LLM}\hskip0.35em \mathrm{in}\ \mathrm{line}\ \mathrm{with}\hskip0.35em \mathrm{EBD}\hskip0.35em }{\mathrm{Number}\ \mathrm{of}\ \mathrm{generated}\ \mathrm{questions}\hskip0.35em \mathrm{by}\hskip0.35em \mathrm{an}\hskip0.35em \mathrm{EBD}\hskip0.35em \mathrm{expert}}\end{array}} $$
Interaction extraction evaluation
Upon answering the generated questions, we utilize the LLM to apply the EBD concept for processing the responses and extracting the interactions. Two key metrics, Interaction Extraction Accuracy (IEA) and Interaction Extraction Comprehensiveness (IEC), could be employed to assess the outcomes of this process (Koh, Reference Koh2023). In this context, correctness refers to the ratio of valid extracted interactions by LLM to all extracted interactions by LLM, as demonstrated in Equation (3). Valid interactions are defined as sentences containing active verbs. As for comprehensiveness, depicted in Equation (4), it represents the ratio of valid interactions extracted by the LLM from all answers responded to the generated questions for a specific design problem, divided by the number of interactions extracted from those exact answers, albeit this time these interactions were extracted by an EBD expert.
$$ \mathrm{IEA}=\frac{\mathrm{Number}\ \mathrm{of}\ \mathrm{valid}\ \mathrm{interactions}\ \mathrm{extracted}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}\hskip0.4em }{\mathrm{Number}\ \mathrm{of}\hskip0.4em \mathrm{all}\hskip0.4em \mathrm{interactions}\ \mathrm{extraceted}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}} $$
$$ \mathrm{IEC}=\frac{\mathrm{Number}\ \mathrm{of}\ \ \mathrm{valid}\ \mathrm{interactions}\ \mathrm{extracted}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}\hskip0.24em }{\mathrm{Number}\ \mathrm{of}\ \mathrm{interactions}\ \mathrm{extracted}\hskip0.24em \mathrm{by}\hskip0.4em \mathrm{an}\hskip0.4em \mathrm{EBD}\hskip0.4em \mathrm{expert}} $$
Functional analysis generation evaluation
The extracted interactions are inputted into the functional analysis generation function in order to produce the necessary elements for creating a functional analysis. The Functional Analysis Accuracy (FAA) can be assessed using equation (5). Essentially, the accuracy of the function can be defined as the ratio of valid recognized tools and objects extracted from interactions by LLM to all tools and objects extracted by LLM. Valid tools or objects are those that, according to TRIZ, can take a material form, and each tool can impact an object through interaction (Rantanen et al., Reference Rantanen, Conley and Domb2017). Furthermore, Functional Analysis Comprehensiveness (FAC), as indicated in equation (6), can be defined as the ratio of valid recognized tools and objects by the LLM for a specific design problem to the number of recognized tools and objects by a TRIZ expert for that specific design problem (Koh, Reference Koh2023).
$$ \mathrm{FAA}=\frac{\mathrm{Number}\ \mathrm{of}\ \ \mathrm{valid}\ \mathrm{tools}\ \mathrm{and}\ \mathrm{objects}\ \mathrm{extracted}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}\hskip0.4em }{\mathrm{Number}\ \mathrm{of}\hskip0.4em \mathrm{all}\hskip0.24em \mathrm{tools}\ \mathrm{and}\ \mathrm{objects}\ \mathrm{extracted}\hskip0.4em \mathrm{by}\hskip0.4em \mathrm{LLM}} $$
$$ {\displaystyle \begin{array}{c}\hskip-44.12em \mathrm{FAC}=\\ {}\frac{\mathrm{Number}\ \mathrm{of}\ \mathrm{valid}\ \mathrm{tools}\ \mathrm{and}\ \mathrm{objects}\ \mathrm{extracted}\hskip0.22em \mathrm{by}\hskip0.22em \mathrm{LLM}\hskip0.32em \mathrm{from}\ \mathrm{the}\ \mathrm{interactions}}{\mathrm{Number}\ \mathrm{of}\ \mathrm{Valid}\ \mathrm{tools}\ \mathrm{and}\ \mathrm{objects}\ \mathrm{recognized}\hskip0.24em \mathrm{by}\hskip0.4em \mathrm{the}\ \mathrm{TRIZ}\ \mathrm{expert}\hskip0.4em }\end{array}} $$
Idea generation evaluation
In order to assess the effectiveness of the idea generation function of the model, it would be beneficial to consider the metrics outlined in Shah et al. (Reference Shah, Smith and Vargas-Hernandez2003). These metrics can include measuring the novelty, variety, quality, and quantity of the generated ideas.
Novelty
Novelty is a metric that gauges the level of uniqueness or surprise associated with an idea in comparison to other ideas. To assess the novelty of ideas, we begin by breaking down the problem into its key functions. Subsequently, we analyze each idea by determining which functions it fulfills. Finally, we determine the novelty score of each idea by utilizing equation (7) as outlined in Shah et al. (Reference Shah, Smith and Vargas-Hernandez2003).
$$ {S}_J=\frac{T_J-{C}_J}{T_J}\times 10 $$
Where the total number of ideas produced for function j, denoted as Tj, and the count of the current solution for that function is denoted as CJ. The ratio is multiplied by 10 to normalize the expression. The novelty score is then multiplied by the weight assigned to each function. The novelty of the idea is graded by summing the products of all weights and novelty scores (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003).
Variety
Variety serves as a metric for the extent to which the solution space has been explored in the process of generating ideas. The assessment of variety pertains to a collective set of ideas rather than individual ones. To gauge variety, the satisfaction of each function is scrutinized. Ideas are categorized based on their hierarchy and the functions they fulfill and are represented in a tree diagram. Subsequently, the variety score for the set of ideas is computed using equation (8) (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003).
$$ M=\sum_{j=1}^m{f}_j\sum_{k=1}^n\frac{S_k{b}_k}{n} $$
where bK is the number of branches at level k in a tree diagram; SK is the score for level k and n is the total number of ideas, and m is the total number of functions;
Quality
In this context, quality refers to the feasibility of an idea and its adherence to design specifications. To assess the quality of ideas, we referred to Table 4 (Rantanen et al., Reference Rantanen, Conley and Domb2017). Evaluators could use the questions in the table to assess each idea. If the solution does not resolve the contradiction, the idea is rejected and receives an overall score of zero. If the solution does resolve the contradiction, it is evaluated using the other questions. The total quality score for each idea is determined by summing the scores from the answers to each question.
Table 4. Evaluating the quality of every generated idea
Quantity
Quantity refers to the overall number of ideas produced by an individual or a group within a specific timeframe or throughout the entire process of going through all the stages in a particular design method (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003).
Test cases and results
In this segment, we will conduct an assessment of the proposed model by employing the evaluation process and metrics delineated in “Evaluation” section. The model will be utilized to address three design problems previously resolved and published by TRIZ experts. Following the application of the model to these design problems, we will evaluate its efficacy and juxtapose the outcomes with the solutions presented by the TRIZ experts.
Test cases
To select the design problems, we surveyed five TRIZ textbooks (Bukhman, Reference Bukhman2021; Orloff, Reference Orloff2016; Petrov, Reference Petrov2019; Rantanen et al., Reference Rantanen, Conley and Domb2017; Savransky, Reference Savransky2000) and the TRIZ journal website, from which we extracted 11 general design problems that have been already resolved by TRIZ masters and could be used for this study as test cases. Among these, three design problems were found to be solvable through a general LLM without the need for fine-tuning or embedding related papers and books on that specific area. Table 5 provides an overview of the design problems.
Table 5. Design problems
We entered the design problem into the initial function and generated the corresponding questions. Subsequently, questions beginning with “what” or “who” were addressed using the proposed function within this model. Other questions, such as where, why, how, and when, were answered based on the available information in the TRIZ source that published the design problem. The model processed the answers and generated functional analysis based on the interactions. In each case study, we identified the exact contradiction that the TRIZ expert had considered in the source. Consequently, we selected that contradiction and utilized the model to generate ideas. The LLM API used in these three experiments was GPT 3.5, which was chosen for its robust capabilities and cost-effectiveness, making it both accessible and convenient.
Results
In this study, we engaged evaluators to compare the output of our proposed model with the results of TRIZ experts. The evaluators were carefully chosen based on specific criteria, including a comprehensive understanding of design methodologies and tools, practical experience in design projects or publications, and strong critical thinking abilities. The evaluators in this study were selected among graduate students with experience in design projects and publications. All of them possess a deep understanding of design concepts, tools, and methodologies.
Initially, two experts in EBD assessed the model’s capacity to produce questions and extract interactions using the metrics outlined in Section titles “Question-generation evaluation” and “Interaction extraction evaluation”. The findings of this assessment are presented in Table 6, demonstrating a distinct correlation in the assessment results. It is evident that the model was capable of producing questions with high accuracy and comprehensiveness.
Table 6. Question generation and interaction extraction capability of the model
Subsequently, the model’s ability to generate functional analysis was assessed by a TRIZ expert based on the criteria outlined in Section title “Functional analysis generation evaluation”, and the results were documented in Table 7. While the model was capable of generating a comprehensive functional analysis, its accuracy was around 50 percent. In the initial case study, 310 items were recognized in the extracted interactions, comprising 108 unique items and the remaining being duplicates. Of the 108 unique items, 61 were identified as tools or objects, while 47 could not be classified as such according to TRIZ concepts. In the second case study, 252 items were identified, including 128 unique items and their duplicates. Among the 128 items, 76 were categorized as tools or objects, while 52 could not be classified in either tool or object category. In the third case study, 140 items were identified, with 81 unique items and their duplicates. Out of the 81 items, 47 were classified as tools or objects, while 34 could not be classified according to TRIZ. It is important to note that the elements of functional analysis for the first case study were not identifiable in the published version of the report, preventing the calculation of the comprehensiveness rate for the generation of functional analysis in this particular case study.
Table 7. Functional analysis generation assessment
Following the evaluation, two design methodology experts utilized the criteria outlined in Section title “Idea generation evaluation” – novelty, variety, quality, and quantity – to assess the effectiveness of the ideas generated by both the model and the TRIZ experts. Each evaluator followed the instructions for measuring the ideas’ effectiveness according to Shah et al. (Reference Shah, Smith and Vargas-Hernandez2003) and calculated the variety and novelty of the ideas generated by the LLM model and TRIZ experts. It is noteworthy that each evaluator used their perspective to extract the required functions for each design problem. Notably, the model generated a few repetitive ideas in each idea generation for each case study, which were not considered by the evaluators and thus removed from the list of ideas. The variety score, novelty score, and quantity of ideas produced by the model and TRIZ experts are detailed in Table 8. It’s evident that both the variety and quantity of ideas generated by the LLM exceed those generated by the TRIZ expert.
Table 8. Ideas assessment results
Additionally, the two experts evaluated the quality of the ideas based on Section title “Quality”. Figure 11 demonstrates box plot diagrams showing the quality of the ideas generated by the model and those generated by TRIZ experts for the exact same case study. In the third case study, the TRIZ expert proposed two primary ideas to address the design issue, but one was rejected by evaluator 1 for not effectively resolving the contradiction. It is evident that the ideas generated by the LLM vary widely in quality, ranging from very high-quality ideas to low-quality ones.
Figure 11. Assessing the quality of ideas.
Discussion
This study aimed to develop a conceptual design framework that integrates TRIZ and EBD based on an LLM. The goal is to aid designers in effectively addressing design problems. The framework involves selecting and simulating various modules from each design methodology. An important question could be raised here: How and why was each module selected and then tailored together?
The selection and adaptation of modules were based on two main criteria. Firstly, the effectiveness of each design methodology in each tool was taken into account. TRIZ experiments demonstrated its strength in problem redefinition and solution generation (Mohammadi et al., Reference Mohammadi, Yang, Borgianni and Zeng2022), while EBD’s studies showed its proficiency in problem-processing and information-gathering (Dubois et al., Reference Dubois, Tate, Conrardy, Nguyen, de Guioa and Gaik-Wad2012). As a result, tools such as systematic question and answering, suitable for understanding design problems and information gathering, were incorporated from EBD, and tools such as functional analysis and solution generation, suitable for generating solutions, were selected from TRIZ.
The second criterion for selecting modules in this framework was to ensure the integrity of the framework. It was important to have a seamless progression through the stages and modules so that designers could effectively address the design problem by following the prescribed steps. For example, after inputting design problems and generating questions and answers, designers are presented with a large amount of information in text format. In this case, a tool from EBD for interaction extraction could be used to convert the text into important sentences in a simple subject + verb + object format. The subjects and objects could then be utilized as elements for functional analysis in the next stage, while the verb represents the relationship between these two objects. To ensure the effectiveness of the interaction extraction module within the model, all three case studies were also analyzed without its inclusion. The responses to the questions, without any prior processing, were directly input into the module designated for generating functional analysis. Subsequently, the accuracy and comprehensiveness of the generated functional analysis were evaluated according to the metrics outlined in Section title “Functional analysis generation evaluation” by a TRIZ evaluator. Table 9 illustrates the accuracy and comprehensiveness of the functional analysis produced with and without the interaction extraction module. As demonstrated in the table, both the accuracy and comprehensiveness of the functional analysis diminished in the absence of the module.
Table 9. The accuracy and comprehensiveness of the functional analyses produced with and without the interaction extraction module
The functional analysis are intended to be based on tools, objects, and the actions that tools exert on objects. The interaction extraction module effectively filters the text to extract the relevant tools, objects, and actions. Without this module, the functional analysis generation process treats each sentence in the text as a potential candidate for functional analysis generation, which leads to a decrease in accuracy. Furthermore, certain phrases in the text may not initially present any apparent tools, objects, or actions; however, they can be restructured to include these elements and be utilized in functional analysis creation. For instance, the phrase “the weather is hot” may not typically qualify for functional analysis due to the absence of distinct tools, objects, or actions. Yet, it can be rephrased as “the sun heats the air,” thereby introducing specific tools, objects, and actions that can contribute to the functional analysis. The absence of the interaction extraction module results in the exclusion of sentences that could be reformulated into usable constructs for functional analysis, thereby removing numerous potential tools and objects. This, in turn, contributes to the observed reduction in comprehensiveness when the interaction extraction module is not employed.
All the modules in the model were also precisely evaluated through specific criteria. The results show that the employed LLM effectively generated comprehensive and almost accurate questions regarding the design problem. Additionally, the LLM successfully identified and extracted interactions from the generated answers. However, it was observed that the comprehensiveness of both question generation and interaction extraction outweighed the accuracy. It is as if the LLM is sometimes unable to adhere precisely to the prompts. For example, in some cases, the LLM selected “have or is” as active verbs despite the specified prompt stating that “have, has, am, is, are” are not considered active verbs. This discrepancy was particularly prominent in the functional analysis function. Although the prompts regarding identifying tools and objects were well stated, the accuracy of correctly identifying the tools and objects was calculated in 50 percent channel which is inadequate.
However, the EBD knowledge acquisition stages facilitated the model to encompass a significantly larger number of components (tools and objects) relevant to the design problem, in contrast to the solutions proposed by TRIZ experts. This can be attributed to the systematic acquisition of a substantial volume of data pertaining to the design problem through the EBD stages. Considering a greater number of tools and objects in analyzing the design problem has the potential to uncover more contradictions and streamline the development of a more comprehensive design solution.
The application of the model for idea generation proved to be particularly captivating. GPT 3.5 demonstrated a greater variety, novelty, and quantity of generated ideas in comparison to those generated by TRIZ experts. Moreover, the range of quality in the ideas produced by GPT 3.5 was extensive, encompassing both irrelevant ideas with low-quality scores and a few that exceeded the quality of those generated by the TRIZ experts.
As previously noted, Ma et al. (Reference Ma, Grandi, McComb and Goucher-Lambert2023) conducted a study to assess the performance of GPT 3 in the conceptual design process. They employed 12 prompts without a specific methodology or process and tasked the LLM with generating ideas for particular design problems. Their findings indicated that the ideas produced by the LLM were more feasible and useful but less novel and diverse compared to those generated by humans. While our evaluation did not include feasibility and usefulness metrics, we found that the variety and novelty of ideas generated by the LLM surpassed those generated by humans. This could be attributed to the incorporation of design methodologies, their tools, and concepts alongside the LLM in the design process or the use of a more advanced version of the GPT, GPT 3.5. Chen et al. (Reference Chen, Tsang, Jing, Sun, Gray, Ciliotta Chehade, Hekkert, Forlano, Ciuccarelli and Lloyd2024c) utilized LLM to support conceptual design using a specific design approach, morphological analysis. Their findings showed that integrating LLM into the design process could enhance innovation, functionality, and, ultimately, the quality of design ideas, which aligns with the results of our experiments.
In their 2024 study, Jiang and Luo introduced Auto-TRIZ, a design model that uses LLM to automate TRIZ. The model takes problem statements from users as initial inputs and directly generates a solution report by following the TRIZ process. One feature of this model is the automatic generation of contradictions by machines, which occurs without the need for designer supervision. The study’s findings demonstrated that Auto-TRIZ streamlines the use of TRIZ and can generate solutions in a short timeframe. However, the study also acknowledged a limitation: the potential for erroneous information in the solutions generated by the model. In our model, we supplemented TRIZ with EBD, particularly in knowledge gathering. Furthermore, the formulation and selection of contradictions are performed based on the supervision of the designer. The majority of ideas generated by our model were assessed as doable, with some even surpassing the quality of ideas generated by human experts.
Chen et al. (Reference Chen, Song, Ding, Sun, Childs and Zuo2024b) introduced TRIZ-GPT, a model that integrates the traditional TRIZ process to identify design issues, extract TRIZ parameters, address contradictions, and generate solutions. The study revealed that the model is capable of producing solutions similar to those generated by humans, but with a wider range. In our model, we have incorporated three stages prior to utilizing TRIZ to systematically gather knowledge and analyze the design problem environment. As previously mentioned, these stages assist designers in conducting more comprehensive functional analysis and identifying additional contradictions, ultimately leading to the generation of more solutions.
This study is also subject to certain limitations, including a few considered design problems and a limited number of evaluators to assess the solutions. Additionally, not all evaluation metrics may be perfectly aligned with the experiments. For instance, comparing the number of ideas between LLM-generated and TRIZ expert solutions in the textbook could be influenced by bias due to the static nature of the textbook solutions. Moreover, all case studies in this research were exclusively conducted using the GPT3.5 API. A more advanced model could potentially yield results with enhanced accuracy and creativity. Notably, the idea generation phase demands a considerable amount of time as, the LLM meticulously considers each TRIZ principle individually to produce ideas for each one. While this process may require 40 to 50 minutes, it leads to a higher quantity and diversity of generated ideas compared to constraining the model to a few principles. It is plausible that this challenge could be mitigated with a more advanced LLM in the future.
When considering the potential use of more advanced LLMs, an important question also arises: Will this framework still be necessary if future LLMs offer significantly improved performance? It’s crucial to take into account the inherent nature of the design problem when addressing this question. The design problem is often ambiguous and not clearly defined for designers in the early stages of the process. In other words, designers themselves may not have a clear understanding of what they want or what the design problem entails, especially in the initial phases of the design process (Nguyen and Zeng, Reference Nguyen and Zeng2012; Y Zeng and Cheng, Reference Zeng and Cheng1991). Therefore, it may not be practical to expect a highly advanced LLM to provide a definitive solution when the problem itself is not yet fully understood. What is needed is an interactive process between the computer and the designer to gradually clarify the design problem and work toward a solution with the assistance of an LLM. Thus, it seems a systematic design model, like the one presented in this study, is always needed to guide designers in dealing with design problems and generating solutions, even if advanced LLM becomes available in the future.
Our proposed model is specifically crafted to assist designers throughout the design process. It does not operate as a black box, simply churning out solutions to ambiguous design problems. Instead, it actively involves designers at every stage, allowing them to review the output and make revisions as needed. Consequently, designers gain a deeper understanding of the situation. This fosters a high level of interaction between the designer and the LLM. However, this study did not explore the impact of designer interaction with the LLM on design projects. This could be a potential area for future research.
Conclusion
The TRIZ methodology is widely recognized for its effectiveness in generating innovative solutions but is limited in its approach to systematically processing design problems and facilitating knowledge acquisition for designers. Mastering TRIZ can also be time-consuming, intricate, and challenging. In response to these challenges, this study proposes a model that integrates EBD and TRIZ by leveraging LLMs to mitigate these limitations. The findings suggest that the model can receive a design problem and provide pertinent, insightful questions to aid designers in acquiring insight relevant to the problem. Furthermore, the model can process the designer’s responses and help generate functional analysis and ideas to resolve contradictions. Importantly, the proposed model does not function as a black box for problem-solving but rather aims to guide designers through the design process stages and foster insights.
The model’s performance was assessed using the GPT 3.5 API. The results indicate that the model’s comprehensiveness surpasses its accuracy. Evaluation of the generated ideas reveals that machine-generated ideas exhibit greater novelty, diversity, and quantity compared to those generated by TRIZ experts. The quality of the ideas generated by the LLMs spans a wide spectrum, encompassing both irrelevant ideas and ideas of superior quality compared to those produced by TRIZ experts.
It is essential to acknowledge the study’s limitations, including a restricted number of case studies and assessments, as well as the sole reliance on testing the model using the GPT 3.5 API. Furthermore, the model’s usability was not evaluated. Moving forward, the model’s usability will be appraised with novice designers in our future work.
Data availability statement
The program and data is publicly available through the following link: https://github.com/design-zeng/EBD-TRIZ-LLM.
Funding statement
This work was supported by the NSERC Discovery Grant (RGPIN-2019-07048). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests
The authors declare no competing interests.
AI tools declaration
AI tools were not utilized in the paper content generation, with the exception of Grammarly tools, which were employed to rectify grammatical errors and enhance manuscript readability through their AI capabilities.
Appendix 1
Note: Each element enclosed within the curly braces {} denotes a variable. These variables serve as potential inputs for functions or as outputs resulting from another prompt.
Open access
