Testing prototypes with users is essential when developing new products. This qualitative study examines and organizes users’ (children) feedback on physical toy prototypes at various fidelity levels; the data collected comes from a multidisciplinary design studio class tasked with developing wooden toys for the company PlanToys. In this research, we use an organizational framework to categorize stakeholders’ feedback, which includes affirmative feedback, convergent critical feedback, and divergent critical feedback. In our analysis, the children’s feedback was compared by prototype fidelity in terms of both form and function, as well as by the feedback categorization. Findings suggest that form fidelity may bias children toward giving more divergent feedback, that function fidelity may have little impact on feedback given, that children tend to give more types of feedback on lower fidelity than higher fidelity models, and that play value, form, interaction, and function were the highest reported categories of feedback. Based on our observations, we encourage toy designers to be aware of their goals for testing with children before beginning to prototype. This paper is a resource for designers to understand how to prototype for children, as well as resource for researchers studying how specific end users influence the product design process.

Evaluating the algorithmic behavior of interactive systems is complex and time-consuming. Developers increasingly recognize the importance of accountability for their algorithmic creations’ unanticipated behavior and resulting implications. To mitigate this phenomenon, developers not only need to concentrate on the observable inaccuracies that can be measured quantitatively but also the more subjective outcomes that can perpetuate social bias, which are challenging to identify. We require a new approach that involves humans in scrutinizing algorithmic behavior. It leverages a combination of quantitative and qualitative methods to support an ethical, value-aligned design and a system’s lifecycle, informed by users’ perception and values. To date, the literature lacks an agreed-upon framework for such an approach. Consequently, we propose an oversight framework, Modular Oversight Methodology (MOM), which aids developers in assessing the behavior of their systems by involving a carefully crowdsourced society-in-the-loop. The framework facilitates the development and execution of an oversight process and can be tweaked according to the domain and application of use. Through such an oversight process, developers can assess the human perception of the algorithmic behavior under inspection, and extract valuable insights that will aid in assessing its implications. We present the MOM framework, as a first step toward tailoring more robust, domain-specific solutions to exercise human oversight over algorithms, as a means for software developers to keep the generated output of their solutions fair and trustworthy.

The reorientation to remote teaching due to the impact of COVID-19 restrictions proved to be both challenging and compromising, particularly in the context of delivering practice-based design education. Central to the challenges faced by many design tutors was the loss of the design studio as a focal point for engagement and learning. However, delivering teaching remotely through a period of enforced separation also proved that through adversity comes new insights, with the accelerated use of emergent technologies to support distributed working revealing new behaviours and opportunities for learning to take place. In response to COVID-19 restrictions, Miro, the digital whiteboard platform was widely adopted within the UK creative industries and universities alike to facilitate remote engagement. Following a return to campus-based delivery through the Autumn/Fall of 2021, it became evident that some of the pragmatic approaches adopted through necessity had the potential to hold lasting value beyond crisis modes of teaching. This position paper presents a series of reflective studies gathered over three academic years with the aim of (1) understanding the impacts of remote learning as experienced by design students (2) establish clear benefits for the application of online platforms within a blended campus-based delivery and (3) identify emergent characteristics in students’ navigation of the post-COVID design studio.

By making building instructions freely accessible to everyone, open-source machine tools (OSMTs) promise to democratize manufacturing by enabling users in marginalized settings to build machines tools by themselves. There is, however, a lack of empirical evidence of the replicability of OSMT designs in low-resource contexts. This article explores OSMT replicability through qualitative and empirical methods to answer the central research question: Are designs that are fully open source also globally replicable? A comparative experiment was carried out by replicating an open-source 3D printer in two different locations: in Germany (resource-rich) and in Oman (resource-poor). The experiment aimed to determine the barriers faced with the replication in each location. It was significantly more challenging to replicate the 3D printer in Oman, primarily due to difficulties in sourcing and manufacturing, necessitating extensive modifications, which demanded greater skills and dexterity from users compared to those in Germany. Qualitative interviews found that limited digital literacy posed a significant barrier for microenterprise owners in replicating OSMT. Finally, design guidelines were proposed to enhance the global replicability of contextualized OSMT designs.

Mixed Reality enables individuals to visualise and interact with artefacts and environments through a combination of physical and virtual assets. It has received increased interest from the design community as a means to accelerate, enrich and enhance prototyping activities. This article concerns MR’s ability to deceive an individual through the combination of virtual and physical assets and their underlying traits (e.g., mass, size), and a user’s cognitive ability to ‘join the dots’. If properly implemented, MR could save time and resources by reducing the required prototype fidelity and the need to fully realise variants. However, there is a gap in understanding how the traits of physical and virtual assets and cognition combine to form reality. This article presents a study investigated the role mass, virtual and physical model size played on users perception of an MR prototype. The relative impact of these factors was determined by varying these parameters and assessing the user’s perceived change. The key finding from this study was that the virtual model size had a far greater influence on prototype perceived by the user. This suggests that the required physical fidelity of an MR prototype can be lower than the virtual. Furthermore, exploring size design variants can be achieved exclusively through changes to the virtual model.

First-of-a-kind engineered systems often burst with complexity – a major cause of project budget and time overruns. Our particular concern is the structural complexity of nuclear fusion devices, which is determined by the amount and entanglement of components. We seek to understand how this complexity rises during the development phase and how to manage it. This paper formulates a theory around the interplay between a problem-solving design process and an evolving design model. The design process introduces new elements that solve problems but also increase the quantifiable complexity of the model. Those elements may lead to new problems, extending the design process. We capture these causal effects in a hierarchy of problems and introduce two metrics of the impact of design decisions on complexity. By combining and incorporating the Function-Behavior-Structure (FBS) paradigm, we create a new problem-solving method. This method frames formulation, synthesis and analysis activities as transitions from problems to solutions. We demonstrate our method for a nuclear fusion measurement system. Exploring different design trajectories leads to alternative design models with varying degrees of complexity. Furthermore, we visualize the time-evolution of complexity during the design process. Analysis of individual design decisions emphasizes the high impact of early design decisions on the final system complexity.

The evolutionary development of advanced systems (AS) leads to a necessary rethinking of how they can be supported methodically and in terms of processes in product development. Advanced systems engineering (ASE) offers a novel and holistically adaptive approach to facing such challenges in a structured way. However, many of the ASE use cases relate to the development of systems as products, product networks or individual projects. The additional consideration of entire modular product families within AS offers a further decisive advantage for companies, organisations and the people in ASE. By considering modular product families along the entire life cycle in a product family engineering (PFE), the approaches of ASE can bring their impact and potential to additional system levels occurring when considering product families. The systems, which become complex through variety and collaboration, are broken down into their system elements in a structured way and prepared for a common interdisciplinary understanding, as conveyed by ASE. In this paper, the PFE is presented in excerpts using examples of various aspects and points in time of the product’s life as a complementary approach for ASE.

Freehand sketching meets a vital need in design for fluid, fast and flexible visual representations that designers build off of and learn from. Sketching more frequently during the design process correlates with positive design outcomes. Engineering designers receive minimal training on freehand sketching, and engineering students do not apply freehand sketching well during the design process. This study examines some of the underlying factors associated with using sketching more frequently. We examine how sketching skills, spatial visualization skills, sketching instruction and engineering design self-efficacy influence designers’ self-reported sketching behavior. We find that higher sketching skills are associated with using sketching in a variety of ways, and spatial visualization skills and design self-efficacy are associated with sketching more frequently. The relationships uncovered were emphasized by their longevity: spatial skills and sketching skills in students’ first semesters predicted sketching more frequently in a senior capstone design course. These long-lasting relationships suggest the need to invest in students’ spatial skills and sketching skills early in the degree program so that they can be leveraged for better design practice.

This article presents a domain-specific language for writing highly structured multilevel system specifications. The language effectively bridges the gap between requirements engineering and systems architecting by enabling the direct derivation of a dependency graph from the system specifications. The dependency graph allows for the easy manipulation, visualization and analysis of the system architecture, ensuring the consistency among written system specifications and visual system architecture models. The system architecture models provide direct feedback on the completeness of the system specifications. The language and associated tooling has been made publicly available and has been applied in several industrial case studies. In this article, the fundamental concepts and way of working of the language are explained using an illustrative example.

Under the umbrella concepts of upscaling and emerging technology, a wide variety of phenomena related to technology development and deployment in society are examined to meet societal imperatives (e.g., environment, safety, social justice). The design literature does not provide an explicit common theoretical and practical framework to clarify the assessment method to handle “an” upscaling. In this nebulous context, designers are struggling to identify the characteristics to anticipate the consequences of emerging technology upscaling. This article therefore first proposes a structuring framework to analyze the literature in a wide range of industrial sectors (energy, chemistry, building, etc.). This characterization brought to light five prevalent archetypes clarifying the concepts of upscaling and emerging technology. Then, a synthesis of invariants and methodological requirements for designers is proposed to deal with upscaling assessment according to each archetype, based on a literature review of existing design methods. This literature review process showed a disparity in treatment for some archetypes, regarding the industrial sector. A discussion is consequently proposed in the conclusion to guide design practices.

This article outlines a human-centered approach to developing digital patient stories, for sharing their experiences in health care, while preserving patient and others’ privacy. Employing a research-through-design approach, the study proposes a design solution using visualization and digital storytelling to document patients’ and families’ experiences and emotions, as well as their interactions with healthcare professionals in the postnatal unit. By transforming selected observational data into animated stories, this approach has the potential to elicit empathy, stimulate stakeholder engagement, and serve as a practical training tool for clinicians. This work was conducted as part of a broader study that aims to contribute to the existing knowledge base by advancing our understanding of stakeholder needs in birthing facilities and through postpartum discharge. This study primarily focuses on strategies for the development of digital stories and summarizes the factors that contributed to the production of digital stories within the context of sensitive data. It may serve as a valuable resource for students, researchers and practitioners interested in utilizing digital stories to encourage discussions, education and ultimately to enhance systems of health care for respect, equity and support.

To improve understanding of prototyping practice at the fuzzy front end of the design process, this article presents an analysis of a prototyping dataset captured during the IDEA challenge – a 4-day virtually hosted hackathon – using Pro2booth, a web-based prototype capture tool. The dataset comprised 203 prototypes created by four independent teams working in university labs across Europe supported by interviews carried out with each of the teams after the event. The results of the study provide nine key findings about prototyping at hackathons. These include elucidation of the purposes of prototypes in physical, digital and sketch domains and characterisation of teams’ prototyping practices and strategies. The most successful strategy focused on learning about the problem or solution space, often via physical prototypes rather than following more prescriptive ‘theoretical’ methodologies. Recommendations on prototyping strategies in hackathons or similar scenarios are provided, highlighting the importance of practical strategies that prioritise learning and adaptation. The results of this study raise the broader question to the wider research community of how design research and teaching should balance high-level strategic approaches with more hands-on ‘operational’ prototyping.

Design occurs in complex socio-technical contexts with conflicting stakeholder goals, requirements and other constraints. These limit solution options and create trade-offs where improvements relative to one goal come at the expense of performance on another. Little is known about how the design context influences trade-offs, or how designers interact with context to manage trade-offs. This article reports on an exploratory qualitative interview study investigating design trade-offs in relation to their socio-technical context. We identified nine themes reflecting engineering designers’ perceptions of the influence of the design context on their ability to resolve trade-offs. Findings suggest that the design context is both a source of trade-offs, and of knowledge and information that helps designers clarify ambiguous requirements to navigate and resolve trade-offs. The results provide insight into how designers interact with the design context to learn about the structure of their design problems and the degrees of freedom available to resolve trade-offs. The findings also contribute to understanding the effects of path dependencies in trade-off situations, and how the sequential distribution of design decisions over time constrains trade-off resolution. We discuss some of the goals and challenges of conducting rigorous qualitative research in design and identify potential directions for further research.

Design rationale is the justification behind a product component, often captured via written reports and oral presentations. Research shows that the structure and information used to communicate and document rationale significantly influence human behavior. To better understand the influence of design rationale on engineering design, we investigate the information engineers and designers include in design rationales in written reports. Eight hundred and forty-six pages of student engineering design reports from 28 teams representing 116 individuals were analyzed using a mixed-methods approach and compared across project types. The rationales from the reports were coded inductively into concepts and later applied to five industry reports consisting of 218 pages. The findings reveal a spectrum of rationales underpinning design decisions. Grounded in the data, the feature, specification and evidence (FSE) framework emerged as a feature-based and low-effort capture approach. We discuss the need to improve design communication in engineering design, through structuring rationales (i.e., using the proposed FSE framework or other representations) and improving technical writing skills. Lastly, by enhancing design rationale communication and documentation practices, significant benefits can be realized for computational support tools such as automatic rationale extraction or generative approaches.

Importance-performance analysis (IPA) is widely used for needs analysis, product positioning, and strategic planning in product design. Previous research on IPA often employs single-source data such as customer surveys or online reviews with unavoidable subjective bias. In contrast, product maintenance records provide objective information on product quality and failure patterns, which can be cross-validated with customers’ personal experiences from surveys or online reviews. In this paper, we propose an integrated framework for conducting IPA from online reviews and product maintenance records jointly. An attribute-keyword dictionary is first established using keyword extraction and clustering methods. Then, semantic groups, including product attributes and associated descriptions, are extracted using dependency parsing analysis. The sentiment scores of identified product attributes are determined by a voting mechanism using two pre-trained sentiment analysis models. The importance of product attributes in IPA is estimated from the impact of sentiments of each product attribute on product ratings with the extreme gradient boosting (XGBoost) model, while the performance is estimated from the sentiment scores of online reviews or the quality statistics from product maintenance records. In addition, we propose two methods to validate the IPA results, in which the IPA results are compared with the actual product improvements on the market or compared with the analysis of customer reviews from different time periods, respectively. The validated IPA results from online reviews and maintenance records are then integrated to obtain a more comprehensive understanding of customer needs. A case study of passenger vehicles is used to demonstrate the framework. The proposed framework enables automatic data processing and can support companies in making efficient design decisions with more comprehensive perspectives from multisource data.

Any system health work must look at decision-making because decisions propagate throughout a system, shaping system dynamics. Usually, human decision-making is conducted from an individualist, objectivist perspective. What happens when we use an approach based on the radical relationality of Radical Participatory Design and Relational Design? This is the fourth paper in a series of papers which introduced Radical Participatory Design in the first two papers and Relational Design in the third paper. In this fourth paper, we explore the decision-making dynamics in Radical Participatory Design and Relational Design projects.

We use the term political ecology to speak about the power dynamics within any ecological system – a geographical population, a community, an ecosystem, and so forth. We analyze the political ecologies of individualist decision-making models. Then we explore how to embody a relational ontology within decision-filled human ecosystems and how a relational way of being changes decision-making. Referring to biology, we discuss ingredients for relational decision-making – relationality, emergent design principles, and autonomy. Those ingredients can lead to emergent and symbiotic design. Emergent design refers to design that emerges from consistently following a few basic principles. Symbiotic design occurs over time when deeply, relationally embedded entities retain autonomy and indirectly evolve to create a design that would not have occurred through an intentional design process. We then introduce Radical Biocracy as a type of decision-making model where decisions are not deliberated by groups or team members but emerge from the relationally autonomous choices and actions of individuals.

Urban co-creation is an approach to urban design that actively involves stakeholders and end-users in the design process. As designers increasingly use digital tools to manage design information, stakeholders and residents may find it difficult to participate, resulting in a lack of engagement. The emergence of metaverse technologies offers a crucial opportunity to employ user-friendly and collaborative tools, enabling more effective participation. In the study presented in this article, a custom-designed digital game with virtual reality environment was used to facilitate a series of co-creation workshops. The study focused on changes in participants’ experience by comparing baseline and endline survey results against the design outputs. It employed a holistic framework considering four dimensions: game design, participatory experience, learning outcomes and co-creation results. The findings indicate that the digitally gamified approach helped enhance participation and knowledge sharing, and even though game design ratings varied, the use of video games motivated engagement, particularly in an intergenerational context. The co-creation workshop design documented in this article offers new methods to enhance community engagement in urban design. Especially during digital transformation, it opens renewed discussions on balancing traditional output-driven approaches with more participant-centric methods and design objectives.

Growth in the complexity of advanced systems is mirrored by a growth in the number of engineering requirements and related upstream and downstream tasks. These requirements are typically expressed in natural language and require human expertise to manage. Natural language processing (NLP) technology has long been seen as promising to increase requirements engineering (RE) productivity but has yet to demonstrate substantive benefits. The recent addition of large language models (LLMs) to the NLP toolbox is now generating renewed enthusiasm in the hope that it will overcome past shortcomings. This article scrutinizes this claim by reviewing the application of LLMs for engineering requirements tasks. We survey the success of applying LLMs and the scale to which they have been used. We also identify groups of challenges shared across different engineering requirement tasks. These challenges show how this technology has been applied to RE tasks that need reassessment. We finalize by drawing a parallel to other engineering fields with similar challenges and how they have been overcome in the past – and suggest these as future directions to be investigated.

Over the last decades, modeling and simulation have become central methods in engineering design. Today’s technologies enable previously unachievable levels of sophistication and accuracy. However, if decision-makers are unaware of the confidence they can place in models and simulations (M&S), they either fail to leverage their potential by not involving them in processes or make judgments based on unreliable results. Assessments to evaluate M&S exist, but factors that enable decision-makers to have confidence and improve acceptance of using M&S need to be researched in more detail. Therefore, a literature review analyzing design requirements and an online survey to measure factors associated with confidence were conducted. As a result, the survey identified nine predictors of confidence: (1) capability, (2) history, (3) validity, (4) reliability and (5) accessibility of the model. Further, (6) integrity and (7) competence of the modeler, as well as (8) trusting nature and (9) risk awareness of the stakeholder were identified. Having confidence in M&S results significantly increases the reliance on them and leads to better-informed decision-making. Therefore, based on the findings, a framework and an initial application model were developed. The results were initially evaluated and are described.

Generative adversarial networks (GANs) have recently been proposed as a potentially disruptive approach to generative design due to their remarkable ability to generate visually appealing and realistic samples. Yet, we show that the current generator-discriminator architecture inherently limits the ability of GANs as a design concept generation (DCG) tool. Specifically, we conduct a DCG study on a large-scale dataset based on a GAN architecture to advance the understanding of the performance of these generative models in generating novel and diverse samples. Our findings, derived from a series of comprehensive and objective assessments, reveal that while the traditional GAN architecture can generate realistic samples, the generated and style-mixed samples closely resemble the training dataset, exhibiting significantly low creativity. We propose a new generic architecture for DCG with GANs (DCG-GAN) that enables GAN-based generative processes to be guided by geometric conditions and criteria such as novelty, diversity and desirability. We validate the performance of the DCG-GAN model through a rigorous quantitative assessment procedure and an extensive qualitative assessment involving 89 participants. We conclude by providing several future research directions and insights for the engineering design community to realize the untapped potential of GANs for DCG.