1 Introduction
Cars are becoming increasingly computerized and intelligent, and aspects of driving can already be assisted by the car itself, such as adjustment of speed, lane keeping and pedestrian safety. A notable global race to produce the first autonomous vehicle can be seen among car manufacturers and non-automotive stakeholders alike. The race is motivated by opportunities provided by technology maturity and the many benefits of self-driving vehicles, such as improved safety, reduced congestion, lower emissions, higher productivity, and increased driver convenience (Rupp and King Reference Rupp and King2010; Davila and Nombela Reference Davila and Nombela2012; Verberne et al. Reference Verberne, Ham and Midden2012). To gain full advantage of these benefits, the technological developments such as improved computer vision, decision making, computing power must be addressed (Liu et al. Reference Liu, Li, Tang, Wu and Gaudiot2017). Equally as critical are the ways vehicles will fit into the social aspects of traffic and car use (Vinkhuyzen and Cefkin Reference Vinkhuyzen and Cefkin2016; Brown and Laurier Reference Brown and Laurier2017).
In traffic, the vehicle and driver operate in a landscape full of social interactions with other road users, both other drivers and vulnerable road users. Socially accepted autonomous driving can be accomplished only when the vehicle can interact with other road users smoothly. This places demands on autonomous vehicles’ ability to communicate intentions and flexibly to follow social rules (Risto et al. Reference Risto, Emmenegger, Vinkhuyzen, Cefkin and Hollan2017). Car use also includes additional dimensions of social relationships. A new social relationship between vehicle and driver, or user, will emerge as the vehicle gains increasing agency through automation. In order to be able to share control of driving, that relationship needs to be clearly established and the vehicle and user must be supported in communicating their respective intentions and goals for the trip at hand (Bruemmer et al. Reference Bruemmer, Gertman and Nielsen2007; Flemisch et al. Reference Flemisch, Heesen, Hesse, Kelsch, Schieben and Beller2012). This places novel demands on the interaction design of the vehicle (Pettersson and Ju Reference Pettersson and Ju2017). Furthermore, as shared services for autonomous vehicles are emerging as an opportunity (Fagnant and Kockelman Reference Fagnant and Kockelman2014), and the focus on interactions with primarily the driver as opposed to all passengers is questioned (Pettersson and Karlsson Reference Pettersson and Karlsson2015), the co-experience (Battarbee & Koskinen Reference Battarbee and Koskinen2005) of autonomous vehicles becomes an important aspect of vehicle use. For all three types of social relationships, understanding how people are to socially interact with autonomous vehicles is critical for their future use and adoption (cf. Rogers Reference Rogers1995), but little is currently known. Autonomous vehicles have to be designed to fit into an existing social context, at the same time as they will trigger shifts and new social aspects that are relevant to design for will emerge.
From our perspective, the social interactions outside, inside, and with an autonomous vehicle are at the heart of a field of practice that is currently subject to fundamental change. There is a need to explore and exemplify directions that autonomous vehicle technology can take in terms of social experience, and to probe into possible futures of socializing with and within autonomous vehicles. We strongly believe that social aspects need to be considered as an integrated part of autonomous vehicle development, from the earliest stages of development. However, at the moment, it is not clear how social aspects should be addressed in the development of vehicles, nor which conceptual direction interaction design should take in supporting social aspects. To be able to develop such a direction, designers of the interaction with autonomous vehicles are required to envision new futures and analyze problems that do not yet exist, which is a complex task to undertake. Social aspects also present a challenge in that they are hard to explore theoretically. Thus, there is a need for methods that can encompass these issues, exploring the future situation before the technology is in place.
1.1 Aim
The aim of this paper is to contribute an exploration of methods for addressing the social interaction between future autonomous vehicles and humans. The methods are explored in three key social areas: the vehicle as a social entity in traffic, co-experience within the vehicle, and the user–vehicle relationship. The research approach at this stage is broad and exploratory, serving as a first step into the problem area.
The objective is to provide an illustrative example of method use and to form recommendations for which types of methods to use in the early design phases of future technology, along with how they should be used. Our findings should allow researchers and designers to get an initial overview of the design space of social aspects of autonomous vehicles, and guidance on how to methodologically incorporate social aspects in an efficient and effective way in the early design phases.
2 Addressing the future and the social
To be useful in an exploration of how people are to socially interact with autonomous vehicles, we see that methods need to enable designers to address two facets of the problem: envisioning future technology and technology use – ‘what could be’ – and capturing critical aspects of different social experiences in relation to autonomous vehicles. The following two sections present the background for each of these facets.
2.1 Suggesting ‘what could be’
As mentioned, the technology that will fully enable autonomous driving is still in the future, which requires developers to envision this future in order to be able to design for it. Design always deals with an imagined future and design methods afford the opportunity to conduct research to understand the current state, and then to synthesize, sketch and prototype to identify an improved future state that is a desired outcome (Zimmerman and Forlizzi Reference Zimmerman, Forlizzi, Kellogg and Olsen2014). Reframing the problem and identifying the ultimate solution from an infinite set of possibilities is a challenging part of the work. It is much too easy to design future products, services, and systems that people and society will never accept; a key challenge is problem selection and solution framing (Dubberly et al. Reference Dubberly, Evenson and Robinson2008; Dorst Reference Dorst2011; Nelson and Stolterman Reference Nelson and Stolterman2012). The user-centered interaction design process implies that a design team starts by observing the situation of interest, understanding the current state, then making new things as a reflection of how the current state might be improved (Dubberly et al. Reference Dubberly, Evenson and Robinson2008). Critical in this process is moving from research findings to conception of a preferred future state. Dubberly et al. (Reference Dubberly, Evenson and Robinson2008) characterized this shift as the analysis–synthesis bridge, and developed a model to concisely describe four activities separated by three transition points (Figure 1).
Figure 1. The analysis–synthesis bridge model, adapted from Dubberly, Evenson, and Robinson, Reference Dubberly, Evenson and Robinson2008.
The model describes how interaction designers work in a typical design process. Designers first collect concrete data about the world (lower-left quadrant). They transition to the upper left quadrant by analyzing their data and producing models that connect many interim findings together. This produces an abstract description of the current state that highlights both problems and opportunities to improve the future through redesign. Designers then progress to the upper-right quadrant by selecting a particular solution as an improved future state. Finally, designers transition to the lower-right quadrant by generating concepts that seek to achieve this preferred future state. Here, designs are iteratively prototyped. Dubberly et al. (Reference Dubberly, Evenson and Robinson2008) note that many designers skip the upper-right quadrant and transition from models of the present to concrete design concepts. When the upper-right quadrant of exploring the design space is neglected, valuable efforts of early exploration of concepts, that can save time in the later costly and effortful building of high-fidelity prototypes, are lost. Early efforts can question assumptions and promote disruptive ideas to emerge, rather than building on incremental changes of the existing (Verganti Reference Verganti2008).
In regard to autonomous vehicles, the novel and complex technology to enable these vehicles is at an early stage. This makes the leap between what can be investigated now, to understand users and requirements, to what could and ought to be designed as a future preferred state, very large. Design methods that allow a design team to ideate and explore interaction and social experience may help to close the design gap (Buchenau and Fulton-Suri Reference Buchenau and Fulton-Suri2000; Davidoff et al. Reference Davidoff, Lee, Dey and Zimmerman2007; Pettersson and Karlsson Reference Pettersson and Karlsson2015). Methods that focus on the upper-right quadrant can provide necessary exploration of novel ideas in the development of potential design solutions, using simple prototypes of potential solutions that function as tools for discovery to probe the interaction between people and technology and to test the impact of new technological solutions in the context of human use in new envisioned contexts (Woods and Christoffersen Reference Woods and Christoffersen2002). From the applications of these methods in a design process, the outputs of design not only feed new product development in industry, but can also develop new research ideas (Woods and Christoffersen Reference Woods and Christoffersen2002; Fallman Reference Fallman2008; Zimmerman and Forlizzi Reference Zimmerman, Forlizzi, Kellogg and Olsen2014).
Methods must therefore support envisioning the future, as it could be, in a way that is suitable for early stages of the design process. Methods in this area are important tools for bridging the gap between analysis and synthesis, but are yet largely unexplored for considering the social interactions in connection to autonomous vehicles. The methods need to concretely address the future design situation, and suggest a model of what ‘could be’. This is especially important for groundbreaking, novel technology, typically found to disrupt current assumptions of use.
2.2 Capturing social aspects
As identified in the introduction, there are several challenges connected to social experience with autonomous vehicles. Broadly, these cover three areas, each comprising design challenges of slightly different character; the social aspects of the vehicle as a social entity in traffic, the co-experience within using an autonomous vehicle, and finally the social aspects of the driver–vehicle relationship.
2.2.1 Addressing the vehicle as a social entity in traffic
Today, to help traffic move along smoothly, drivers explicitly communicate with other car drivers and unprotected road users by multiple means, including gestures, posture and eye contact, and by using the car itself, for example, by slowing down and flashing the headlights (Rakotonirainy et al. Reference Rakotonirainy, Schroeter and Soro2014). Similarly, other road users interpret these signs as aspects of the drivers intentions and awareness, such as the eye contact established between driver and pedestrian when crossing the road (Sucha Reference Sucha2014; Lundgren et al. Reference Lundgren, Lagström, Habibovic, Andersson, Nilsson, Sirkka, Fagerlönn, Edgren, Fredriksson, Krupenia, Saluäär and Larsson2016). However, with the increasing level of automation in vehicles, drivers will no longer be able to represent the vehicle’s actions, and will instead be engaging in tasks such as reading, socializing or sleeping. Nevertheless, the vehicle will still need to cooperate with other cars and unprotected road users, and act as a social entity in traffic without relying on today’s driver-centric cues for communication. Thus, in the design of the communication between automated vehicles and other road users, it is crucial that the car conveys its awareness and intentions, and that it is possible to negotiate with (Vinkhuyzen and Cefkin Reference Vinkhuyzen and Cefkin2016; Eden et al. Reference Eden, Nanchen, Ramseyer and Evquoz2017). Initial studies of real-world interactions, using so-called ghost drivers and analysis of real-world data have revealed that the absence of clear indications of the vehicle’s intent and the vehicles lacking ability to interpret and react to the social norms in traffic are met with mistrust and frustration (Rothenbücher et al. Reference Rothenbücher, Li, Sirkin, Mok and Ju2016; Brown and Laurier Reference Brown and Laurier2017). The complex interaction between autonomous vehicles and other road users is still in need of further exploration, and methods need to be able to aid designers in imagining situations where these subtle social negotiations between drivers and other road users will appear in the future.
2.2.2 Addressing co-experience within the vehicle
As mentioned, automation frees up the driver to take part in non-driving activities and experiences (Kun et al. Reference Kun, Boll and Schmidt2016). This also opens up for a move from today’s strictly forward-facing and driver-centric information in-car environment. The interior may become more like a living room, increasingly used for social interactions, productivity and relaxation, creating opportunities for new interior designs, but also questioning the focus on the ‘driver’ in the design of autonomous vehicles (Ive et al. Reference Ive, Sirkin, Miller, Li and Ju2015; Pettersson and Karlsson Reference Pettersson and Karlsson2015). The joint experience and use of an interactive system, or the co-experience of use (Battarbee & Koskinen Reference Battarbee and Koskinen2005), is an issue so far not sufficiently addressed in the research of autonomous vehicles. Methods addressing co-use and co-experience should explore possible and desirable social situations. To do so, methods will need to capture use by multiple persons, including how to create a meaningful flow of information between several persons and the vehicle and come to a mutual understanding of what should happen. This may require designing new types of interactions, for new types of users and addressing emergent issues such as negotiation and privacy.
2.2.3 Addressing the user–vehicle relationship
There is an additional social concern that comes with increasing automation: the relationship forged between an intelligent vehicle and the occasional, or former, driver. The needs for communication and collaboration will change and vary depending on how control and driving tasks are shared between driver and vehicle. This means that content, delivery and detail of the communication will differ along with the experience of use. For autonomous vehicles to be adopted it is central to find the appropriate relationship between vehicle and user in all imaginable scenarios (Flemisch et al. Reference Flemisch, Heesen, Hesse, Kelsch, Schieben and Beller2012). Creating a design that is trusted and that is transparent about the user’s role appears to be important aspects of creating a viable relationship between vehicle and user (Banks & Stanton Reference Banks and Stanton2016). Trust has so far been the main adoption variable researched. While creating trust is important, it presents challenges as it decreases with increasing automation (Rödel et al. Reference Rödel, Stadler, Meschtscherjakov and Tscheligi2014), and can turn into overtrust when the human’s trust exceeds the vehicle’s ability (Lee and See Reference Lee and See2004). Thus, it is important that the vehicle communicates both its capabilities and limitations to the driver (Bruemmer et al. Reference Bruemmer, Gertman and Nielsen2007), and enables the formation of ‘co-driving’ practices (Brown and Laurier Reference Brown and Laurier2017). Similar to the communication outside of the vehicle, the vehicle will need to communicate to the user at the level of intentions, awareness and goals, and be open for, as well as clear in its demand for, negotiation about the actions it should take. Methods will need to support the exploration of co-driving practices, and help designers challenge the strongly held assumptions about the current driver–vehicle relationship, where the driver is always in control. Methods should also help to establish a two-way communication of intentions and adaptation; it is not only a matter of informing the user, the human will need to adapt the strategy to the automated system just as much as the system will need to adapt its strategy to the user (Flemisch et al. Reference Flemisch, Kelsch, Löper, Schieben, Schindler and Heesen2008).
3 Study approach and methods explored
We selected a set of design methods for exploration in this study. The choice was made based on how design methods might capture social dimensions and envision the future, that is, their suggested potential to aid the design of future autonomous vehicles capable of supporting social interaction. We hope to provide an open exploration of methods and to provide a first step into a future design solution space. We explored these methods to understand their ability to question assumptions and evolve thinking about the future autonomous vehicle’s interactions with users, between users, and with other road users.
A workshop format with researchers and practitioners linked to the fields of autonomous vehicle development and interaction design was chosen as the approach for the study, as it provided a way to gain broad insights into the topic from multiple perspectives at the forefront of development. The workshop aimed to assess how well the selected design methods might probe the social experience of autonomous vehicles at early design stages. We specifically chose not to make an exact or controlled comparison between the techniques, as the approach was exploratory. No users were involved at this stage, partly to scale down on the complexity of the activity and partly as it is yet unclear who those users would be. The focus of the activity was the span of automated vehicles where driving can be partly to fully autonomously conducted by the car, i.e., an autonomous level of 3–5 in the SAE definition of driving automation (SAE, 2014).
Table 1. Workshop themes and the methodological exploratory approaches used in them
3.1 Methods for each area of social experience
The workshop covered the three identified themes within social interaction with vehicles, each theme using different methods (see Table 1). The methods have different epistemological roots and are meant to reveal different angles of the social aspects of autonomous vehicles, as briefly described below. All of the methods include both ideation as well as an engagement in, and analysis of, the suggested future, bringing out qualities in social communication, such as how information is requested and exchanged between agents and how a common understanding of current states and intentions emerge between cars and users. In addition to these specific methods, each theme also included elements of creation and making, in order to push participants to transfer their observations and abstract ideas into actual tangible representations; in other words, bridging the gap between Dubberly et al. (Reference Dubberly, Evenson and Robinson2008) upper quadrants; moving from interpretation to suggesting.
3.1.1 Methods for capturing the vehicle as a social entity in traffic
For addressing the issue of social interactions with other road users, in particular pedestrians and cyclists, two techniques were used: small-scale scenarios and the Wizard of Oz method. Small-scale scenarios, or table-top scenarios (Broberg et al. Reference Broberg, Andersen and Seim2011), have been employed in participatory simulation of complex scenarios, for example to overview physical layouts and spatial conditions to be examined in design (Steinfeld Reference Steinfeld2004; Watkins et al. Reference Watkins, Myers and Villasante2008). The value of lo-fidelity visualizations of the interactions of multiple stakeholders in workplace design has been explored in for example ship interaction design (Österman, Berlin and Bligård Reference Österman, Berlin and Bligård2016) and hospital design (Nyholm Andersen and Broberg Reference Nyholm Andersen and Broberg2015). The method offers a low-cost, flexible way of approaching early design stages scenarios and to propose design suggestions. Small-scale scenarios provide a ‘helicopter’ overview of a complex context with different stakeholders with differing needs (Ruohomäki Reference Ruohomäki2003; Nyholm Andersen and Broberg Reference Nyholm Andersen and Broberg2015; Österman et al. Reference Österman, Berlin and Bligård2016). The Wizard of Oz (WOz) methodology is a well-established rapid approach for prototyping and evaluating user interfaces (Landauer Reference Landauer1987; Wilson and Rosenberg Reference Wilson, Rosenberg and Helander1988). A human Wizard simulates the system’s intelligence and interacts with the user through a real or mock interface, meaning that a contingency in the interaction between system and users is allowed. The method is rooted in a pragmatic interaction design approach of lo-fidelity prototyping. In the automotive field, it has been used in driving simulator explorations of autonomous vehicle interactions (e.g. Baltodano et al. Reference Baltodano, Sibi, Martelaro, Gowda and Ju2015; Mok et al. Reference Mok, Sirkin, Sibi, Miller and Ju2015) as well as on-road explorations of pedestrian reactions (Rothenbücher et al. Reference Rothenbücher, Li, Sirkin, Mok and Ju2016) and interaction (Habibovic et al. Reference Habibovic, Andersson, Nilsson, Malmsten Lundgren and Nilsson2016; Charisi et al. Reference Charisi, Habibovic, Andersson, Li and Evers2017).
3.1.2 Methods for capturing co-experience within the vehicle
For researching the social interactions and co-experience between users of the vehicle, enactment was chosen as a method. Enactment of future technology has been used previously as a rapid means of jointly improvising new ideas as well as critically probing into future use in an evaluation-like setting (Arvola & Artman Reference Arvola and Artman2007; Odom et al. Reference Odom, Zimmerman, Davidoff, Forlizzi, Dey and Lee2012), also for groups of users (Jorlöv et al. Reference Jorlöv, Bohman and Larsson2017). By the use of drama or enactment, the future use situation can be evoked (Brandt and Grunnet Reference Brandt and Grunnet2000). A number of similarly founded techniques have been developed over the range of almost 30 years, where aspects of drama and the use of the body in imagining new interactions with technology has been a common theme, such as in bodystorming (Burns et al. Reference Burns, Dishman, Verplank and Lassiter1994; Buchenau and Fulton-Suri Reference Buchenau and Fulton-Suri2000), speed dating (Davidoff et al. Reference Davidoff, Lee, Dey and Zimmerman2007) and in the overarching area of embodied design ideation (Wilde et al. Reference Wilde, Vallgårda and Tomico2017). The roots reside in phenomenology and the lived bodily experience (Macaulay et al. Reference Macaulay, Jacucci, O’Neill, Kankaineen and Simpson2006; Marshall et al. Reference Marshall, Antle, Van Den Hoven and Rogers2013; Wilde et al. Reference Wilde, Vallgårda and Tomico2017), where bodily engagement is critical to elicit the knowledge that is not easily addressable by words.
3.1.3 Methods for capturing the user–vehicle relationship
For the social interaction between user and vehicle, design metaphors were chosen as a methodological base. Design metaphors are used to frame design problems, create meaningful product experiences and guide interaction (Hey et al. Reference Hey, Linsey, Agogino and Wood2008; Cila Reference Cila2013). By linking new ideas to well- understood objects and processes metaphors can shape perceptions and actions without the user noticing them (Lakoff and Johnson Reference Lakoff and Johnson2003; Bruemmer et al. Reference Bruemmer, Gertman and Nielsen2007). The use of design metaphors in autonomous vehicle design was suggested by Flemisch et al. (Reference Flemisch, Adams, Conway, Goodrich, Palmer and Schutte2003) as a way to create a uniting vision for the design team, as well as to help the user create an initial mental model of the system. Metaphors can help to give concrete properties to abstract ideas (like that of the character of a relationship), and should help clarify the division of control and responsibility, communicate intentions and goals, and set the tone of relationship (Bruemmer et al. Reference Bruemmer, Gertman and Nielsen2007). Different metaphors for the relationship between user and autonomous vehicle have been suggested but no consensus has been reached. The design metaphors were represented using ideation cards. According to Lucero et al. (Reference Lucero, Dalsgaard, Halskov, Buur, Markopoulos, Martens, Malins, Coninx and Liapis2016), cards serve three purposes for design teams: they support collaboration by being shared objects for discussion, objects that can be moved, positioned and annotated; they trigger combinational creativity by making it easy to put concept together; and they serve as tangible idea containers around which arguments can be anchored. Together metaphors and cards should help challenge assumptions about relationships with cars, and serve as tangible representations of possible futures.
3.2 Analysis
As stated earlier, the aim of the study was to explore methods for imagining, investigating and ideating future social situations with autonomous vehicles. To analyze whether the methods were successful, we examined the design concepts and research directions identified for each theme of the workshop, along with the discussions taking place during the activities themselves. The participants’ assessments of their experience with the methods during the workshop were also taken into account. The data were analyzed using a qualitative, comparative lens, based on observation notes, videos, the participants workshop outputs, and the concluding joint discussions on concepts and methods held at the end of the workshop. In the analysis, the methods were assessed in their ability to support the participants in envisioning the future and their ability to capture social aspects (cf. Section 2). The outcomes from the different themes were compared to find common traits in the methods that were successful in addressing both the future and the social, as well as commonalities in their implementation which contributed to their level of success. Based on this analysis, we tried to formulate recommendations for which methodological directions are relevant to pursue further.
4 Setup of the workshop
The one-day workshop was initiated, arranged and realized by a multidisciplinary team of researchers from academia and industry and was held at the NordiCHI’16 conference (Pettersson et al. Reference Pettersson, Rydström, Strömberg, Hylving, Andersson, Klingegård and Karlsson2016). On the day of the workshop, 18 researchers and industry practitioners gathered to participate. Most participants had previous experience of autonomous vehicle development or research, with a few exceptions. The day started with a joint session where an invited industry practitioner from Volvo Cars presented the company’s view on autonomous vehicle and their research approach. This was followed by a presentation on relevant ongoing research within the area by the workshop organizers.
Following the initial presentations, participants gathered for the main workshop in predetermined groups, each group addressing one workshop theme. Each group received more targeted information and specific instructions for their task (see Sections 4.1–4.3). On a general level, all three groups had the same task: to explore potential future social situations in highly automated vehicles and design the interaction necessary to support a chosen scenario. They followed a similar two-part structure comprising an exploratory analytical part followed by a synthesis part. In the analytical part, the participants were encouraged to playfully interact and learn about the problem space. Here, the workshop was arranged so that the participants tested different methodological ideas on how to encourage exploration, empathic connections to the users, and address the analysis–synthesis bridge (see Table 1). These methodological ideas are further described below. The analytical part was followed by synthesis, i.e. an ideation exercise where the participants were asked to create an interaction concept for a specific theme based on their understanding of the problem. Participants were encouraged to consider all aspects of the vehicle as a possible interaction surface, including its behavior and connected services. Participants were provided with material to sketch and build prototypes, such as cardboard, various colored papers, modeling clay, pens, pipe cleaners, scissors, and glue. The day ended with a joint presentation and evaluation of the outcomes, where each concept was presented by the groups and discussed in plenum, followed by a joint reflective discussion regarding the value of the methods and future work.
4.1 Theme A: the vehicle as a social entity in traffic
At the beginning, a brief introduction was given regarding the challenge of how communication with other road users should take place in the future when the person in the driver’s seat is not necessarily active in driving the vehicle. The design task given to the participants was to explore how vehicle intent can be communicated with other road users in the future, with the goal of creating new alternatives to those already proposed. To encourage insights and discussions of how autonomous vehicles are experienced, a field exercise was arranged where the participants could observe pedestrians (other participants in the workshop) and interact with a seemingly autonomous car (see Figure 2). The vehicle was set up using a Wizard of Oz approach in a real-world setting. The actual driver and steering wheel in a right-hand driven car were covered by an additional seat making the car seem driverless (Rothenbücher et al. Reference Rothenbücher, Li, Sirkin, Mok and Ju2016). The participants were invited to interact with the autonomous vehicle, to stimulate design reflections. The vehicle was driven around in a secluded block with sparse traffic, and the participants, acting in the role of pedestrians, were asked to interpret the intent of the vehicle and make decisions accordingly to cross the road in its presence. This was purposefully set up to trigger the participants with practical experience in the problem space, and aid the creation of new models of what could be in the upper-right quadrant of the analysis–synthesis bridge model in Figure 1.
The WOz approach was combined with the creation of small-scale scenarios where props (including a traffic mat, toy model die-cast cars, in addition to the material provided to all participants) and rapid prototyping techniques were used to represent traffic scenarios. The miniature representation was used to stage relevant use cases and develop ideas for further elicitation in the work group and finally as a presentation to all workshop participants.
4.2 Theme B: co-experience within the vehicle
The design task in this theme concentrated on exploring social situations in daily car use where the vehicle could play a supportive role, and then ideating design solutions that would facilitate co-experience and co-use in a selected situation. To initiate the work, a short pairwise interviewing session was held about personal daily commuting, including positive and negative social experiences, and design ideas derived from this. This was done to create awareness among the workshop participants about the social aspects of daily commuting and the co-experience of using vehicles. After the pairwise discussion, a joint discussion and ideation of design ideas was held. The discussion was gradually moved into a mock-up of a car, outlined on the floor with four seats, making use of the ‘setting the stage for self-driving cars method’ (Pettersson and Karlsson Reference Pettersson and Karlsson2015). The setup with an outline of a car served as a stage for enacting, conceptualizing and envisioning new social situations and facilitators. Props and discussion triggers for a future moving living room were available, such as personal possessions and clothing representing different types of users (e.g. children, teenagers, grown-ups, elderly), as well as prototyping material such as clay, paper, cardboard and pens. The participants were free to choose a scenario to their own liking, using their initial discussions as a springboard for ideation.
4.3 Theme C: the user–vehicle relationship
After a short presentation of the changing preconditions for interaction when vehicles gain agency, participants were first asked to explore which user–vehicle relationship they found desirable for the future and which variables needed to be considered in the exploration (e.g. hierarchy, trust, character). The theme utilized design metaphors to trigger the exploration of future human–vehicle interaction. Participants were presented with a selection of 10 currently existing relationships that could act as metaphors for the future desired relationship that should be established between the vehicle and the human occupant. The metaphors were collected from relevant literature, and included, for example, the relationship between a horse and rider (Flemisch et al. Reference Flemisch, Adams, Conway, Goodrich, Palmer and Schutte2003), and players on a sport team (Davidsson and Alm Reference Davidsson and Alm2009), together with a few others. Each metaphor was briefly described on a card and illustrated with a photo to form what is commonly known as design cards (see Figure 5). After the initial exploration, participants were asked to choose one metaphor and create an interaction based on it, considering the full range of interaction possibilities that a car offers and could offer in the future. Prototyping material was available to concretize the design and build a testable version of the interaction.
4.4 Participants
In total, 18 academic and industrial researchers participated in the research activity. The group consisted of 12 senior researchers, two PhD students, two master students and two research assistants. They covered different disciplines including human–computer interaction design, human factors, ethnography, human-centered design, robotics, engineering psychology and communication systems. Most of the participants were European. The participants were recruited via the conference website, and sent in applications to attend the workshop. To be accepted, participants described their view and experience of the social aspects of interaction with, and within, autonomous vehicles and indicated which of the three subthemes they were most interested in. The participants were then assigned by the organizers to a theme to ensure a good mix of disciplines and different levels of experience in each group.
5 Results
This section presents a description of the outcomes for each of the three themes when using methods addressing the social aspects of the interaction. For each of the themes, the character of discussion and the role that the methods played is first described. This is followed by a description of the concept the team developed and the implications for design and research they identified based on the workshop.
5.1 Theme A: the vehicle as a social entity in traffic
Seemingly, as a consequence of the tangible WOz field experience, the participants started out with the need to be user-centered in the design approach. The first-hand experience of interaction with a seemingly autonomous car led the participants to quickly engage in detailed discussions of the challenges with autonomous vehicle–pedestrian interaction. These discussions covered different considerations of communication in order to achieve effective interaction, including communication content, language, modality, and direction, as well as traffic, situations, attention demand and stimuli outside the car. The participants concluded that information should be presented in a clear, unambiguous manner, immediately comprehensible in context without training. Subsequently, the mode of communication was recognized as important. Visual communication was seen as an obvious answer, but its exact implementation needed further exploration. Finally, the direction and nature of communication was discussed in terms of whether the car should offer suggestions to traffic around it, simply announce its own intentions, like the WOz car they experienced, or also be open to accepting communication from surrounding traffic and interact with such information accordingly. The discussions culminated in the question ‘How can road users and the vehicle occupant come to a mutual understanding of the cars situational awareness?’
The further exploration of this question was aided by the use of the small-scale scenario setup. The miniature traffic mat and scale models of cars helped the participants create a representation of a complex traffic interaction (Figure 3). Pedestrians and cyclists made from modeling clay added to the exploration of scalability in the communication between an autonomous vehicle and surrounding traffic. Keeping in alignment with design methodologies like rapid prototyping and quick-and-dirty lo-fidelity prototyping, the participants used mobile phone cameras to gather pictures and video footage from the vehicle and vulnerable road user point of view. Iteration was fast-paced and tangible, helping to arrive at the final design concept.
Together, the Wizard of Oz and small-scale scenario methods were fruitful in providing a good collaboration basis for the team, which was needed to address social aspects of autonomous vehicles, as they gained a united understanding of the problem and scenario at hand, as well as a hands-on scene for exploring design solutions.
Figure 2. Workshop participants interact with a seemingly driverless and autonomous WOz-vehicle.
Figure 3. Miniature scenario enabling a shift in perspective through a ‘helicopter view’ of the traffic situation.
5.1.1 Proposed concept
In order to keep all involved parties informed of the car’s perception of its surroundings, the group’s final concept designed communication for occupants of the vehicle as well as for bystanders. The solution was a projection of the car’s understanding of the situation on the windscreen for the external and internal users.
A video feed captured in front of the car in its direction of motion is scanned for potential road users. The captured feed is then displayed to the vehicle occupant either on a display with the recognized road users highlighted in real time. This allows the vehicle occupant to be aware of whom the vehicle has recognized. The captured real-time feed is also shown to external traffic, after noise and background features of the scene have been removed. To attain maximum clarity and minimize ambiguity, only the road users identified as obstacles by the car is projected onto the vehicles windscreen. The windscreen acts in a manner similar to a mirror, and road users can ascertain whether the car has noticed them or not depending on whether their reflection is visible in the mirror. The foundation upon which this concept was envisioned was the idea to externally represent the car’s situational awareness by mirroring its knowledge, and thereby instilling a sense of safety to the surrounding road users.
In the evaluative discussion, after the concept was presented, the other workshop participants praised the two-way communication and multiple user perspective, but brought up worries about how it would handle crowds, the reliance on the pedestrian looking at the car, privacy and recording issues, as well as concerns for agents not directly in front of the windscreen such as dogs and children.
5.1.2 Identified research needs and design implications
While the proposed concept offers a way to show what the vehicle sees in traffic, it does not communicate what it is going to do about it. For it to be a plausible solution that can negotiate with road users, the concept will need to indicate the vehicles intention as well. Additionally, it could be necessary to identify and acknowledge gestures or instructions by other road users in order to create a two-way, fully social experience. A further design issue is how the communication should cater to people with functional variations, especially visual impairment.
Even though the windscreen has been suggested as a viable design space in autonomous vehicles in prior literature (Haeuslschmid et al. Reference Haeuslschmid, Pfleging and Alt2016), it remains to be seen if using it as the canvas for visual communication is ideal, or causes unwarranted side effects. Furthermore, there is room for this solution to be used not only in the front of the vehicle, but extended to a 360 projection of the car’s awareness. In such a scenario, other design spaces may need to be considered. Scalability also remains an important factor; while identifying and displaying a handful of traffic participants within the limited display space is possible, unambiguous communication of a multitude of pedestrians, cyclists, or other cars within the car’s viewing trajectory also poses an interesting design challenge. Furthermore, computation power to process such information-heavy video feed real time and project it on screen also poses a potential challenge for the overall design.
5.2 Theme B: co-experience within the autonomous vehicle
The discussions first circled around the personal daily commute and aspects of this. Interestingly, there was a common need to be unsocial during daily commuting rather than social, and there were several examples of practices for refraining from social interactions during commuting. Family settings for commutes however set much more demands on social interactions and the co-experience of technology use inside the car. The discussion moved on to the broader possible effects from autonomous driving on social interactions within society, touching on issues such as personal data security and safety. As the group entered the pretend car as a stage for enactment, a concrete scenario and following design concept began to emerge where group interactions with, and within, the vehicle became the focus, rather than the more general reflections on a personal or societal level that flavored the initial parts of the activity. Rather than one dominating perspective as in the initial group discussions, the efforts now encompassed a wider scope of viewpoints and ideas.
The enactment method in combination with props worked as a strong enticement to become more specific in design ideas, over the very broad and unprecise ideas taking place in the prior group discussions. Enactment allowed a focus on multiple users’ interactions and experiences, bringing up important topics of trust and joint value when co-using the car. However, the scenario enacted was extensive, and could have been more focused to better detail the co-experience and co-interaction with the autonomous car. The activity thus had to constantly balance between the overly broad (such as discussions on societal levels) and becoming more specific on co-experiencing the in-vehicle space.
5.2.1 Proposed concept
The concept described a car-sharing service, that also had an array of additional connected functionalities. The concept presented the idea of sharing ride by a mobility service, with others attending the same activity. The service included a functionality for checking the profiles of the suggested co-riders, allowing the users to make an assessment of safety and the ability to accept/decline the suggestion. When accepting the co-riders and inviting them into the car, the user(s) also had the possibility of using a panic button for separating them in the car and consequently stopping the ride and leaving the passengers behind. When not in use, the car could suggest that it could be active for charity driving, and was thus allowed to drive others while the booking continued. In summary, the concept consisted of three modes:
(i) Chauffeur Mode: ordering a mobility service to destination of choice.
(ii) Wingman Mode: mobility as well as social service connecting users with similar mobility patterns/interests/activities – The car can thus drive several users to the same destination.
(iii) Samaritan Mode: providing the ability to offer others to use rented car, for example persons with limited mobility.
The car could suggest activities and actions by way of voice (see Figure 4 where the voice of the car is acting from the background). The main interaction modality was thus voice-based interactions available for all the passengers, as well as a small screen available for example for the panic button functionality, available for only the initial users of the car. It was expected that voice interactions can serve as a facilitator for complex interactions with services and a more ‘intelligent’ interface than cars of today. The other groups questioned the role of automation in this concept, as it could theoretically be done today, but with a human driver. The social connections (Wingman and Samaritan modes) would however require an algorithm to keep track of suitable connections.
Figure 4. The chauffeur mode on the way to a night out. The vehicle interaction was mediated by the voice assistant hiding behind the curtain.
5.2.2 Identified research needs and design implications
When designing for shared services and co-experiences, the need for safety and trust increases. It is a challenge to design for trust and security when sharing such an enclosed space. In the workshop’s design concept, it was approached by the ability to assess profiles of users and having access to panic function. While we see a panic button as an extreme solution, the design must address the possible drawbacks of sharing services. In that regard, vehicle design can learn from existing systems and carefully adopt previously successful functionality and behavior from other services with social elements, i.e. multiuser mobile applications for sharing, dating et cetera. Co-experience should be further studied in autonomous vehicle research to find relevant aspects when vehicles are used by a number of users, in social settings. This could be accomplished using further methods previously applied for co-experience such as co-discovery (Lim et al. Reference Lim, Ward and Benbasat1997) and a mix of open, naturalistic methods (Battarbee & Koskinen Reference Battarbee and Koskinen2005).
In the group discussions, an identified research need was to increase efforts in more multidisciplinary research and collaboration. This was recognized as digital, social and material aspects of autonomous cars need to be connected, in order to create successful solutions that can fit in well in people’s daily lives. Another identified research need was on what an autonomous vehicle can do on a societal level, raised by the Samaritan mode included in the concept. This represents a new direction for the private car’s role in mobility and indicates how design can be used as an enabler for more socially and environmentally sustainable behavior.
5.3 Theme C: the user–vehicle relationship
The discussion started to broadly approach the topic, covering a wide range of aspects related to autonomous driving and human–vehicle interaction, such as safety, liability, personalization, the learning curve and which activities would be done in the vehicle. The safety perspective came to dominate the discussion throughout the session. There was also a touch of more philosophical discussion on what autonomy meant as a concept in transport, and whether we could learn anything by exploring our own autonomy. This broad and abstract approach led to some difficulties in getting into the more specific design of the interaction, and the group needed to be prompted to return to the main task of the workshop.
In the parts of the discussion that centered on the interaction, the hierarchy of the interaction was very much in focus. This was in part triggered by the metaphors and the different levels of hierarchy that they covered, as well as the safety implications of who really was in charge. The discussion partially tried to flesh out which agents and intelligences were present in the system in order to try to understand which parties needed to be social with each other, and thus understand whom to apply the metaphor to. Is the user interacting with an intelligent vehicle, or interacting with an intelligent agent controlling a normal car, or even an intelligent agent controlling a highly automated car? The difference between the systems in terms of which agents were included made it difficult to choose a metaphor as the experience of interaction pattern was conflated with the reliability of the automation itself. You might want a humorous intelligence interacting with you, but you do not want a humorous automation.
The metaphors thus managed to trigger high level discussions that questioned the ingrained assumptions of driver–vehicle interaction. However, it was notably hard for participants to move into the designing phase, and understand how to translate the choice of metaphor into an interaction design. One problem discussed was the lack of context or scenario which could set limits for the design possibilities (the group was free to choose their own scenario). As can be seen in Figure 5, the cards were used, sorted and annotated during the discussion and served as a record of which options had been eliminated, but they were not used to create a new interaction.
Figure 5. Participants discussing with the help of the cards and guided by discussion questions placed in the middle of the table.
5.3.1 Proposed concept
After much debate, the participants settled on ‘the private driver’ metaphor. This was translated to mean a competent and reliable vehicle, that would have understanding of your usual routes and could ask you if you wanted ‘The usual, sir?’. The metaphor also comes with a formality in the interaction that the group liked. However, they felt rather uncomfortable with the person-to-person aspect of the metaphor and thus tried to interpret the driver as a thing. The motivation was that the situation (driving) was too serious for a more humanlike representation – again the safety dimension dominated. The group chose to try to embody the metaphor in a person-to-technology interaction format, thus focusing on ‘traditional’ means of interacting with technology, such as button and dials. This was felt to afford explicit physical control of the vehicle; you physically could turn it on and off. The concept was thus represented as a big yellow button that could be pressed to engage and disengage automated driving when entering or exiting automated drive zones.
To further enhance the safety critical aspect and the precision it was deemed to demand, the formality gained from the metaphor was further enhanced by imagining a more Military-type interaction. Based on previous discussion, it was concluded that orders between human and vehicle should be made in complete clarity and with complete obedience. Once again precision was desired; the interaction could not be too playful.
In the evaluative discussion, the other groups commented on the dangers of accidentally switching the system off, in situations where it had control and the driver did not, when the system had to obey at any cost. The target group and scenario for the solution was also questioned – would everyone appreciate such a strict concept or could it be adapted?
5.3.2 Identified research needs and design implications
One major issue discussed was personalization: people are different, and will probably have different demands on the interaction. More research is needed to understand which individual aspects will affect which design aspects. In addition, people will change with usage and experience, and the initial learning curve is something that is very tricky to handle in design as features designed to help teach the driver will quickly become annoying. The interaction requirements are also likely to vary daily depending on mood and the activities performed in the vehicle. All these factors are currently unknown and need to be researched and designed for. In terms of design, the group also identified the important conflict between the reliability needed in the situation and the surprising element and excitement that form a part of a good user experience.
A further research need identified is the question of what can be learned from other automation processes. Is automated driving really different from interacting with any other type of automation? Which lessons learned are transferable, and which are not? Research is emerging within this vein, but cross-disciplinary collaboration could further transfer of knowledge even more.
6 Analysis and discussion
Designing for autonomous vehicles includes the same challenges as designing any novel technology, including the difficulties of imagining and ideating future use, while at the same time considering how the technology should fit into an already existing, and highly interactive traffic environment where safety is imperative. As automation increases, the relationship between the vehicle, driver and other stakeholders will radically change as well. Based on these challenges, we proposed that finding which, and in which way to employ, methods in order to manage to both address futures and social aspects in a fruitful way in early design phases was very important.
The aim of the paper was to explore methods for addressing the social interaction between future autonomous vehicles and humans, at an early stage of the development. Based on reflections on the outcomes of the workshop and with basis in previous work (see Section 2), we suggest that there are core components in successful method approaches that manage to address futures and social aspects: the methods should provide the ability to experience possible futures, challenge current assumptions, and truly engage in social experiences. We discuss whether methods used in the workshop had such abilities, and which aspects of them provided those abilities. We also touch upon the concepts produced during the workshops.
6.1 Experiencing possible futures
The findings from literature and workshop show that methods need to help their users create a space to imagine and test what possible future experiences could be like, both in terms of which situations will be relevant to address, which problems can occur in them, and which preferred future states are possible. Dubberly et al. (Reference Dubberly, Evenson and Robinson2008) concluded that designers often skip the explorations of models of what ‘could be’ in order to (too) quickly advance into detailed concepts, indicating that methods should aid and encourage this stage and help bridge the wide gap between analysis and synthesis and address the third quadrant. The design methods incorporated in the workshop activities contained both analysis and synthesis elements, approaching both the ideation of tangible representations of the ideas and the exploration of them in future situations. However, we found that the tested methods were to a varying degree able to effectively explore possible futures, that is, able to bring future situations to life in the exploration and support reflection on which interactions problems would be relevant in such a future.
In the theme with the WOz demonstration, the activities quickly managed to put participants in the shoes of the future users of an autonomous vehicle by their role as road users. By testing real-world interaction with the autonomous vehicle, the participants acquired an understanding for the social interplay and negotiation of the situation that quickly landed them in the question of what a pedestrian needs to know from the vehicle. This was an experience they then took with them into the small-scale scenarios and served as a reflection during the further development, meaning that the experienced situation from the outside could be made accessible indoors by the helicopter view of the encountered scenario, mapping out the different stakeholders’ perspectives of the scene, as found in previous research (e.g. Broberg et al. Reference Broberg, Andersen and Seim2011). This enabled multiple perspectives on the problem at hand.
For theme B, the social aspects were much more directly addressed when the group moved into the vehicle and participants engaged in roleplaying, than in the initial discussion. In the mock-up vehicle, dimensions of how the physical environment would affect the social situation became evident (from sharing an enclosed space with strangers came the ideas of e.g. the panic button) as well as engaging in making the problems tangible in representations such as buttons and screens and modes for Samaritan efforts. As found in previous research (e.g. Buchenau and Fulton-Suri Reference Buchenau and Fulton-Suri2000; Arvola & Artman Reference Arvola and Artman2007), the enactment, grounded in a consideration of embodied experiences (Wilde et al. Reference Wilde, Vallgårda and Tomico2017), did play an important role for bringing concreteness into the exploration and capturing the dynamics and the tacit aspects of the interaction design as well as provide a space for the group of designers/researchers to improvise and together create a common focus of a future design. All through the experimentation there was a balancing act between overly specified and close-ended scenarios and aiming for too wide scopes to be handled during one ideation session. This was especially evident in theme B, that extended long beyond the initial focus on co-experience inside the car to encompass many societal issues connected to future transportation systems at large. For theme C, using the metaphor cards, the session seemed to have remained too abstract and in the group discussions it was argued that a more precise scenario and set preconditions would have been beneficial, as the type of relationship that could be imagined depends very much on the state of the technology. In the end, the group chose a future quite close to reality with automated drive in specific, controlled, zones. The other two themes instead focused further into the future with their ideas, where autonomous cars had become a natural element in both the traffic environment and daily practices.
In conclusion, two of the explorations – WOz with small-scale scenarios and enactment – gave more fruitful discussions concerning possible futures, both in terms of discussions and in design ideas. We see a number of reasons for this. One important feature that was present in both WOz and enactment was contextualization. In both cases, the participants were invited into the situation they were designing for in a concrete form by experiencing the WOz car in a real-life traffic situation, or by enacting the situation in the pretend vehicle. This indicates the importance of creating immersed experiences of future technology in order to be able to design the interactions with it, i.e. simulating situations that can probe into the future. Both approaches included of physical artifacts; the WOz car, as well as toy cars and play mats, were used to concretize the participants experiences and ideation, while the enactment approach used chairs arranged as a vehicle and also a collection of props to help them enter different roles during the role play. The use of physical objects to relate to and to modify has been exemplified in foremost participatory design approaches as useful for understanding use and users (Brandt and Grunnet Reference Brandt and Grunnet2000; Sanders et al. Reference Sanders, Brandt and Binder2010; Broberg et al. Reference Broberg, Andersen and Seim2011). Contextualization appears to have triggered a higher degree of empathic awareness and presence (cf. Buchenau and Fulton-Suri Reference Buchenau and Fulton-Suri2000) than the more philosophical eyes on the issue triggered by the cards in the design metaphors. It also appears to have induced a more direct involvement with the solutions, again in comparison to the more abstract discussion when using design metaphors.
6.2 Challenging underlying assumptions
Since cars are such a big part of current culture, we carry many deeply held assumptions of how we should interact with them. To establish a preferred future interaction, it appears necessary to challenge these current practices of interacting with the vehicle itself, and which role it will play in future interactions between people. At the early stages of autonomous vehicle development, methods need to guide the exploration of new conceptual directions and allow new themes to emerge in the design space, before developing concrete solutions.
We found that the methodological approaches were to a varying degree able to surface current underlying assumptions and challenge them, in terms of for example highlighting the substantial role that social aspects already play in traffic. The WOz experience expanded participants view on what the social situation was, that it is not only one vehicle and one pedestrian, but a social interaction within a traffic situation with multiple road users. This enabling of multiple perspectives sets the research apart from previous research examples, where only the one to one interaction has been considered (e.g. Lundgren et al. Reference Lundgren, Lagström, Habibovic, Andersson, Nilsson, Sirkka, Fagerlönn, Edgren, Fredriksson, Krupenia, Saluäär and Larsson2016; Rothenbücher et al. Reference Rothenbücher, Li, Sirkin, Mok and Ju2016; Beggiato et al. Reference Beggiato, Witzlack and Krems2017), and contributed to a disruption in current thinking. In the theme of social interactions within the vehicle, the design concept also expanded from concentrating on one individual, which is the dominant perspective in todays research and design (e.g. Volvo Concept 26 and Mercedes-Benz F 015). The enactment highlighted the challenge to design for trust and security when sharing the car as an enclosed space, as well as service functionality for sharing car as an act of promoting social and environmental sustainability. In theme C, the metaphors as descriptions of potential future relationships with the car did not challenge the ‘one user’ perspective, but they did bring the discussion of what an autonomous car really changes in terms of interaction. They managed to put the vehicle’s new agency in focus, and disrupt assumptions regarding what it is that we are interacting with when the intelligence of the vehicle increases; the vehicle itself like before, the intelligent computer driving the vehicle, or a number of computers controlling different parts of a vehicle? However, while the theme A and B reinterpreted the social situations and their associated assumptions, theme C chose to stay with the current interpretation of direct control of the vehicle as they felt it to be safer and less ambiguous. Not being disruptive is not necessarily a bad thing, but the concept through its design ignored some important sensitive social situations, for example negotiation and conflict of intentions.
The methods also worked to include different perspectives of research into the topic; they enabled teams to gather cross-disciplinary views connected to social experiences around highly automated vehicles, especially so in the enactment and the small-scale scenario, which were more open to free manipulation, showing and telling. The workshop gathered researchers and practitioners from different fields, which provided an additional dimension to the workshop, perhaps the most important positive outcome of the activity; we emphasize that multidisciplinarity will be necessary to address the social aspects connected to automated vehicles, given the multitude of perspectives the area touches upon. For example, competences from interaction design need to closely collaborate with software developers, for instance, within machine learning, as well as hardware developers and user researchers. The workshop format provided a way to test the design approach and the specific techniques in a cross-disciplinary group.
In this context, the contextualizing techniques, which provided an experience and empathic understanding of the users in the context, served an additional purpose. They acted as a leveler among the participants, placing them all on the same page and with the same ownership of the problem at hand. This was for example evident in theme B, where the creation of concepts through enactment served as a leveler of the discussions, where the initial discussions were dominated by the ethnographic perspective. This can be compared to the role that methods play in creating team participation in, for example, usability studies (Wixon Reference Wixon2003). In the design metaphors approach, which had a very limited experiencing dimension, there was a tendency for one issue, safety, from one discipline to dominate the discussion more. Safety is a crucial issue when it comes to human–vehicle interaction, but was not the main interest in this research activity.
In essence, all tested methods managed to surface underlying assumptions and alternative interpretations by providing insight into a what a potential future could be like and bring different disciplinary perspectives to the forefront. However, the contextualizing techniques managed to also bring the groups to a joint and evolved understanding on which to develop a concept.
6.3 Engaging in social experiences
To be able to address the social aspects of autonomous vehicles, we found that it was necessary to use methods that help the participants to experience and engage in the social situations. Engagement can help define what types of collaboration and social relationships that are likely to emerge in autonomous cars, and hone in the exploration and ideation on the communicative aspects, in a way that highlights the vehicle’s new agency and new situations of use. As stated, the communication and flow of information needs to go both ways between humans and vehicle when automation increases and there is a need to create a mutual understanding between vehicle and human(s) of current states and intentions (Beller et al. Reference Beller, Heesen and Vollrath2013; Sibi et al. Reference Sibi, Ayaz, Kuhns, Sirkin and Ju2016; Pettersson and Ju Reference Pettersson and Ju2017). It is worth noting that each tested method only addressed a part of the design space and social aspects. The use of these specific methods may limit the concept development in specific directions. For instance, there is reason to expect that the ‘setting the stage’ methodology (Pettersson and Karlsson Reference Pettersson and Karlsson2015) in the enactment approach had implications on the modality of interaction – much of the communication between human and system in the concept was voice-based, possibly because ease of staging voice interactions in contrast to the extra efforts of creating visual or haptic interfaces for communication in the mock-up car. However, it also enabled the inclusion of multiple users, and was thus fitted to the topic of co-experiences of mutually using an autonomous vehicle, as previously explored (Jorlöv et al. Reference Jorlöv, Bohman and Larsson2017).
The use of the metaphors certainly triggered discussion about what the relationship between the vehicle and human should be like, at least in terms of the hierarchy and who should make the decisions. As mentioned, much of the discussion focused on where social interaction should unfold. The metaphors appear to not have included enough structure or information content to settle the issue, as they could be interpreted differently depending on reading and preconceptions. In a sense, the discussion also centered around whether you wanted to be social with the vehicle at all, or whether it was useful to consider the vehicle to be a social agent rather than just an advanced piece of machinery. The de-contextualized and abstract discussion had effects on the developed concept, which did not evolve much during the session. The initial discussions of theme B held a similar abstract pattern, but became more balanced and more detailed when entering the enactment stage.
Both the group addressing social co-experiences, and the group addressing the relationship with the vehicle explored a range of social situations, or how the social dimensions came to play in different future contexts of use, which allowed them to gain novel insights and challenge assumptions. This can be connected to their broad point of entry, lacking a set scenario. Theme A on the other hand quickly limited their social situation to only pedestrians crossing the street in front of the vehicle because this was the social scenario that they experienced in the WOz. This allowed them to dig deep into this particular situation and the communication of intentions and awareness, but may have narrowed the transferability of the solution. The addition of the small-scale scenario method played an important role in bringing in more social situations in the exploration. In the discussion and ideation, participants were able to shift between the WOz and small-scale perspectives, giving them a systems perspective on the issue at hand. The helicopter perspective (Broberg et al. Reference Broberg, Andersen and Seim2011) of the scene allowed the researchers to access the situation after leaving the field and served as a probe for communication between the participants. To a degree, theme B also included elements of this shift in perspectives. The initial interviews and discussion related both to the perspectives of different users and stakeholders and the role of autonomous cars on a society level, which led to a concept based on the potential for sharing cars and service development connected to autonomous vehicles. The enactment in the vehicle representation on the other hand made the individual experience concrete and clarified design demands stemming from such new services. The added insight of these shifts indicate that multiple methods should preferably be used to capture the social dimensions from a systems perspective.
6.4 Recommendations
The analysis of the results indicates important methodological aspects to ensure that single, and multiple methods, can achieve the desired effects, i.e. enable an understanding of future experiences, challenge assumptions, and engage in social aspects. We summarize these and recommend the use of methodology by which:
(1) The interaction is contextualized, in other words the interaction is placed in one or more specific, future scenarios. This can be promoted by for example including a mix between field work, establishing a united understanding of the problem, and exploration using tangible objects, such as enactment scenes with props as in theme A and B.
(2) The interaction between social entities is addressed in suitable detail. This requires a balance between being abstract in describing designs and specific in scenarios. While going deeper than a general discussion is necessary, there should be enough flexibility for exploring different paths of communication between user(s) and system, and room left for ideation and challenging assumptions. In theme B, enactment was especially useful for progressing beyond general discussion and instead becoming concrete about interactions, whilst retaining speed and plasticity in ideation.
(3) The investigation supports multiple perspectives on social interaction. This can for example be achieved by combination of methods that enables shifts in perspectives, like the shift between road user and helicopter view in theme A, or that every personal or disciplinary perspective is brought to the table as in the peer-to-peer interviews in theme B.
(4) The communicative aspects are focused upon, i.e. the ability to create mutual understanding of each others intentions and abilities. This requires that both human and vehicle agency can be represented and explored in the methods. In theme A and B, the WOz-vehicle and the enacted scenario facilitate such understanding by a physical meeting between agents where the meaning of intentions and abilities can be explored.
We believe that researchers and practitioners alike can benefit from these recommendations when envisioning possible futures and designing for social interactions with autonomous vehicles.
6.5 Limitations and future work
This paper presents a first exploration of the design space of social aspects in autonomous vehicles and the methods that could be used to address them in the early stages of the design process. The set of methods were approached with slightly different scenarios, aim and set of participants, and of course, no structured concluding comparison could be made. This limits the power with which the recommendations can be made. Our hope is that it can serve as a platform for necessary further studies and experiments to confirm or contradict the results of this study.
The methods selected for this workshop and discussed here are not the only methods available for bridging the synthesis and analysis gap in this very early stage of the design process (see e.g. Halse et al. Reference Halse, Brandt, Clark and Binder2010, Pettersson and Ju Reference Pettersson and Ju2017). Additional methods could probably be used with similar results, as long as they contain the necessary contextualization of future situations, physicality and experience to challenge assumptions and unite around, as well as possibility to explore two-way communication. Using this type of quick-and-dirty exploration and ideation methods allows for the combination of several methods and rapid shifts in between them, which was seen as fruitful. The exact combination of methods needs further investigation in future studies, for example addressing questions of whether to start wide and then narrow down as in theme B, or start narrow and then broaden as in A.
Compared to more detailed concept explorations, the ideas that come out are less resolved and harder to evaluate thoroughly. However, as mentioned in the introduction, at this early stage of autonomous vehicle development it is important to discuss the conceptual directions of development and what could be, and follow Dubberly et al. (Reference Dubberly, Evenson and Robinson2008) advice on avoiding jumping to detailed solutions directly. The workshop was indeed a short version of the synthesis–analysis bridge process, but as such was able to raise important concerns for methodologies suited for multiperspective explorations needed in a rapidly developing area of automated vehicles.
As pointed out earlier, poor design solutions for autonomous cars can lead to potentially very problematic social experiences, and the earlier we can access, explore and guide design in these situations, the better. Nevertheless, there are improvements to be made for future activities. A common reflection from participants was that it would have been helpful if the scenarios they had to work with had been more defined. It was perceived as beneficial if the parameters of design space, such as target demographic, context constraints, and specific situations, could have been given from the start of the workshop. However, being too constrained in scenario can also limit the possible futures explored and the social dimensions covered.
In the workshop, the themes of different types of social interaction were explored separately. An important next step would be to include all themes in a holistic approach in order to explore potential conflicts between social dimensions. In the end, there must be a unified concept for a vehicle that can handle all identified social interactions (and those yet to be imagined). The workshop also produced worthwhile ideas for future development and research, such as the concept for creating a mutual understanding of the vehicles vision and intentions, as well as interface for increasing safety and trust in shared mobility. These ideas should be explored further, and fed into the next cycle of analysis and synthesis. This will include more detailed concept, more situations, but importantly also potential future users. By expanding the activities to include users and their reception of the concepts, cultural aspects could also be further explored, which is another pressing issue in relation to autonomous vehicles (Vinkhuyzen and Cefkin Reference Vinkhuyzen and Cefkin2016). User inclusion would require further consideration to ethical implications, especially in field activities such as WOz.
7 Conclusions
In this paper, we have illustrated how the suggested design methods can be used to explore the social aspects of autonomous vehicles. Despite the limited study, the findings indicate method factors that appear to especially support fruitful outcomes, enabling the addressing of future scenarios at early design stages and balancing the co-development of creating insights into the problem and the solutions to solve that problem. The analysis also indicates that there are requirements for how the methods should be implemented in order to produce results. It appears necessary to apply multiple methodological techniques which offer a contextualizing dimension, a tangible way of experiencing problems and solutions as well as the ability to shift perspectives between social entities (e.g. user/car/pedestrian). In addition, we can conclude that professionals from multiple disciplines need to be invited to the design process to successfully address social aspects of autonomous vehicles, and that the design methods have an important role to play in creating a cohesive multidisciplinary team. Fruitful concept ideas that emerged in the workshop addressed the notion of trust in social interaction, e.g. the autonomous vehicle informing pedestrians and passengers of the vehicle’s intent and capability, and also for trust in relation to other people using shared mobility services.
8 Financial support
This work was supported by the Swedish Innovation Agency (Vinnova, FFI), through the projects HaTric (H.S., A.R.) and AUX (I.P.). It was also supported by The Knowledge Foundation (KK-stiftelsen) through the project AIR (J.A., M.K.). For the remaining authors (J.D., D.D.) this research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Open access
