2. IMPROVISATION, INTERSUBJECTIVITY AND ELECTRONIC INSTRUMENTS – CONTEXT AND STATE OF THE ART

Music performance cultures can be analysed according to the degree of variation allowed or desired in regard to an underlying model, be it explicitly written or tacit (Nettl Reference Nettl2013). In the larger picture, one may observe a continuum between practices that emphasise strict adherence to a given model and those where challenging the model is given fundamental value (Bailey Reference Bailey1992). The Western instrumental tradition of fixing music to a high degree by writing may appear as an exception from a global viewpoint (Lortat-Jacob Reference Lortat-Jacob1987). Live electronic music includes a strong improvisatory current, partly due to the diversity of tools, mappings and practices that tend to elude any attempt at strict notation, and partly due to the founding of live electronics in the transgressive aesthetics of the 1970s avant-garde of pioneering figures such as Alvin Lucier and Gordon Mumma (Dean Reference Dean2003: 20). Live electronics has presented the improvisatory counterpart to the ‘fixed media’ paradigm of the electroacoustic and acousmatic traditions.

A characteristic shared by many improvisatory musical practices is the propensity to emphasise the individuality of the improvising musician. The improviser is held to possess a culturally valued personal genius that enables one to express one’s individuality through a unique voice (Sawyer Reference Sawyer2014). In this view, particularly embraced in jazz-related improvisation (Berliner Reference Berliner2009), there is a fusion between the instrument and the player, and one may hear a person’s individual personality expressed through her instrument, as if the musician was speaking.

Electronic musical instrument design can be held as inheriting the ‘individual instrumentalist’ ethos to a large extent. For example, classic and authoritative approaches to experimental electronic instrument design, such as Cook (Reference Cook, Jensenius and Lyons2017), Tanaka (Reference Tanaka2000) and Machover (Reference Machover1992), embrace a single-user control paradigm, at least as an ideal if not in actual implementation. In this approach, the electronic sound device is seen as a personal instrument that offers affordances for an individual musician’s expression. In popular culture, the individual control ethos of ‘digital musical instruments’ (DMI) can be found in commercial branding discourse; for example, in names such as Native Instruments’s ‘Komplete Kontrol’ product line.

According to Magnusson (Reference Magnusson2019: 5), cultural artefacts such as musical instruments are epistemic tools that encapsulate knowledge, constituting instances of solidified thought and cultural knowledge. A musical system is hardwired into the design of a specific instrument. Following our research focus on intersubjectivity, the epistemic function of musical instruments can be extended to the social framework of a given instrument. Similar to the manner in which scales, tunings, or timbral spaces are built into instruments, so are the social conventions of their use. Instruments model the social space they are used in. Expressed in terms of actor-network theory, instruments exert an agency on the social forms of music-making. For example, a wind instrument does not allow for multi-user scenarios, whereas the four-hands piano is an established genre. Multi-player guitar is physically possible but rarely seen, and when actuated it holds a special symbolic value, such as in the 4-tissimo quartet playing on two guitars (Illarionov, Kossinskaja, Lonskaya and Shelyazhenko Reference Illarionov, Kossinskaja, Lonskaya and Shelyazhenko2009), or even further, the Trio Balkan Strings playing six-handed on one guitar (Starcevic, Starcevic and Starcevic Reference Starcevic, Starcevic and Starcevic2010). The synchronisation of arms and bodies on a single instrument has a spectacular effect due to its unusual intersubjective fusion. Beyond the solo or co-sounding conditions of a given instrument, the ensemble forms and presentation practices of music are largely shaped by the instruments’ physical characteristics. One may think of entire music cultures and repertoires as formed through the physical constraints and affordances – that is, agencies – of instruments and spaces of music praxis.

Electronic instruments also imply their social norms, which, following our argumentation, often tacitly inherit the individual instrument ethos of acoustic instruments. Experimental electronic instrument design has been closely associated with human–computer interaction (HCI), which is an area that largely focuses on single-user interface scenarios in accordance with our current computer and mobile device mainframe culture. The HCI approach has deeply influenced electronic instrument design, especially the digital musical instrument category. Thus, electronic instruments may be seen to stem from an ethos emphasising individual control inherited both from the acoustic instrumental tradition and from the HCI area. In this perspective, the electronic instruments are seen as personal instruments that offer affordances for an individual musician’s expression. Group musicianship within this paradigm would be understood as the coming together of the instrumental ‘voices’ of each player. However, electronic instruments break down the causal chain of energy transduction found in acoustic instruments (Cadoz Reference Cadoz1999), and they therefore offer a free space for designing associations between interfaces, sounds and diffusion systems, thus allowing for alternative social playing scenarios to be developed.

Important contributions have been formulated for the development of collaborative, shared and distributed electronic music instruments. For example, Sergi Jordà (Reference Jordà2005) and his team have developed landmark research in collaborative instrument design, giving rise to the notorious Reactable, a tabletop multi-user interface (Kaltenbrunner, Jordà, Geiger, and Alonso Reference Kaltenbrunner, Jordà, Geiger and Alonso2006). Further exploration of collaborative agency on the Reactable has been undertaken by Marley and Ward (Reference Marley and Ward2015). Fencott and Bryan-Kinns (Reference Fencott and Bryan-Kinns2010, Reference Fencott, Bryan-Kinns, Holland, Wilkie-McKenna, Mulholland and Seago2013) developed collaborative software design scenarios on the laptop. Ramakrishnan, Freeman and Varnik (Reference Ramakrishnan, Freeman and Varnik2004) have explored distributed, real-time, collaborative instrument design. Mayton, Dublon, Joliat and Paradiso (Reference Mayton, Dublon, Joliat and Paradiso2012) have studied a networked, remote control of a modular synthesiser via a custom-made web to analogue module conveying control messages over the internet to a hardware synthesiser. In the area of electronic instrument design theory, a dimension space for evaluating collaborative musical performance systems has been proposed by Hattwick and Wanderley (Reference Hattwick and Wanderley2012), with a multi-axis analysis comprising the notions of texture, equality, centralisation, physicality, synchrony and dependence. The affective and psychophysiological features of collaborative musical improvisation have been studied by Morgan, Gunes and Bryan-Kinns (Reference Morgan, Gunes and Bryan-Kinns2014). Blaine and Fels (Reference Blaine, Fels, Jensenius and Lyons2017) present a comprehensive summary of collaborative instrument designs, providing a relevant historical overview of the situation in 2003. Embodied music interaction studies have provided relevant theoretical approaches to shared electronic musicianship; for example, in Lesaffre, Maes and Leman (Reference Lesaffre, Maes and Leman2017). In the practice of HCI design, the work of Ben Swift (Reference Swift, Holland, Wilkie-McKenna, Mulholland and Seago2013) on ‘computer supported jamming’ is akin to our approach, showing an emphasis on the intersubjective value of music-making, although with important methodological differences between HCI and practice-based approaches. In regard to this considerable body of work on collaborative music systems, our contribution emphasises a practice-based design approach rooted in musical praxis and its inherent aesthetics.

3. AN INTERSUBJECTIVE TURN IN ELECTRONIC INSTRUMENT DESIGN

Two central arguments can be made to motivate our emphasis on human intersubjectivity and its technological mediation in the music domain, especially in improvisation. First, a quasi-exclusive focus on the human–machine relationship as a design space has led to the omission of the musicological and sociological evidence that music is essentially a social phenomenon. Indeed, collaboration is a prevailing feature of music history across cultures. Direct, in-time shared musicianship is clearly manifested in the myriad and omnipresent ensemble forms such as orchestras and bands. However, even seemingly individual musical activities such as composition involve an asynchronous network of collaboration with musicians, other composers and listeners. Improvisation stands out as a paramount domain of musical dialogue, where an open and engaged relation with the other is central.

Small (Reference Small1998) has been the foremost promoter of viewing music as an essentially social activity, coining the term ‘musicking’ to emphasise music as a socially embedded activity. Prior to Small, Bernard Lortat-Jacob (Reference Lortat-Jacob1987) has used the French equivalent musiquer in the context of improvisation studies, framing improvisation essentially as a means of communicating, being together and connecting within an emotional dimension. Improvisation requires putting oneself to play, taking risks and trusting, engaging in the dynamics of a ‘freedom of the self and freedom from the self’ (Frisk Reference Frisk and Schroeder2014: 153). Research in the area of ethnomusicology has brought forward an abundance of cases where music-making, and in particular improvisation, occupy a multitude of social functions that differ from the categories of concert and broadcast that are so paramount in current Western culture. As an example, one may think of Inuit throat singing (katajiak) discussed by Nattiez (Reference Nattiez, Falck and Rice1982), which is traditionally regarded as a playful duo game, without reference to a general concept of music. The Gbaya people of the Central African Republic have a tradition of ‘thinking-songs’ (chants à penser), that takes place in a restricted circle of just a few musicians, and occupies a recreational function (Dehoux Reference Dehoux1986). Music can also be an element of social construction, as shown by Anthony Seeger in his study of the Suyá in the Amazon, where singing is an activity that structures space, time and human relations (Seeger Reference Seeger1987).

Sawyer (Reference Sawyer2017) has focused on the inter-individual psychological processes of collaborative playing, giving rise to a shared feeling of conducive joint action with the term ‘group flow’, meaning a shared and psychologically rewarding state of concentrative absorption that involves merging of action and awareness, a sense of control and ease and alteration of temporal perception and reflective self-consciousness. Waters (Reference Waters2007) argues for instruments to be thought of as ‘performance ecosystems’ including the social dimension. Both ‘musicking’ and ‘group flow’ are by now widely accepted concepts and group flow has been researched quantitatively; for example, in Walker (Reference Walker.2010). In the light of this evidence, focusing on a dyadic relationship between a single player and the instrument thus misses an essential part of music’s ontology.

Second, DMI and augmented instrument design face the ‘gesture-to-parameter problem’, that is, the mapping of human gestures to the instrument’s parameter space, which is often very large. For example, the n-dimensional parameter space of a contemporary software synthesiser presents an overwhelmingly large number of parameters that could be musically meaningful via gestural control. Gesture-to-parameter mapping constitutes a modular and arbitrary space of associations, contributing to a highly volatile and heterogeneous environment to the musician. From one instrument (or instance) to the next, the outcomes of a given gesture will be different. The musician is brought to operate with an instrument that might present a cognitive overload due to too many simultaneous gesture-control variables, or a restricted environment where some variables are not included in the control mappings. A more social approach to music-making offers a possible solution to the gesture-parameter mapping issue: the parameters could be distributed amongst several musicians, resizing the parameter space of each individual to a manageable framework.

4. METHODOLOGY

This article builds on sustained practice-based research conducted in duo format by the authors, starting in October 2019 and continuing by the time of writing in 2021 The basis of the work has been formed by an in-depth experimentation on three successive electronic music creation systems: modular synthesizer, Pioneer DJ controller and web-mediated musicianship involving audio streaming between digital audio workstations as well as hardware control message transfer. On the basis of these three systems, the research aim has been to employ the electronic music instruments’ inherent plasticity in order to design user scenarios that advocate, or even enforce, synergetic aspects of improvisation.

The work has been plural in nature, involving research in the areas of intersubjectivity studies, music system design and music creation. In order to address the complex and entangled nature of the research object, we have adopted a cluster of methodologies within a practice-based approach. We have conducted exploratory duo sessions of co-musicianship, co-design and co-reflection, rooted in the research-creation methodology that brings artistic praxis and technological development into a mutually feeding loop (Gosselin and Le Coquiec Reference Gosselin and Le Coguiec2006). From a design standpoint, our approach is influenced by human-centred design, stemming from an immersion into improvisation and constantly identifying emergent properties from our use of electronic instruments (Boy Reference Boy2012). In order to maintain an analytical distance in our workflow, we have employed systematic video documentation, analysis and self-interview methods building on participatory video research (Milne, Mitchell and De Lange Reference Milne, Mitchell and De Lange2012), as well as participant observation (Skinner Reference Skinner2013). On a more general level, our approach is linked to artistic research, incorporating an assumption that music system design processes are relevantly conducted within music praxis and informed by it. Artistic praxis as a form of inquiry is able to mobilise the non-verbal epistemic qualities found in music, and may lead to insights that could hardly be revealed by external, technical or verbal-analytical methods alone.

The essential challenge in our approach has been how to think, conceptualise and evaluate intersubjective qualities of a collaborative system. Intersubjectivity is experiential in nature and has a fundamentally ineffable quality. There is no established definition of what intersubjectivity consists of, and what methods are functional in assessing it. Our approach to intersubjectivity has been informed by the video-based interaction analysis advocated by Sawyer (Sawyer and DeZutter Reference Sawyer and DeZutter2009), as well as by the conversation analysis developed within the University of Helsinki Finnish Centre of Excellence in Research on Intersubjectivity in Interaction (Simonen Reference Simonen2017). De Jaegher, Pieper, Clénin and Fuchs (Reference De Jaegher, Pieper, Clénin and Fuchs2017) have developed a self-immersive method for researching intersubjectivity, on the basis of ‘the idea that researchers of social understanding are themselves one of the best tools for their own investigations’ (ibid.).

In sum, we have adopted a self-immersive methodology grounded in artistic practice and aesthetics, incorporating an iterative design cycle as well as video and verbal analysis. The duo format was adopted on a pragmatic basis for fluid workflow organisation, but provided an effective setting for getting beyond a single person’s ‘lifeworld’ via a constant dialogical process. A development of the research setting towards multi-musician scenarios is planned for future phases of the work.

The research has been conducted in regular duo sessions, with an iterative structure comprising the following stages:

1. 1. Identification and implementation of a system as a starting point.

2. 2. Duo improvisation praxis on the given system.

3. 3. Video documentation of the improvisation.

4. 4. Oral and written feedback, collected on a shared document.

5. 5. Documenting the technical schema: patch, signal flow, mappings.

6. 6. Analysis of outcomes and assessment of the design’s qualities and shortcomings.

7. 7. New set-up design informed by the process, then reiteration of the process from 2) onwards.

A total of 23 duo sessions have been conducted, with 21 of them documented on video. The full video and written documentation including system description and comments have been uploaded on the project’s Research Catalogue page (see end of section 1 for hyperlink). The documentation constitutes essential auxiliary material to this article, as the video recordings are able to convey complementary information on the gestural, relational and aural aspects of the research presented here.

5. BODY OF RESEARCH

The body of research informing this article consists of three successive phases of enquiry on different electronic music creation systems, identified by the authors as possessing interesting potential for emphasised intersubjectivity in improvisation. The idea is to develop these systems towards engaging players to work at the ‘meta’ level of collaborative sound genesis rather than the traditional music ensemble model implying individual control of separate instruments in order to achieve a shared result. As a duo, our working metaphor has been the ‘four-hands electronic instrument’.

The three systems are, in chronological order:

1. 1. Modular synthesizer (04.11.2019–25.11.2019); videos 1 to 11.

2. 2. Pioneer DJ controller (05.03.2020–12.03.2020); videos 12 to 15.

3. 3. Web-mediated improvisation system, using digital audio workstation software, audio streaming and remote hardware control (26.03.2020–16.4.2020); videos 16 to 21.

In the following, we present the rationale, actual experiments with pointers to the online video documentation, as well as outcomes for each system.

5.1. Duo work on modular synthesizer

The first phase of exploration was realised on a modular synthesizer system comprised principally of oscillators, filters, low-pass gates, voltage control amplifiers, envelopes, touch plate controllers, midi to cv converter, random voltage generator and sequencers. The choice of the modular synthesizer as a platform for exploration was undertaken on the basis of its affording quick, intuitive and physically present ways to explore signal routings, as well as the smoothness and versatility of the control of signals in the analogue flow. As a duo, we could work seamlessly between patching and playing, developing new signal routings and interaction designs on the go, with an uninterrupted, high-quality, musically relevant audio output. In a sense, modular synthesizers appear as a meta-level of electronic instruments, offering basic building blocks that can be reconfigured and thus demonstrating the plasticity of electronic instruments. Eleven improvisation experiments were conducted and documented on video, exploring different configurations for creating synergy and connectedness between the players.

5.1.1. Additive synthesis metaphor

Our starting point was the metaphor of additive synthesis: in the manner of signals coming together in order to create one sound, we have staged two players to operate on the same signal path, the sonic result being the sum of two persons’ actions. The underlying idea is to engage two players in shaping a constant flow of signal on a shared modular synthesizer, in order to implement an inherently collaborative framework. Here the starting point was a relatively simple modular synthesizer patch, schematized in Figure 1, comprising three oscillators and a linked three-band filter, organised as follows:

• 1st ‘main’ oscillator (sine, saw, triangle) featuring a frequency knob as well as a switch for high register, LFO, sync and lock functions.

• 2nd oscillator, which was modulating the frequency of the first oscillator via an attenuator knob, providing controls for frequency and high register, LFO, Sync and Lock functions.

• 3rd oscillator (LFO), which was used to modulate the filter frequency, with controls for oscillator frequency and a switch for passing between normal register and LFO modes.

• Linked three-band filter (high, centre, low), providing controls for frequency modulation via a bipolar attenuator knob, main frequency knob, span (band separation) knob and Quality (Q-factor/resonance) knob.

Figure 1. Signal flow in the ‘additive synthesis’ design. White functions are controlled by player 1, black functions by player 2.

The distribution of control functions between the players was the object of sustained exploration. We made the observation of a tendency to ‘territorialise’ the interface, in line with findings and vocabulary by Kaltenbrunner etal. (Reference Kaltenbrunner, Jordà, Geiger and Alonso2006): one would quickly identify preferred control functions, and crossing the ‘border’ to the other player’s ‘territory’ of controls was subtly and subconsciously avoided. In response to this, we worked on mapping out a continuum of options between a ‘non-territorialised’ design where each player was free to act on any interface feature, and a ‘territorialised’ approach where each player’s scope of actions was restricted to a ‘private territory’. The whole collection of modular synthesizer video documentation shows variations of ‘territorialised’ (videos 1 to 5, as well as 8 and 9) and ‘less territorialised’ (videos 6 and 7) experiments.

In line with Fencott and Bryan-Kinns’s (Reference Fencott and Bryan-Kinns2010) findings on privacy and awareness in collaborative music, we found that even in a ‘less-territorialised’ setting, each player would intuitively concentrate on a personal set of interface features. This might reflect an interiorised model of the individual instrument, and it might also be related to the physical distribution of the interface features.

Video 1 (video example 1 in OS 26(1) sound and movie examples) shows a ‘territorialised experiment’ with the following division of tasks:

• Player 1 (on the right in the video documentation) was controlling the three-band filter’s cut-off frequencies, frequency modulation via a bipolar attenuator knob, span (band separation) knob and Q-factor knob. In addition, he was controlling the 1st oscillator’s main frequency knob and register function switch.

• Player 2 (on the left in the video documentation) was modulating the frequency of the 2nd oscillator and controlling its register (high register, LFO, sync, lock functions), as well as modulating the filter frequency with the 3rd oscillator’s (LFO) frequency knob and register switch.

In our view, video example 1 shows a sense of improvisatory sonic discussion stemming from the shared operation of a synthesizer, as well as the possibility to create larger timbral and formal variation than what would be readily produced by a single-user synthesizer. Here, four simultaneous control gestures on a single synthesizer patch are integrated into a coherent interplay of listening and action between the two players.

5.1.2. Co-sounding condition

A second notable design scenario arising from our exploration of improvisation on the modular synthesizer was the implementation of a primary condition where sound output occurs only when both players make a playing gesture. Thus, mutual action is needed in order to create a sound event. The ‘co-sounding condition’ was built on top of the previous additive synthesis patch, involving shared sound sculpting on a single signal path. We added two capacitive sensor gestural interfaces, namely Make Noise ‘Pressure Points’ controllers, as well as an output module comprising an envelope generator, low-pass gate (LPG) and a voltage-controlled amplifier (VCA) (Figure 2). Video 6 (video example 2 in OS 26(1) sound and movie examples) shows an example of improvisation on the ‘co-sounding’ system. The mapping of signals from the capacitive gestural interfaces was distributed in the following manner:

• Player 1 (on the right in the video documentation) had the same control functions as in the ‘additive synthesis’ design described previously, but the control functions were mapped from the Pressure Points gestural interface. In addition, player 1 triggered the envelope that controls the low-pass gate CV in, enabling him to affect the output stage. In this manner, both players were in charge of the output – both players needed to trigger the LPG and VCA in order for sound output to happen.

• Player 2 (on the left in the video documentation) was controlling the envelope generator’s attack, decay, release and sustain (on/off switch), as well as the LPG’s frequency, resonance, modulation CV, and modulation/ping signal switch. In addition, player 2 also controlled the output VCA (i.e., output gain) through the gestural interface.

Figure 2. Signal flow scheme for the ‘co-sounding condition’ duo improvisation design. The hands depict capacitive gestural interfaces. White functions are controlled by player 1, black functions by player 2.

In terms of interaction between players, the result of implementing a co-sounding condition is a situation where both players have equal power and responsibility over the sound. Both players may stop the entire sound at any moment, and both can equally influence the sound. Upon analysing the video documentation of the co-sounding experiments (videos 5–11), the term ‘negotiation’ stood out as a relevant descriptor of the situation. Shared responsibility over sound genesis leads to a constant and mutual engagement, where each player needs to continuously assess, react and propose in respect to the other’s actions.

The essential qualities of this interaction principle were found to be in the first place a heightened state of listening induced by an engagement in a constant negotiation of the sound. Second, the analysis pointed to an acute awareness of each other’s actions, analogous to team playing in sports, where each player is engaged in an action-reaction loop while simultaneously attempting to establish strategies or scenarios for future actions. Third, the aesthetic result was characterised by variety and surprise, as each player constantly brings fresh input to the improvisation. The signal shaping is enriched by the simultaneous operation of two minds and bodies, giving rise to sonic diversity, comprehensive use of the available parameter space, as well as more complex structural articulations resulting from the mutual sonic negotiation.

As a result of our exploration, the co-sounding condition seems to effectively implement a framework that enforces attention to the other person, creating an immediate feeling of communication and connection. The heightened interaction was observed as communication via gestures, verbal and facial expressions, laughter, as well as playfulness that, according to Sawyer (Reference Sawyer2017), constitute positive contributing factors towards group flow.

5.2. Four-hands electroacoustic deejaying

Following our exploration of intersubjectivity-emphasising improvisation on modular synthesizer, we wished to include digital audio processing and pre-recorded material in our work and proceeded to investigate alternative approaches and systems. We researched existing electronic music duo set-ups and joint deejaying caught our attention; for example, the Finnish female DJ collective Electronic Market, joining two persons on vinyl decks in ‘back-to-back’ DJ style, as well as the techno duo Mind Against who perform their co-composed music side-by-side on a digital DJ controller. We set out to investigate the potential of a digital performance tool specifically designed for DJ work: Pioneer DJ System XDJ-RX2. The controller, originally designed to allow for syncing, manipulation and effecting tracks from a digital library by a solo DJ, stands out as an elaborate controller for digital audio processing, offering extended possibilities of control signal and audio routing. A similar controller has been used for synthesis and sound processing by Jean-François Charles (Reference Charles2017).

We were particularly interested by the controller’s symmetric set-up and the physical proximity induced by it, with two ‘turntable’ rotating controllers and a central mixer section, making it an ideal system for side-by-side duo work. We envisioned the idea of ‘electroacoustic DJ’s’, turning a beat-music-oriented performance tool towards non-isochronous, sound-based improvisation.

We were able to conduct four experiments with the Pioneer DJ controller before the Covid-19 pandemic enforced a shift to social distancing and web-mediation. The work started from duo deejaying on pre-existing tracks in order to get accustomed to the interface (video 12). We then moved on to duo deejaying using our own sonic material, but found that the approach resulted in limited interconnection. The post-session comments point to a shared feeling of playing an instrument side by side, but the sounds and intentions did not really mix (video 13). The idea of the cadavre exquis (exquisite corpse) form emerged as a response: could more connection be found by sampling and manipulating each other’s sounds? Video 14 shows an excerpt of the cadavre exquis principle being explored. Consequently, a ‘dynamic cross-sampling’ system was patched and programmed, allowing for each player to record the other’s output at any moment, and use it immediately as a basis for his own sound work, as documented in video 15 (Video example 3 in OS 26(1) sound and movie examples), and depicted in Figure 3. The system is a ‘hack’ that goes beyond the pre-designed features and intended use of the Pioneer DJ controller by combining it with a max/MSP ‘sample and retrieve’ patch and creative signal routing. In addition, a double-stereo monitoring system was installed, allowing for a more immersive sound field and a more precise aural image of each other’s sonic gestures.

Figure 3. Schema of the dynamic cross-sampling system, showing signal flow and the respective ‘territories’ of player 1 (white) and player 2 (black) on the Pioneer DJ controller.

The ‘dynamic cross-sampling’ system had three notable effects that were found to enhance intersubjective connection and promote an aesthetic coherence in the improvisation. First, listening is heightened, as the source of one’s own material lies in the sounds that the other is making. One constantly listens to the other player in search for interesting materials to sample and process. Second, the cross-sampling assures a certain homogeneity of the material, as sounds are exchanged and processed to new forms. However, this does not result in monotony as the processing tools are potent enough for an extended and inspiring transformation work (the Pioneer DJ controller software’s built-in audio functions enable pitch control, slicing, looping, scratching, envelope manipulation and effect processing). The system induces a felt balance between improvisatory possibilities and a connection vehiculated through the sonic material itself. Third, sampling creates a memory of the improvisation, accessed via the recorded file database, enabling one to browse past moments of the improvisation and to re-introduce them into the improvisatory flow. The capacity to re-enact the past creates fertile options for the formal construction of the improvisation.

The ‘dynamic cross-sampling’ system resulted from an iteration of experimental improvisation sessions combining playing and system design. In our work, it marked a step from seeking intersubjective connection through control parameter mappings (as with the modular synthesiser), towards an intersubjectivity based on shared sound material. In our view, the emergence of the cross-sampling system constitutes a rewarding example of a research-creation process, where the actual research substance becomes apparent by a shared immersion into the improvisatory praxis. Figure 4 shows the four hands in action on the Pioneer DJ controller.

Figure 4. ‘Conversation of hands’: four-hands improvisation on the Pioneer DJ controller.

5.3. Web-mediated duo improvisation

Our duo work on the Pioneer DJ controller was interrupted by the Covid-19 pandemic and physical distancing measures forced us to halt our in situ experiments. Thus, a shift to a third music creation system, namely online duo musicianship, was enforced upon us in order to be able to continue the research. However, with hindsight the shift has proven to be beneficial for the overall research, as it has allowed us to include the context of web-mediation to our enquiry. Web-mediation poses an enormous challenge to intersubjectivity, as it rules out direct physical presence and embodied communication. Turning the situation upside down, web-mediation actually simplifies and breaks down the problem, allowing for an exploration of ‘what is left’ in the intersubjective situation, that is, the audio, visual and data exchange channels. In this sense, web-mediation can arguably serve as an effective platform for researching intersubjectivity and improvisation as it simplifies an otherwise highly complex and elusive setting.

Shifting to web audio involved the preliminary issues of signal transfer and latency. We tested two audio streaming solutions, Cockos Ninjam and Audiomovers Listento vst-clients, and settled for the latter for the lower latency and larger choice of audio compression formats, namely AAC 128kbit/sec compression with an average 500ms transfer latency. A 500ms latency is considerable and rules out coordination of sharp, attack-oriented musical gestures. However, we found that it is possible to co-work with such latency within the framework of relatively slowly evolving electroacoustic gestures. Psychologically, the situation felt like we were engaged in a shared improvisation that still fits into the human perception of real-time, although the set-up, signal routing and communication were heavy to manage.

The dislocation of listening environments in group musicianship is an unhabitual situation for musicians accustomed to working in a shared physical space. The online situation blurs established points of reference such as sound ownership and physical source localisation. Web-mediation creates different instantiations of the same audio material actuated in parallel, leading to different mixes at each end of the transmission. There is no longer a single reference ‘truth’ of the audio, rather, each musician works in a personal instantiation of the shared material. In our experiments, the multi-localisation led to situations where each musician was working on a radically different mix of the two audio streams from the other. The dichotomy was only discovered afterwards via our video recording footage. Videos 17a and 17b constitute an early example of web-mediated duo work, filmed at each side of the transmission channel. They effectively show the difference in the mixes of a shared online music session. Nevertheless, the verbal feedback collected on the session is not overly negative. Interestingly, even when working simultaneously on radically different mixes it was still possible to operate together, engaged in a shared activity. This finding points to interesting questions that are left outside this article’s discussion: to what extent does intersubjectivity in improvisation require sharing the same sonic material? How important is the role of the sole intention of being and working together?

Our second and third experiments in web-mediated musicianship followed the cadavre exquis strategy that we had previously developed with the Pioneer DJ controller (see section 5.2). The idea consists of a simple seed audio material being streamed between the players, and both simultaneously processing it, as shown in videos 17a and 17b. The idea was developed into the ‘dynamic cross-sampling’ strategy previously explored, as shown in videos 18a and 18b. This process gave rise to a functional way of working on shared material. However, the situation did not have the immediacy and connectedness that were present in the physical side-by-side music-making with the Pioneer DJ console. In addition, the dislocation of monitoring with individual mixes at both ends of the transmission, plus the addition of the web video call with its own sound, led to a situation with ‘two parallel realities’ that does not fundamentally support intersubjective connection.

Trying to address the issue, we switched to the ‘four-hands modular’ strategy previously explored (see section 5.1), but this time with remote control of the modular synthesizer routed via User Datagram Protocol (UDP). This approach allowed us to have a single sound source and a single mix, binding the dislocated players and monitoring systems into a unique sonic entity. The set-up consisted of Alejandro Montes de Oca controlling the modular synthesiser via the Pioneer DJ controller’s MIDI output on a local network, and Otso Lähdeoja sending MIDI control data from an Akai MPK mini from a remote location. The audio was transferred as previously, using Audiomovers plugins. An added feature in regard to the previous web-mediation was the addition of supplementary video feeds. In order to enhance the impression of physical communication, both players had two cameras: one showing the hands and the controller, the other showing the face and torso. The whole video feed was multiplexed using Zoom.us, as shown in video 19.

The move to a unique, shared sound engine provided coherency and consistency compared with the previous cadavre exquis experiment, as there was a ‘ground truth’ sonic reality both players were referring to. However, the situation felt very different from the physical four-hands modular, as there was no immediate and intimate knowledge of the other player’s gestures. The two-plus-two video monitoring proved to be ineffective due to the small video images’ lack of presence and non-synchronisation with the sound (sound and video were transferred via different channels and protocols: Audiomovers and Zoom respectively). The whole visual feedback system was not supporting the improvisatory dynamics, rather provoking disorientation. The system entailed a confused impression of both players’ agency and gestures, as if trying out blindly a system one does not know.

The frustration with the visual feedback led to the idea of a graphic monitoring of the sound, which proved to be the major finding of our web-mediated experiments. Various audio visualisation methods were tested before landing on the Izotope Insight plugin with its spectral histogram of user-definable length, allowing the shared audio stream to be perceived as a constantly refreshed topographical map. The histogram creates a medium-term visual memory for the improvisation, allowing time-related structural strategies such as creating patterns, rhythms, contrasts, or gestures in spectrum and time to be devised. The visualisation also allows for clearly perceiving the other player’s input and its spectromorphological character, fostering dialogue and interplay. The post-session verbal feedback made apparent that in a dislocated situation, with no functional visual feedback of the other player’s gestures available, a graphic visualisation of the audio could significantly enhance the coherency and connectedness of the improvisation. Video 20 shows the visual feedback window for an improvisation on a single shared signal path, comprising the Izotope spectral histogram as well as two camera views on each player (head and hands). Video 21 (Video example 4 in OS 26(1) sound and movie examples) displays the same system with two signal paths, adding more sonic complexity. Figure 5 is a screenshot from video 21, depicting the spectral histogram’s use in duo improvisation.

Figure 5. Screenshot of a web-mediated duo improvisation session, showing visualisation via a spectral histogram and four multiplexed video streams from the two players.

Being able to monitor the improvisers’ actions through a shared visual representation was a key shift in our web-mediated experiments. In analysis, the graphic representation stands in line with the pervasively visual environment of the internet. Current web environments are predominantly constructed with a paramount emphasis on visual modality. As such, it felt like the shared visualisation ‘speaks the native language’ of the web environment. Beyond the interest of this approach within web-mediated situations, the advantage of a shared visual representation in terms of shared histogram and graphic map of the musical discourse makes it a tool of interest for physically situated improvisation as well, particularly adapted for electroacoustic aesthetics where music-making is readily approached through spectromorphological entities.

6. OUTCOMES AND FURTHER PERSPECTIVES

The research work presented in this article addresses three interwoven dimensions of technologically mediated improvisation: first, the intersubjective connectedness generated by improvisation and the prospect of its positive mediation via technology; second, improvisation as a practice-based method for design and development of electronic instruments, with aesthetics as the guiding factor; third, the perspective of shared improvisation constituting an alternative social function in regard to concert and broadcasted music.

In the area of fostering intersubjective connectedness, the research has tapped into the plasticity of signal routing and mapping afforded by electronic instruments, in order to conceive systems that would go beyond the ‘collaboration between individuals’ scenario towards a fusion of individual agencies. The work gave rise to terminological discussions, pointing towards a shift from the term ‘collaborative systems’ to ‘synergetic systems’,Footnote ¹ emphasising the fusional and emergent properties sought after. Joint signal paths, cross-modulation and cross-sampled audio material have constituted the core strategies of exploration, giving rise to four relevant principles that were found to foster the shared impression of connection and flow. The ‘additive synthesis’ metaphor provides a way to immerse improvisers into a shared agency over a single sonic discourse. The ‘co-sounding’ principle reinforces the fusion by making simultaneous gestures a condition of sound output. Both approaches renounce the ideal of the autonomous creator (Frisk Reference Frisk and Schroeder2014), trading individual agency for emphasised intersubjectivity. The ‘dynamic cross-sampling’ system preserves individual agencies by staging side-by-side sound work, and emphasises listening as a channel for intersubjective connection. As the other player’s sounds become one’s own material, a heightened sense of awareness to the other’s actions is created. Finally, a visual feedback system plotting a spectrogram with a time dimension was found to be an effective means for creating coherency and the impression of coming together on a shared sonic improvisation.

This research demonstrates that improvisation can constitute an effective methodology for research-creation, given that it is combined with analytical tools such as video and interviews. An iteration of improvisatory praxis and analysis cycles brought us to contemplate the essential questions of our enquiry, rising directly from the praxis itself and its aesthetic dimension. The felt qualities of the music being made as well as of the intersubjective connection were our central concern and the guiding line of the design process.

The articulation of the musical qualities in regard to intersubjectivity emerged as the essential meta-question haunting our enquiry. Does intersubjective connectedness between improvisers contribute to the quality of the resulting music, such as in terms of adequately placed interventions, refined sonic gestures, formal development or sonic originality? It is possible to imagine a seasoned improviser working with peers in an intersubjectively disengaged way, analysing the aural environment and mobilising her skillset to contribute in an adequate manner. In addition, one can imagine an overly emotionally engaged approach as actually blindfolding an improviser, making her less apt to listen and react. A possible approach to the dialectical setting between musical and intersubjective aesthetics is to examine improvisation as an aesthetic and social category in itself.

The hypothesis emerging from our enquiry is that improvisation between peers may constitute an alternative social function of music, with its associated aesthetic value system, emphasising intersubjective connection over structural aspects of music. As ethnomusicological data shows (see section 3), concert and recorded music are far from being the only functions music can be – and has been – used for. ‘Being together’ may in fact constitute the principal motive for music-making, as argued by Small (Reference Small1998). Technological mediation holds transformative power for the ways people come together, apparent in recent culturally paramount shifts such as the rise of social media. In this respect, the plasticity of electronic music systems, web-mediated or physically situated, can afford the creation of novel scenarios for intersubjective synergy.