2.1. Intellectual effort on display

One of the core values of live coding, at least as it is practised today with the TOPLAP manifesto’s ‘show us your screens’ bullet (Ward et al. Reference Ward, Rohrhuber, Olofsson, McLean, Griffiths, Collins and Alexander2004: 247) still resonating through the community, is to put the intellectual effort of musicking on display, usually by somehow projecting the code next to the performer. However, Palle Dahlstedt claims that if we are working with algorithms to make music and we want them to return good results, an embodied perspective is crucial (Dahlstedt Reference Dahlstedt, McLean and Dean2018: 42). He argues for different strategies where algorithms can be anchored in the physical world and where they become possible to ‘relate to, play with, and react to … in very much the same way as we relate to music coming from fellow musicians’ (ibid.: 44). One of the examples is the exPressure Pad, an instrument that explores dynamic mapping strategies (see, e.g., Dahlstedt Reference Dahlstedt2008), studied as a performer together with Per Anders Nilsson in their duo pantoMorf. Utilising pressure sensor-based controllers, the instrument explores a complex parameter space of timbral trajectories, but keeps some of the basic designs usually found in acoustic instruments such as the volume being proportional to physical effort and that ‘every change in sound corresponds to a physical gesture’ (Dahlstedt Reference Dahlstedt, McLean and Dean2018: 46). Similar ideas have also been implemented as hyper-instrumental extensions to keyboard instruments, enabling ‘an intricate search algorithm through … physical playing, in an embodied exploration which at the same time is music making’ (ibid.: 47). The notion of an embodied search algorithm explored in front of an audience has obvious connections, not only with free-improvised music in general, but also with the tradition of projecting your algorithms during a live coding performance, avoiding all kinds of obfuscation, putting yourself at risk and simulate a state of emergency:

In computer music we do not need to care about a correlation between effort and result. The decision to interconnect them is up to the composer or performer. On the opposite end of the continuum where a traditional live coding performance, with code and intellectual effort projected on the wall is on one end, we find the concept of acousmatic listening. Here, the act of non-referential listening is put in focus by avoiding visual stimuli and, when necessary, trusting the sound itself to carry the understanding of a composition. However, as Godøy (Reference Godøy2006) points out, listening is a multi-modal activity, and we understand sonorous objects through our bodies. He further states that ‘[t]he idea of motormimetic cognition implies that there is a mental simulation of sound-producing gestures going on when we perceive and/or imagine music’ (ibid.: 155), which suggests that the idea of the acousmatic might actually be impossible. From a wider, musicological perspective, Christopher Small argues that this extreme focus on listening within Western art music is a special type of musicking, which, to a large extent, is driven by capitalism (Small Reference Small1998). He exemplifies with the complex economics associated with traditional concert halls and further explains how these ‘place[s] for hearing’ (ibid.:19) separate themselves from the outside world to enable a ritualistic act of listening. According to Small, these, along with the record industry, have had such a strong influence on the Western idea of music that we have almost forgotten that the main purpose can also be the actual making of it, that is, the performative act could just as well be the principal aim and more important than the sonic result.

Dahlstedt seemingly puts a strong emphasis on effort-based instruments to enable empathetic perception by others and possibly a sense of understanding of underlying complex algorithms by an audience. Such instruments allow for traditional synchronisation if ‘the presence of performers and listeners who physically share the same space provides the framework for synchronization’ (Chagas 2006, quoted in Dahlstedt Reference Dahlstedt, McLean and Dean2018: 53). If ‘[m]usical embodiment is a temporal experience that requires the synchronization of temporal objects and events’ (ibid.), then such instruments might be less suitable for, for example, telematic performances where the lack of physical instrumental bodies and spaces are obvious. However, ‘algorithms can, in several different ways, help create an augmented mind’ (ibid.: 57). They can aid in a performance, as a tool for memorising or open up for non-linear thinking. Here, the algorithm becomes part of a cybernetic feedback loop, where the human and the machine (the algorithm) inform each other.

In the modular synth community there is a growing number of both professional and amateur musicians that use social networks for a kind of introspective musicking, akin to the TOPLAP manifesto’s call for ‘show[ing] us your screens’ (Ward et al. Reference Ward, Rohrhuber, Olofsson, McLean, Griffiths, Collins and Alexander2004: 247). Here, instead of code, the idea is to share patch experiences or so called noodles, not necessarily intended for pure listening enjoyment. The focus lies on the performative act of patching and musicking with modular systems. An interesting aspect of this is that there seems to exist a common, quietly negotiated agreement on the high value of these noodles. The shared experience of understanding and when the act of patching becomes an extension of your musical mind appears to be central here. There are of course critics looking for a pure listening experience, which, as Christopher Small puts it, takes for granted that music is intended for ‘listening to rather than performing, and … that public music making is the sphere of the professionals’ (Small Reference Small1998: 71), but the question of how it sounds seems secondary here. Similar situations, where the sonic material’s social function is primary, for example, exist within folk music and in rave culture, where dancing and interacting with other humans can be more important than the listening experience. A party night experience at a techno club where the DJs adapt their sets to the vibe on the dance floor is another example of such shared agencies.

2.2. Modularity and shared agency

A significant feature of a modular system is its ability to change. Thus, a modular synthesiser is more of a set of musical intentions than an instrument. It is a collection of ideas for sound design and musical structures, divided into subsets of functions, patched together for complex results. In such a system it is the patch that becomes an instrument or a composition, rather than the modules the system comprises, and ‘[t]he unpatched modular synth is thus nothing more than a possible future actualisation’ (Petersson Reference Peterssonin press). On the other hand, the patched-up modular system – with all its components and agencies – becomes an embodiment of intentions. It is also a system in flux where a live performance can involve everything from slight adjustments of single parameters to a transformational re-patching into a new instrument. Accordingly, Magnusson and McLean refer to the patch programming interfaces of these instruments as a special kind of tangible live coding (Magnusson and McLean Reference Magnusson, McLean, Dean and McLean2018: 250).

Cybernetic theories of feedback and control have also been thoroughly discussed and explored within modular synth communities. One example is the thread ‘Cybernetics and AI with Serge’ (Haslam Reference Haslam2020a), started by the musician Gunnar Haslam’s alias mfaraday on the Modwiggler forum. Using a small modular synth, he shows ways to explore concepts such as neurons and neural networks for artistic purposes on his YouTube channel La Synthèse Humaine (Haslam Reference Haslam2020b). Akin to Dahlstedt (Reference Dahlstedt, McLean and Dean2018), Haslam also takes a stand against art produced using these algorithms without human input, referring to classic cybernetic literature by Norbert Wiener (e.g., Wiener Reference Wiener1948, Reference Wiener1956) but also socialism and politics as in the writings of Karl Marx. For him, the goal is to explore a cybernetic system, rather than to create something predictable, and human agency is crucial as input to the systems in all these cases, asking the listener ‘to use [their] humanity to guide [them]’ (Haslam Reference Haslam2020c). The focus in such systems lies on how ‘all the actors co-evolve’ (Latour Reference Latour and Law1991: 117) and how ‘[a] cybernetic ontology would refuse any strict division between the human and non-human’ (De Souza Reference De Souza2018: 159). The sociological aspects of musical instruments are also interesting in regards to accessibility. Live coding environments are usually free and open source software and within the modular synth community there is a strong tendency towards DIY. This opens up for practically anyone with a computer and time and inclination to engage with the scene. Other musical instruments, especially venue dependant ones such as pipe organs, are traditionally more inaccessible, but technology can aid in diversifying access to such instruments.

2.3. Hyperorgans

In general, hyperinstruments are referred to as traditional instruments, extended by means of technology. The reasons are often to explore new instrumental affordances and uncharted sonic territories, but these extensions can also be added to provide ‘unprecedented creative power to the musical amateur’ (Machover Reference Machover1992: 3). Both reasons are relevant in regard to large pipe organs. As instruments surrounded with strong musical traditions and even religious symbolism, they are generally difficult to access, not only physically, but also in regard to gaining the necessary knowledge and trust to work with them. By extending instruments with such hyper-functionality, artists and musicians from other fields and backgrounds can gain access and at the same time develop novel approaches to their performance.

Similar to a modular synthesiser, an organ could also be thought of as a collection of subsystems that comprise a set of musical intentions:

Thus, the different musical functions of an organ could already be thought of as timbral extensions to one another. In a hyperorgan, such as the Utopa Baroque organ (Fidom Reference Fidom2020) at Orgelpark in Amsterdam, the signal flow has been made externally accessible through control interfaces. By means of protocols such as OSC and MIDI, this unlocks new possibilities for interaction and enables ‘[t]he inner modularity of the hyperorgan [to] be expanded with new modules, human and non-human, and even other hyperorgans’ (Harlow et al. Reference Harlow, Petersson, Ek, Visi and Östersjö2021: 5). Furthermore, it opens up for ontological questions regarding both material and immaterial aspects of the organ as a musical instrument. Bringing in telematic performance and live coding into the mix also enables sociological perspectives of access to these instruments and to the spaces where they are usually situated.

2.4. Musicking with patterns

Already in the early 1980s Laurie Spiegel (Reference Spiegel1981) made an attempt to provide a generalised set of useful pattern operations in music intended for computer musicians. As patterns in programming languages are often represented as lists of numbers, they can easily be manipulated in all the ways that she suggests, including, for example, transposition, reversal and rotation operations. In most languages there are extensive possibilities for list manipulation, but to make them correspond to intended musical results, aesthetic choices need to be made. As shown by Magnusson and McLean (Reference Magnusson, McLean, Dean and McLean2018), even a seemingly simple and straightforward operation, such as reversal, requires many interpretative artistic and aesthetic choices. Thus, instead of trying to unify methods in tools for musicking, Magnusson and McLean argue for systems where the ‘method names … become semantic entities in the compositional thinking of the composer or performer [that] outline the scope of the possible’ (ibid.: 262). Such semantic entities, with the ability to permeate and transform a musical mindset, could also be found in classic composition techniques such as counterpoint. For example, the notion of a cantus firmus, a foundational melodic line from which other material is derived by means of pattern operations similar to those listed by Spiegel (Reference Spiegel1981), could be understood as algorithmic composition carried out within, and taking advantage of the realm of traditional staff notation. In acknowledging that the music we write is influenced by such notation and how we leave important parts of the music to the interpreter to add into that system, Magnusson and McLean also discuss how programming languages can aid in going ‘beyond the usual dimensions of music notation’ and instead become ‘a process of live exploration, rather than description’ (Magnusson and McLean Reference Magnusson, McLean, Dean and McLean2018: 246). However, programming languages can also be ‘designed with increasing constraints, to the degree that they can be seen as musical pieces in themselves’ (Magnusson Reference Magnusson2013: 1). In such a system a composer’s artistic manifest could be inscribed into the language, not only similar to how a musical instrument embodies music theory (Magnusson Reference Magnusson2019), but also in how it proclaims a strong work identity.

3.1. The live coding system

Paragraph is a custom-made live coding interface for the creation of and interaction with SuperCollider patterns, which includes modular signal routing facilities, discrete as well as ambisonics panning, and takes advantage of multi-core processors (utilising the Supernova server). It stems from artistic practice and personal compositional techniques developed by the author over many years and was made as a tool to formalise and simplify coding of such patterns, and at the same time be able to interface with external signals in an efficient way. The name comes from the idea of writing short paragraphs that are representations of musical ideas, with the notion that everything written within the same paragraph technically refers to the same receiver (a pattern), and artistically to the same musical idea. The section sign, §, (which is called paragraf in Swedish) is used to syntactically refer to receivers. It was chosen partly due to the translational pun, and partly due to its accessibility, just below the escape key, on a Swedish keyboard layout. Receivers can be either patterns, where the section sign is followed by a number or unique name (e.g., §1 or §kick), or refer to the whole system with only the § symbol.

The syntax could be thought of as a dialect, derived from the standard SuperCollider syntax, constructed using the built-in preprocessor to filter the input code and return valid syntax. A basic objective was to keep the code interface minimal with as few delimiters as possible, in order both to fit on small screens and to be fast and responsive. Thus, spaces and line breaks are the only separators used, and the most common keywords have been abbreviated to either one or three letters. For example, the regular SuperCollider Event keys \instrument, \degree, \dur and \legato have been replaced by i, n, d and l, respectively. Value patterns that set those keys are expressed in a similar abbreviated way, (e.g., wht 0.1 0.5) instead of SuperCollider’s regular syntax Pwhite(0.1, 0.5);. Many useful default values for patterns and devices for generating or processing sound are predefined in the backend of the system. In SuperCollider lingo, this, for example, includes a default chromatic scale, a number of SynthDefs for playing back samples and simple synths for routing signals in the system, but the library is easily extendable as new needs and ideas arise. Figure 1 is a simple example of a Paragraph pattern named §1 playing such a predefined synth by setting the instrument parameter i to \sine on line 1. On line 2, the note degrees, n, are set by a rnd pattern that randomly picks a value from an array. In this example, the default chromatic scale is used (i.e., not set explicitly) and single note values and chords, expressed as sub arrays, are combined. A chromatic transposition value, cxp, is added to the note degrees on line 3, also chosen by means of a rnd pattern, but here with a supplied second array of probability weights. Both the duration d and legato l parameters on lines 4 and 5 are set by another type of random generator – the previously mentioned wht – which produce uniformly distributed values within the given range. The div parameter on line 6 subdivides and repeats the current event’s duration and the prb value of 0.4 sets a probability for an event to be played to 40 per cent. For the amplitude a, set on line 8, a sequence (seq) of numbers is used. The array of sequenced values also shows an example of a nested pattern, enclosed in parentheses. Here, the sequence enters a random pattern that chooses between three values four times before moving on. Further, the values produced are modulated by an independently running sine wave LFO with a frequency of 0.4 Hz and an amplitude of 0.2. As shown here, modulators are expressed with the | symbol and, besides simple oscillators, they can also be user-defined functions, hardware, OSC or MIDI inputs. All sounds generated by or passed through the pattern system can be processed by means of a global amplitude envelope, low and high pass filters, a tempo synced delay, dynamics and saturation. In Figure 1, the amplitude envelope and the saturation is set. On lines 9 and 10, the envelope’s attack and release times have been set to fixed values. Finally, on line 11, the saturation parameter (sat) is sequenced by a pattern that gets its values duplicated a number of times according to another sequence. Here, Paragraph uses the commonly used exclamation mark shortcut for duplication of values, found in standard SuperCollider syntax.

Figure 1. Example of basic Paragraph syntax.

The system is modular and can integrate external signals, hardware and controllers as well as internal modules (e.g., instruments, effects, midi devices, VST plugins, cv). From a user perspective, those modules are syntactically similar and, once defined in the system, referred to by a chosen name. The def method handles these definitions, following the syntax [def type nameTag arguments], where the number and type of arguments vary depending on the type. Reading from the top in Figure 2, line 1 defines a name tag for a sample and loads it into a buffer. Here, the provided file path points directly to a sound file. A folder of samples can also be defined in a similar way as shown on line 2, here labelled with the tag \kit. All samples in that folder now become accessible as instruments from any pattern as \kit followed by a number referring to the index of the sample in that folder. It is also possible to assign an array of several tags (e.g., [\kit \drums \perc]) to the same sample or folder of samples in order to facilitate different organisation of larger sound banks. An external MIDI instrument is defined on line 3. Here, besides the unique name tag, we also need to supply the MIDI device and port as well as the physical hardware input that the instrument is connected to. On lines 4 and 5 two send effects are defined. As mentioned earlier, the backend of the Paragraph system includes a number of such effects, written as SynthDefs in the regular SuperCollider syntax. Those are defined for use in patterns by providing a name tag and the name of the SuperCollider SynthDef to use. Figure 2 shows how these send effects can be used in a pattern (lines 9 and 10).

Figure 2. Defining and playing instruments and effects in Paragraph syntax.

Departing from my own practice, tools and aesthetics, Paragraph also facilitates musical ideas and concepts often returned to, to be expressed in an efficient and comprehensible way. As such it implements a personalised composer’s tool kit that includes certain methods for pattern generation and manipulation, developed and refined to fit my own musical voice. It is beyond the scope of this article to go into all the details of this toolkit, but the previously discussed notion of a cantus firmus has, for example, been implemented into the language. These are also patterns, but they run independently on a global hierarchical level. Thus, the data output of them could be shared among patterns on the lower level. A cantus firmus pattern consists of two subpatterns – one for the values and one for the durations of those values. Single or several such cantus firmus patterns could be defined, thus enabling the notion of co-players with agency over other pattern algorithms. In Figure 3, such a cantus firmus pattern with the name \chords is defined on line 1. This is used to set the scale (scl) in three patterns that independently play over the note degrees in these chords. The score next to the code shows one possible result of four bars, rendered as traditional notation. However, due to the randomness in the patterns each new rendition will be different.

Figure 3. A cantus firmus-driven pattern in Paragraph and one possible outcome in standard notation.

3.2. The TCP/Indeterminate Place Paragraph system

The TCP/Indeterminate Place Quartet (Figure 4) was formed with an intention to explore different types of interactions with the hyperorgans in the context of the Global Hyperorgan (Harlow et al. Reference Harlow, Petersson, Ek, Visi and Östersjö2021) project. A modular approach was used, both to the organs and to the instruments and agencies of the quartet. The notion of an organ as a modular system enables the addition of both new controllers and sonic extensions. Here, these extensions are other musical instruments, involving Robert Ek’s clarinet fitted with a motion sensor on the bell (ibid.), an electric MIDI guitar played by Stefan Östersjö, and Federico Visi performing with a laptop and gestural controllers. Visi’s setup navigates a sonic corpus of pre-recorded guitar sounds that is sonically extending Östersjö’s guitar and sending controller data to the organ network. Finally, the Paragraph live coding system described earlier, functions as an instrument played by the author of this text. Thus, the auditory output of a performance consists of the controlled local and remote organs, the clarinet, the electric guitar and any sounds produced or processed by the laptops. Visually, the organs have been a dominating part of the audiovisual stream due to the nature of the project, but also code projected on or next to organs have occurred.

Figure 4. The TCP/Indeterminate Place Quartet performing as part of the NIME 2021 conference, controlling the Skandia Cinema Organ in R1 (pictured), Stockholm, and the Utopa Baroque Organ in Orgelpark, Amsterdam, 16 June 2021.

From the perspective of the live coder, the Paragraph system, as it is used within the TCP/Indeterminate Place Quartet, can be understood as a complex arpeggiator playing geographically distributed instruments. So far, these instruments have involved different types of pipe organs, extended with remote control facilities such as MIDI or OSC. In a regular arpeggiator, the pitches corresponding to keys pressed on a keyboard are played one by one according to a selected pattern and synchronised to a clock. In many synthesisers featuring arpeggiators, patterns can be changed on the fly, and continuous modulation sources such as after touch, wheels and joysticks can be mapped to articulation (i.e., envelope parameters) and timbre qualities. This allows for less static performance. Staying within the analogy of arpeggiators, in this setup (Figure 5), the function of pressed keys is outsourced to a MIDI guitar. The guitar sends note numbers on separate MIDI channels, one for each of the six strings. These channel numbers are used to index the note values and store them in an array of six elements in the Paragraph system (Figure 5). In practice, this can be thought of as dynamically altering the pressed keys for the generated note events. Similar to the cantus firmus example earlier (Figure 3), the live coder maps the guitar-controlled array to the scale parameter in the patterns. Unlike a common arpeggiator, the patterns playing the chord notes are not limited to simple directional or random ones. Instead, those patterns can be of arbitrary length and produce both note degrees and chords within this scale, taking advantage of both SuperCollider’s extensive pattern library and any user-defined pattern algorithms. Typically, scales are represented as ascending arrays of numbers; for example, [0, 2, 3, 5, 7, 8, 11] for a harmonic minor scale. Scrambling such an array to something like [5, 3, 2, 7, 0, 11, 8], means that a note degree of 0 would not return the tonic (0) as one would expect, but instead 5. A degree of 6 returns 8 instead of 11 and so on. From a music theory perspective, such unsorted arrays are rather considered as sets of pitches than proper scales. While the initial idea indeed was to sort the incoming guitar notes incrementally, the affordances of an unsorted array, which instead independently stored the last played note of the respective guitar string, turned out to be more musically rewarding. The history of played notes is projected as sound events in the system of pipe organs through the live coded pattern, thus shaping the context in which the future notes are played by the guitarist. This particular affordance of the system unlocks both the notion of mimicry and opposites – where, for example, the guitarist can mimic the pattern generated by the live coder while at the same time rewriting the scale utilising the guitar as a live coding interface, and the coder can in turn decide to either follow or go against the texture produced by the guitarist.

Figure 5. The TCP/Indeterminate Place system from the perspective of the live coder.

Further, the gate length of the note events (i.e., the legato parameter, l) produced in Paragraph have been outsourced to the clarinettist, playing a customised instrument with a sensor-fitted bell, sending data wirelessly to the system. In this setup, the up and down movement (the Euler y-axis (pitch) of the sensor) of the clarinet was used to dynamically alter the note events produced between staccato and legato. The clarinet bell was also used to directly control the wind speed of the organ, which enabled new sonic possibilities beyond the tonal. Similar to the agency of the guitarist, the connection between the Paragraph system and the clarinettist establishes a multi-directional system of shared instrumentality where the sonic affordances of the live coder’s patterns can be radically altered by the clarinettist, who in turn acts within the dynamically changing performance ecosystem (Waters Reference Waters2007) created by the quartet.

As a final outsourced parameter, the control of tempo was shared between the live coder and the second laptop performer (Federico Visi) in the quartet. This was accomplished by means of Ableton Link (Golz Reference Golz2018). Utilising a Max patch and gestural controllers to navigate both a sonic corpus of pre-recorded guitar material and to send note and controller data to the connected organs, this performer’s instrument also incorporates sequencing and looping (see Harlow et al. Reference Harlow, Petersson, Ek, Visi and Östersjö2021 for further description). In practice, dynamic global tempo changes were usually avoided during the performance. Instead, clock dividers and multipliers were used to individually relate to the common clock.

The distributed parameter space utilised in this system thus forms a shared cantus firmus, exerting an obvious agency on the live coder. Controlled by the three other players, it forces an exploration of Paragraph’s affordances as a constrained instrument, where some of the most important parameters are outsourced. Further, as an aesthetic decision was made not to use any other sounds or treatments, except the controlled hyperorgan, the system limits the live coder to a pure exercise in quick adaptations to the incoming data, thus enabling an in-depth exploration of pattern affordances.

A similar system has been used in several different performances, both with the full quartet, sometimes physically dispersed, performing on one or more hyperorgans, and in duo settings with live coder and guitar or live coder and clarinet. With the full quartet, we have also performed on geographically distributed organs in Piteå and Amsterdam (see Video Example 1 and Visi Reference Visi2021a), and Stockholm and Amsterdam (see Video Example 2 and Visi, Reference Visi2021b), but also as part of larger constellations of musicians, telematically playing five hyperorgans located in Amsterdam, Düsseldorf, Piteå, Stockholm and Berlin.