2. THE STATE OF THE ART

It is widely observed that electronically synthesised music undertook a major transition in the 1960s from predominantly academic and experimental settings to the popular realm due to musicians such as Wendy Carlos with her Switched On Bach releasing crossover works with broad appeal to the public (Schrader Reference Schrader1982). Similarly, the 2010s have seen musical robotic and other non-loudspeaker-based electroacoustic music suddenly gain significant traction with a number of well-known artists internationally. Examples include Björk’s ‘Thunderbolt’ (Guðmundsdóttir 2011) which makes use of bolts of plasma from Tesla coils as ethereal sounding bodies, Richard D. James’s Computer Controlled Acoustic Instruments Pt2 (James 2015), which utilises musical automata developed by the Logos foundation to actuate acoustic instruments, Tom Jenkinson’s Music for Robots (Jenkinson 2014), which was realised using custom-built, pneumatically powered anthropomorphic musical robots and Pat Metheny’s Orchestrion Project (Metheny 2012), which enlisted a team of musical roboticists including Eric Singer and Ken Caulkins to create a large array of electronically augmented and actuated instruments. These artists all released internationally distributed albums or EPs making use of non-loudspeaker electroacoustic instruments as primary features.

The business world has also begun to take notice of the appeal of these alternative methods of electroacoustic music composition, and several large corporations have sponsored the creation of robotic musical ensembles. Examples of this include Anheuser-Busch commissioning Andy Cavatorta to create robots that utilise Stella Artois beer glasses to create music (Cohen Reference Cohen2014), Molson Coors Japan commissioning a team of engineers to build the Z-Machines musical robot ensemble to promote their Zima alcoholic beverages (Smith Reference Smith2014), Moleskine commissioning Felix Thorn to create the Moleskine Orchestra using their products (Thorn Reference Thorn2012) and Festo working with Roland Olbeter to create their Sound Machines and Sound Machines 2.0 (Festo 2012).

These large-scale projects and corporate sponsorships were certainly not without precedent. Figure 1 shows the explosion in the number of artists making use of musical robots to create electroacoustic music that occurred in the 2000s and 2010s. The shape of the observed growth suggests that these types of works will become increasingly prevalent in the future, and also shows how this trend can be traced back to the earliest examples in the early 1970s. The most prolific pioneers of this 1970s renaissance were Godfried Willem Raes, whose works have accumulated to become the world’s largest machine orchestra – the Logos M&M Robot Orchestra (Maes, Raes and Rogers Reference Maes, Raes and Rogers2011), Trimpin, who over the previous four decades has created some of the most large-scale and innovative works of non-loudspeaker electroacoustic music in the field (Esmonde Reference Esmonde2011), and Ken Caulkins, whose musical automata business Ragtime West has provided a very large range of installations and performance instruments internationally for high-profile clients such as Disney and Pat Metheny (Caulkins Reference Caulkins2016).

Figure 1 Approximate dates of entry of musical roboticists.

While Raes has focused more heavily on robotic instrument creation, Trimpin has tended more towards installation projects and Caulkins has focused on commercial endeavours. Each has drawn much inspiration from the breadth of historical musical automata and played their parts in seeding the field of musical robotics with their respective unique artistic aesthetics and technical inventions. All these pioneers are still very active in the field, creating new artworks and technologies in the present day.

Following these early innovators, the 1980s saw the beginning of larger companies understanding the potential of these new types of musical instruments. In 1982 Nippon Gakki (now Yamaha) developed the Disklavier (Palmieri Reference Palmieri2004) and in 1985 Bösendorfer developed the 290SE, both computer-controlled acoustic pianos and both influenced by Wayne Stanke’s work of the late 1970s and early 1980s electronically automating pianos (Moog and Rhea Reference Moog and Rhea1990). These robotic pianos and subsequent iterations were used extensively in commercial, artistic and academic contexts at the time such as Jean-Claude Risett’s 1989 work, Duet for One Pianist (Risett and Duyne Reference Risett and Duyne1996), and have endured to the present day in works such as the Lexikon Sonate (Essl Reference Essl1992) and the Mirror Fugue project by Xiao Xiao (Xiao Reference Xiao, Tome and Ishii2014).

Another notable development during the 1980s was the beginning of the anthropomorphisation of musical robots. One of the most notable groups was in the Waseda University Science and Engineering Department who unveiled WABOT-2, a humanoid robot that was capable of playing a keyboard (Sugano and Kato Reference Sugano and Kato1987). Throughout the 1980s and beyond, the group has continuously developed a range of anthropomorphic musical robots including a flautist (Solis, Chida, Taniguchi, Hashimoto, Suefuji and Takanishi Reference Solis, Chida, Taniguchi, Hashimoto, Suefuji and Takanishi2006), saxophonist (Petersen, Solis, Ninomiya, Yamamoto, Takeuchi and Takanishi Reference Petersen, Solis, Ninomiya, Yamamoto, Takeuchi and Takanishi2009) and other robotic musicians (Solis, Ninomiya, Petersen and Takanishi Reference Solis, Ninomiya, Petersen and Takanishi2009).

The 1990s saw a steady stream of new artists join the established pioneers from the 1970s and 1980s, each contributing novel techniques and new artistic aesthetics to the field. While academic laboratories such as the Takashima lab of Hosei University utilised their engineering expertise to build cutting-edge saxophone, trumpet and trombone-playing robots (Takashima and Miyawaki Reference Takashima and Miyawaki2006), several artists made use of the technology to introduce their own creative aesthetic. Meiwa Denki emerged with various novel robotic guitars and percussion instruments imbued with a sense of humour and playfulness (Grunebaum Reference Grunebaum2010), Jay Vance created his somewhat theatrical Captured! By Robots robot band complete with automatic guitar, bass, drum and horn players, and Roland Olbeter presented Afasia, a stage production that included robotic violin, guitar, drums and bagpipes (Jordà Reference Jordà2002). It was also in the late 1990s that QRS music technologies released their own take on the computer-controlled acoustic piano, additionally releasing a robotic violin called the ‘Virtuoso Violin’ (Futterman Reference Futterman1998). These works brought the world of non-loudspeaker electroacoustic music onto the theatre stage and often merged them with a range of hyper-instruments and other related art forms such as contemporary dance.

The 2000s is the decade where non-loudspeaker electroacoustic music achieved its tipping point and started to see seemingly exponential growth. The accessibility of micro-controller technology coupled with the broad attainability of information via the internet lowered the previously high technical barrier to entry and enabled artists and musicians with comparatively lower levels of engineering training to participate in and contribute to the area. A possible contributor to this rise in popularity is electronic dance music’s explosion in popularity at the time, which left the art world eager to branch away from traditional acousmatic forms towards new electroacoustic musical experiences, and this new capability provided them with the means. In ‘Trimpin: An Interview’, Trimpin provides an explanation for the popularity of musical robotics in the twenty-first century:

During this decade new collectives such as Eric Singer’s Lemur (Singer et al. Reference Singer, Feddersen, Redmon and Bowen2004), Kurt Coble’s P.A.M. Band (Sobh, Wang and Coble Reference Sobh, Wang and Coble2003), the Karmetik Machine Orchestra (Kapur, Darling, Diakopoulos, Murphy, Hochenbaum, Vallis and Bahn Reference Kapur, Darling, Diakopoulos, Murphy, Hochenbaum, Vallis and Bahn2011) and EMMI (Rogers, Kemper and Barton Reference Rogers, Kemper and Barton2015) arose, most of them facilitated by academic institutions and each contributing a range of new musical robotic technologies. As musical robotics began to be accepted as a discipline within the wider field of computer music, international electroacoustic music-oriented conferences dedicated paper sessions, workshops and keynotes to the topic (NIME 2007). A number of articles such as Kapur (Reference Kapur2005), Murphy, Kapur and Carnegie (Reference Murphy, Kapur and Carnegie2012) and Maes et al. (Reference Maes, Raes and Rogers2011) provide more detailed information on the developments made during that period.

So, it is clear that the recent mainstream breakthrough electroacoustic works of Jenkinson, Björk, Metheny and James did not occur in isolation, each having ties to innovative work outlined above that occurred in the 2000s and earlier: the technology behind ‘Thunderbolt’s musical Tesla coil was made possible by the work done by Joe DiPrima and Steve Ward in 2005–06 (Long et al. Reference Long, Bailey, McVay, Carnegie and Kapur2015), Computer Controlled Acoustic Instruments Pt2 was enabled by decades of research conducted by Godfried Willem Raes and the Logos Foundation in Belgium (Raes Reference Raes2014), and the Z-Machines themselves were created by a team of engineers commissioned by Zima, no doubt inspired by the earlier robot band Compressorhead.

3. THE EARLY BEGINNINGS OF MUSICAL ROBOTICS

While it is relatively straightforward to trace the origins of the musical Tesla coil and plasma speaker back to Nikola Tesla’s invention of the original Tesla coil in 1891 (Tesla Reference Tesla1891), the roots of the seminal musical robots of the 1970s run much deeper. While the widespread availability of the transistor in the 1970s can be seen as a major catalyst that launched the fields of analogue synthesis, musical robotics and computer music, earlier technologies such as relays were also used to realise non-loudspeaker-based musical works.

An example is Godfried Willem Raes’s Reference Raes1970 Bellenorgel (Figure 2), a pre-microcontroller automatic sound sculpture project constructed around an electromechanical counter used to trigger actuations of a range of bells, horns, sirens and other sound-objects (Raes Reference Raes1970). This was enabled not by software or even control voltage signals but by a set of telephone relays. Trimpin also refers to working with relay-based logic before transistors became commonplace (Leitman Reference Leitman2011). There have also been many instances throughout the last decades of electroacoustic work which is created without software or firmware, but via simple electrical circuits that provide power to actuators such as electric motors or solenoids. Examples include many of the works of Joe Jones, and more recently Zimoun and Pe Lang.

Figure 2 The Bellenorgel. Credit: Godfried Willem Raes.

Given that these instruments and installations do not utilise any semiconductors or formal control logic, rather than labelling them ‘electronic music’, they could be more accurately described as ‘electric music’ (Davies n.d.). This delineation can be useful in distinguishing the difference in musical effect between systems that make use of electronic semiconductors and digital logic for control, and those that merely use electricity as a power source to actuate real-world sound objects. However, is the presence of electricity itself really the most important musical characteristic of such a system? To an observer of Zimoun’s 5 pvc-hoses 1.0 mm, compressed air installation (Figure 3), would the effect or the experience offered by the work be any different if the resulting movement and sound was actuated by an electric air compressor, a gasoline powered or hydraulic unit, or indeed the wind? It would seem that there are more important distinctions that differentiate electroacoustic music from traditional musical performance than the mere fact that electrical current flow is present in the devices used to create it.

Figure 3 Zimoun’s 5 pvc hoses 1.0 mm, compressed air.

Some of these more relevant distinctions are discussed in Barry Schrader’s Introduction to Electro-Acoustic Music, where the author outlines some of the reasons composers opt to make use of electroacoustic techniques in their works as opposed to traditional score writing for human performers:

Indeed, these aspects that Schrader identifies as being key reasons why composers opt to create works in the electroacoustic realm apply equally to concrete musical composition practice for non-loudspeaker-based works, be they powered by electricity, air, water, or other means. The following section explores a number of such works.

4. ELECTRONIC MUSIC SANS ELECTRONS

Pierre Schaeffer’s musique concrète is indeed a seminal development that introduced the concrete ‘sound object’ into our lexicon. In contrast to a history of acoustic music that was composed, interpreted, performed and then vanished into the realm of memory, the development of an enduring object of sound that could be manipulated into a new work of sound-art has indeed been a paradigm shift in the entire philosophy of music-making and listening. However, while the transliteration of sound-pressure waves to magnetic tape and rotating discs enabled this process to be realised with loudspeakers, another very significant and influential type of direct-to-medium concrete musical composition was already being undertaken with similar, though not electrically powered, technology.

The piano roll is a piece of sequencing technology that was primarily utilised in player pianos of the late nineteenth and early twentieth centuries. The remarkable invention was patented in 1883 by Emil Welte (Welte Reference Welte1883) and fulfilled a similar purpose to modern transistors – utilising low power signals to switch higher power ones, though using the power of air-flow rather than electrical current flow. The mechanisms consist of a roll of paper, comparable to a roll of magnetic tape, but instead of writing a signal via displacing magnetic fields, the signal was written by perforating the paper.

This concrete musical sequencing technology allowed composers such as Paul Hindemith, Ernst Toch and Gerhart Münch in the early twentieth century to radically expand the traditional role of the composer and entirely bypass the performer’s interpretation of the work, delivering an unadulterated, precise performance of the work exactly as the composer intended (Patteson Reference Patteson2015). Similar to electrical varieties of tape-music, these rolls of tape could be cut, spliced, reversed, slowed down, sped up, dynamics altered, auditioned, mixed and matched, and they enabled musical compositions that were far beyond anything that a human being sitting at a piano would be capable of performing.

Possibly the player piano’s most prolific composer was Conlon Nancarrow, who wrote over 50 studies specifically for the instrument from 1951 to 1993. Nancarrow’s work with this non-loudspeaker-based concrete music cannot be overstated. Indeed, his piano-roll-based studio composition paradigm is largely enduring to the present day, with all major packages of music-making software including piano-roll-inspired interfaces and workflows. Just as the audio waveform editor of a digital audio workstation is reminiscent of Schaeffer’s cut and splice composition workflow in his studio in Paris in 1950, the digital audio workstation’s piano-roll editor is equally descended from Nancarrow’s workflow in his Mexico City studio at the same time (Gann Reference Gann1995) (Figure 4).

There is also a very strong link between Nancarrow’s pioneering work and the first wave of musical roboticists. Trimpin was a great admirer of Nancarrow and forged a friendship with the composer, even building a machine to digitise Nancarrow’s paper piano rolls, converting his works into MIDI (Focke Reference Focke2011). Trimpin also named one of his large-scale electroacoustic installations Conloninpurple in honour of the late composer. Both Trimpin and Godfried Willem Raes created their own takes on the computer-controlled automatic piano, Contraption IPP 7512 (Focke Reference Focke2011) and Piano Vorsetzer #2 (Raes Reference Raes1994) respectively, which have allowed both artists to recreate and expand on Nancarrow’s work. In fact, Nancarrow’s final work before he passed away in 1997 was written for and dedicated to Trimpin’s Contraption IPP 7512 instrument (Focke Reference Focke2011). It is a testament to how ahead of its time his work was that it would not be difficult to imagine one of Nancarrow’s works fitting in seamlessly alongside contemporary electroacoustic pieces in a Disklavier, Contraption IPP 7512 or Vorsetzer-equipped concert in the present day. Expressive and enduring as the piano-roll paradigm has been, it too is based on the earlier methods of concrete composition explored below.

Figure 4 Player piano roll (left) and modern digital piano roll (right).

5. THE DISC ERA

Though the piano roll can be thought of as a pneumatic predecessor of magnetic tape, it was certainly not the earliest method of composing concrete music for musical automata. It is widely known that Schaeffer’s first musique concrète works for loudspeaker were realised using shellac records, a technology directly inspired by disc music boxes of the late nineteenth century (Ord-Hume Reference Ord-Hume1973). Though they did not suit a cut and splice composition workflow like rolls and tapes (and had a more limited playtime), disc music boxes were sophisticated instruments which used encoded punching on rotating discs to trigger musical events in a turntable-style player.

The discs and their players were essentially the height of musical automata development during the field’s golden age in the 1880 to 1920 period (Reblitz and Bowers Reference Reblitz and Bowers2001), by being the first form of reproducible music playback which was able to be mass-produced by machines in a factory setting. This dramatically reduced the cost and increased the supply of music discs to the public, allowing music that had been precisely programmed by the composer or arranger to be enjoyed by the listening public without third-party interpretation of the music being inserted.

This technology was directly responsible for a similarly convenient and cost-effective format being used for shellac and later vinyl records and the format still sees widespread use today (Figure 5). Indeed, the format that started with disc music boxes became instrumental to the development of hip-hop music, turntablism and electronic dance music culture. Despite the development of digital media and distribution, the record-player format is still being used as a musical storage and playback medium, with many of the contemporary non-loudspeaker-based albums cited above as the ‘state of the art’ available in the format.

Figure 5 Disc music box (left) and modern turntable (right).

6. THE 1000-YEAR SEQUENCE

Before the industrial revolution made precisely sequenced music available to the masses (Ord-Hume Reference Ord-Hume1973), musical sequences were reproduced by hand by skilled artisans in a format that is both the earliest known example of concrete musical sequences and one which is still used around the world today: the pinned-barrel mechanism. The mechanism works by precisely affixing pins or screws to a cylindrical barrel that is subsequently rotated. The rotating barrel is then positioned in order to make contact with an actuating mechanism, so that as the cylinder is rotated, the placed pins activate sound objects in the programmed sequence. The actuating mechanism may either be a musical instrument that the pins strike or pluck directly, or a set of levers that are connected to such actuators elsewhere by way of ropes and pulleys or other connections. Though in the twenty-first century pinned barrel mechanisms are predominantly found in toys, souvenirs and novelty items, they have an extremely rich history as a method of musical reproduction and creativity from the most common popular musics, to the highest art-musics of their times.

Throughout the 1800s, pinned-barrel mechanisms were used to drive all manners of musical automata, from large music boxes, portable barrel and pipe organs, through to large-scale orchestrions, equipped with string, percussion and wind instruments (Bowers Reference Bowers1972). In fact, the early Panharmonicon orchestrion built by Johann Nepomuk Malzel in 1804 was used by none other than Ludwig van Beethoven to create his 1813 Wellington’s Victory. There are also records of other well-known composers such as Haydn and Mozart writing music specifically for these types of automatic instruments (Flø and Wilmers Reference Flø and Wilmers2015).

While the portable tuned-comb music box was invented by Antoine Favre in Switzerland in 1796 (Nijsen Reference Nijsen1984), it was the result of hundreds of years of miniaturisation which began during the thirteenth century in Flanders. During that period, large pinned-barrel sequencers like the one illustrated in Figure 6 were used in bell towers throughout the Low Countries of Western Europe, and as the towers were integrated as instruments, they became known as carillons. Arrangers and composers working with these carillons were effectively using the same studio-compositional approaches as contemporary composers placing events in a DAW’s ‘piano roll’ timeline.

Figure 6 The pinned barrel mechanism as used in a carillon.

7. THE FIRST SEQUENCER

The earliest known device for creating concrete musical sequences is the Banu Musa programmable flute automaton (diagram of workings shown in Figure 7). This fascinating machine dates back to approximately 875ce and has broad implications across several disciplines. Its creation was described in the Book of Ingenious Devices (Shakir, Shakir, Shakir and Hill Reference Shakir, Shakir, Shakir and Hill1979), an extremely influential text which provided sketches and instructions for building around one hundred artefacts, devices, components, kinetic sculptures and automata. The book was written by three brothers Muhammad, Ahmad and al-Hasan, sons of Musa bin Shakir (the Banu Musa) during the ninth-century golden age of Arabic-Islamic science in Baghdad, drawing knowledge and inspiration from earlier texts translated from Greek into Arabic (Biggs and Karlsson Reference Biggs and Karlsson2010).

Figure 7 A diagram demonstrating the action of the Banu Musa programmable flute automaton.

The translated name of the water organ is ‘The instrument, which plays by itself’ and its concept was that it would be able to continuously play melodies as long as a steady current of water was supplied to it. From the perspective of an onlooker, the instrument looked like a model of a human flautist playing a flute-like instrument with its fingers. Behind the scenes, there were several mechanisms working together to achieve this effect. First, a hidden chamber inside the player utilised the flowing water to provide air pressure through the figurine’s mouth to the flute instrument. The flowing water also powered the rotation of the first pinned-barrel mechanism in existence. The pins on the barrel then triggered the movement of the figurine’s fingers over the flute by way of a number of strings and pulleys, thus allowing the first musical automat to play endless melodies.

While this discovery grants the Banu Musa flute automaton the title of the world’s first sequencer and first musical automaton, it is also the earliest programmable machine, and can be considered an ancestor of the first computers (Koetsier Reference Koetsier2001). This means that in an indirect way, it is also an ancestor to the field of computer music itself.

While there are earlier examples of water- and air-powered birds and flute players documented in ancient Chinese and Greek literature (Davies Reference Davies1979), the important contribution made by the Banu Musa was the pinned-barrel mechanism, which allowed programmable variation in the rhythm and melody of the music. In an English translation of parts of the manuscript, Farmer quotes the intention of the Banu Musa (Farmer Reference Farmer1931: 88):

These desires greatly reflect those cited by Schrader above, and many of the electroacoustic musicians mentioned in this article. Artists who utilise musical robots, sequencers and other pieces of technology to provide the flexibility, control and longevity required of their compositions and installations are in fact participating in a millennium-long struggle to achieve musical feats that are impossible or impractical for human performers alone, and to enhance their ability to musically express themselves with powered technology.

When ‘The instrument, which plays by itself’ performed its inaugural melodies, there must have been much celebration. It is not known, however, if the composers and listeners of the very first precisely sequenced automatic acoustic music truly appreciated the ramifications of their discovery, and could have imagined that it would be instrumental in shaping the music world to the present day, over a thousand years later.

8. REFLECTIONS

This article has been a tour from the very earliest examples of automatic musical instruments from the Islamic golden age, through the musical automata golden age to the modern fields of musical robotics and, more generally, electroacoustic music. We examined the compositional trends that followed the development of technology through formats from the rotating cylinder to the rotating disc, the piano roll to the magnetic tape and eventually the modern digital audio workstation. Doing so has demonstrated that regardless of the exact source of power and with or without loudspeakers, the concept of the concrete musical phrase as a compositional tool is one which has a deep heritage and has greatly influenced all manners of electroacoustic music to the present day, and will continue to do so in the future.

Viewing the history of organised sound through this lens can allow alternative ways of thinking about electroacoustic musical practice in this modern age, and provides contemporary composers and listeners with an additional perspective to the traditional electroacoustic canon. Furthermore, it imbues every form of contemporary sequenced music, from the acousmatic loudspeaker installation and the robotic Aeolian Cello to the simple elevator jingle, with a deep sense of heritage and significance.