An instrument in the traditional sense controls sound in a very specific, fixed way. A piano has 88 keys, each of which produces one specific frequency at an amplitude determined by the player’s touch. A cello has four strings, with the placement of a player’s finger determining frequency and bow force determining amplitude. A theremin uses two metal antennae and the distance of a player’s hands from them to determine frequency and amplitude. All of these can be boiled down to control over these two fundamental attributes, frequency and amplitude.
In designing a modern software instrument, one has the flexibility of deciding exactly how to control each parameter, from the mode of interaction to the reactive constraints governing the input and output ranges. One could map almost any mode of interaction to sonic parameters, from buttons to sliders to tilting to anything else that can be defined by some form of input. One can even incorporate controls that execute control changes, such as adjusting the frequency range of a slider or changing its timbre.
I’ve previously talked about live algorithmic sound as a compositional medium. It also has direct ramifications to modern software instrument design, for two reasons. Firstly, algorithms themselves present as controllable parameters for performer control. Rather than simply controlling the immediate frequency and amplitude of one musical gesture, what if each input could control a range of frequencies, start and stop a certain behavior, adjust a process already in place?
Secondly, autonomous behavior ensures that any parameters not under the performer’s direct control can still continue behaving on their own. They can move as little or as much as a composer’s algorithmic design allows until a performer retakes control. A performer can at times become more of a passenger to a vast range of sonic processes in progress. A performer’s control becomes a compositional consideration, where the trajectory of a work can be determined by a performer’s degree of control or non-control over sound.
Early Exploits: Void
I present a very early work of mine as an example of the limits of simple parameter-centric instrument design. My work Void (2021) involves two light sensors and three buttons. Each button toggles a different partial of a fundamental frequency determined by the light sensor housed in the beer can. The pinkie of my right hand covers and uncovers a second light sensor, controlling the overall amplitude of the sound. Without any autonomous processes, the piece requires constant input. The work seems to be plateauing until I pick up the beer can halfway through to let in more light, and even then the sound remains fairly consistent in character. All of this seems fairly consistent with a traditional instrument – consistent sound profile, only sound when asked to, and invite the composer to exploit the instrument’s full potential by incorporating different techniques. For an electronic instrument, it falls short of demonstrating anything beyond what a traditional instrument could do.
A Fortuitous Revelation: //run with caution
I ended up exploiting autonomy somewhat on accident when I wrote //run with caution (2022). The source sound is the result of a catastrophic band pass filter glitch in SuperCollider that I recorded into a buffer for playback. The two light sensors involved in this work track the position of my hand, which determines the direction and speed of playback through the recorded sound. After setting it up, I realized that the sensitivity of the light sensor was way higher than I imagined, making the sound behave similarly erratically. But that in turn led me to realize that there’s so much potential behind behaviors that don’t require my input. I could steer the sound with one hand, with both hands, or take my hands away and let the sound just open up endlessly. In this case, it’s not even an algorithm, just some oversensitive light sensors picking up scattering ambient light, but it opened a revelation. The work started as a single sound controllable by me, but I realized that the sound pushes back. This struggle for control became the foundation for my current developments in interactive sound.
Control the Control: Phosphenes
My work Phosphenes (2024) is the result of treating control as a compositional parameter. The work is controlled only by the inertial sensors on the iPad. This imprecise, limited design keeps the performer from too much direct control and leaves plenty of room for the electronics to have their say. As such, I start with a very limited scope for my control. I tilt forward to gradually add partials to the background texture, and flick side to side to execute pings. I don’t control the frequency of the pings nor the flicker rate of the background texture; all of those I leave to algorithmic control.
As the work progresses, after hitting certain thresholds or requirements, such as tilting forward a certain amount, I hit checkpoints that adjust my controls, unlocking new features. What used to only be a fixed frequency ping turns into a wider random range. I still have some control over the primary sonic parameters of frequency and amplitude, as any traditional instrument would, but deliberately restrained for the most part in a balancing act with the autonomous electronics. The progression of the work, the large-scale form, is determined by my evolving control and balance with the algorithms. Controlling the control.
