Comments (2)

Leave a Reply | Trackback URL

1. Jim Plamondon says:

   December 9, 2009 at 7:11 pm

   Phil –

   Music-control interfaces are tricky, because they must control two separate things:
   - The set of notes currently being played, and
   - The sound of those notes.

   In most instruments’ interfaces, there’s a trade-off between polyphony (the number of notes that can be played at once) and expressive potential (the number of continuous variables the performer can apply to a given note). For example, a trumpet or sax player can change th timbre of each successive note via the embouchure, breath pressure, etc., but can only play one note at a time, whereas a piano player can play many notes at a time, but has very little control over those notes’ timbre. Players of electronic keyboar synths can chose among a wide variety of timbres, to but change the timbre of the keyboard’s notes in real time, they must stop playing notes with one (or both) hands to manipulate dials, sliders, or other such devices, thus temporarily trading away polyphony for expressive power.

   Touch screens such as the one showed in the video do not address this problem. At any given moment, each given finger-point can be devoted to controlling note-selection (increasing polyphony) or to controlling timbre (increasing expressive potential), but not both.

   One way to solve this puzzle is to make a keyboard-like instrument so small that is can be sttrapped to one forearm, much as an accordion is strapped to one’s chest. The keyboard would then be fixed in position relative to the performer’s fingers, no matter how the performer waved his/her arm around. Then, Wii-like internal motion sensors could be used to control timbre, pitch bend, and the like. This combination of a tiny keyboard instrument (using multi-touch screens, physical keys, or whatever) and internal motion sensors would thus provide both high polyphony and high expressive potential.

   For one possible example of such an instrument, see this video: http://www.youtube.com/watch?v=CYdFM97ybgA

   Thumtronics — the company that was developing the Thummer — unfortunately went bankrupt before a commercially-viable version of the Thummer was completed, so no Thummers will be forthcoming anytime soon. But the ideas are still “sound,” so to speak.

   Point being: Multi-touch screens don’t solve the fundamental problem of musical interfaces, which is the trade-off between polyphony and expressive potential.

   A deeper point, not addressed by any of the above, is the level of abstraction with which a given UI displays and controls musical information. Does it display and control pitches (frequencies), for example, or intervals between pitches?

   Here’s an example. Guido’s original staff denoted intervals, not pitches. That is, his interlocking system of hexachords, solmization, staff, and sight-singing was all based on a system similar to today’s “movable Do with a La-based minor.” Before they could sight-read a given piece, he had to give them the tonic pitch (and remind them of the piece’s mode). With that, they could then sight-read the piece in any given key (that is, with any given tonic pitch, in the given mode). See Pesce’s 1999 English translation of Guido’s writings if you doubt these assertions.

   Later, however, the staff was re-interpreted to denote pitches (frequencies) rather than intervals. our note-names, and fixed Do solfege, both reflect this same pitch-focused pevel of abstraction.

   This re-interpretation was great for instrumentalists, because it provided a one-to-one correspondence between “marks on a page and keys to press.” As Bach said, all you had to do was press the indicated keys at the indicated times.

   It was a disaster for vocalists, however; it robbed them of the tonal context which Guido’s higher-level abstraction provided them. Whereas formerly the staff denoted intervals, thereby enabling easy transposition to accomodate a scratchy throat’s lower range or a church apse’s different resonating frequency, now it denoted specific pitches, making on-the-fly transposition much more difficult.

   The promise of modern technology, then, two-fold: first, it can offer new solutions to the trade-off between polyphony and expressive potential. But at least as importantly, it can also enable both vocalists and instrumentalists the opportunity to recapture the benefits of displaying (and controlling) musical information at a higher level of abstraction — i.e., intervals, rather than pitches — thereby exposing a piece’s tonal content directly.

   IMHO.
2. Phil says:

   December 16, 2009 at 12:28 pm

   Jim.

   Thanks for your thought provoking comments. In answer to your main points I offer the following thoughts.

   While I agree that ‘most’ musical interfaces/instruments involve a certain amount of negotiation between polyphony and expressive potential, I think that a couple of other issues should be flagged up in this discussion.

   1. There are exceptions that prove the rule, namely the guitar and associated effects that with sensing technology, looping and MIDI synthesis can potentially create unlimited polyphony, unlimited timbres as well as more subtle envelope changes, dynamic shaping, etc.

   2. One could argue that this notion of the instrument is limited in its focus on one performer. To take a case in point, the Gamelan ‘orchestra’ is regarded as a single instrument played by many people. Could this not apply to technology as well.

   In addition, and to deal with your first main point, multi touch interfaces are not limited to a single axis like traditional musical keyboard designs. A piano keyboard is limited to pitch change which occur on the x axis. However, if a touch interface were set up such that the x axis were pitch and the y axis was timbre then the potential is clear. In fact one need not be limited to to discrete pitches or timbres as gradual change is very possible (of course at a fine grained level the changes will be discrete but if imperceptible to the human ear then this distinction is purely academic.

   To move on to consider your comments on the user interface display, I would like to draw a distinction between instruments and musical displays or if you permit ‘musical notation’ (here I am talking about any attempt to display musical information in a graphical format).

   You make an interesting point here and provide a key example in Guido from the specific perspective of the development of western staff notation. However, anyone who has attempted to transcribe or notate non-western musics, modern music and western folk or popular music styles will be all too aware of the limits of this form of notation.

   If we step outside this system for a moment then any talk of ‘control frequencies’ and ‘pitches’ I think becomes too fixed to be helpful. When considering electronic music do we want to be limited to 12 semitones and 8 octaves? With such a step away from staff notation the problem can be moved to how do we represent changes in frequency (and timbre, dynamic, etc) over time and at what resolution.

   I think the answer then depends upon our purpose: reproduction, analysis, preservation…? For example, if reproduction or ‘performance’ is our aim then we may be better positioned if we work on a notation based upon movement, much like dance notation; if you like a process rather than product based notation. Returning to your comments, this is something perhaps hinted at in the early Aquitanian system and Guido of Arezzo’s early intervalic system (before he added the pretty coloured lines .

   Thanks for keeping me thinking!