Studying Music is Difficult and Important: Challenges of Music Knowledge Representation

Studying Music is Difficult and Important:

Challenges of Music Knowledge Representation

—or,

Writing Poetry About Music is Like Dancing About Architecture

Donald Byrd

School of Informatics and Jacobs School of Music

Indiana University, Bloomington

Rev. 9 March 2009

In my view, music is one of the most complex, difficult to study, and important to study of all cultural phenomena—in fact, among the most complex, difficult, and important of any phenomena whatsoever. The following features of music in general make it so. Some of this applies particularly to computer processing of music, but most of it applies in any situation.

1. Music is an art. Therefore the composer/artist can use its elements any way they like—for example, to confound music-IR systems, as the amusing essay “Composing to Subvert Content Retrieval Engines” points out (Collins 2006). Obviously, few if any composers/artists have that goal in mind, but a great many—at least in some cultures, including ours— try to use its elements in original and interesting ways, not straightforward and conventional ways. This (among other things) makes content-based retrieval a great deal harder with music than with expository prose, the type of text that text-retrieval systems usually deal with and that music retrieval is usually compared to. Similarly, there is a story that Marc Chagall said, in response to criticism of his drawing by an art critic, “Of course I draw poorly. I like to draw poorly.” That is, in his art, Chagall had no intention of using the element of drawing the way it was ordinarily used. The story may be apocryphal, but the point is that a creative person can always find original ways to do whatever they do; that’s virtually the definition of creativity.

One of the implications of this observation is that we should not expect information retrieval of music to be much like retrieval, in the text domain, of prose; it’s more like retrieval of poetry, where the denotations of words as given in dictionaries may be less important than their connotations or even their sounds. But this phenomenon applies to many problems of music informatics, and, indeed, to many problems of doing anything at all with music. For example, in Byrd (2009), I show and briefly discuss some surprising examples of “rule violations” in music notation; some are purely graphical curiosities, but many strike much deeper.[1] And I have commented elsewhere (Byrd 1994) that “It is tempting to assume that the rules of such an elaborate and successful system as CMN [Conventional Music Notation] must be self-consistent. A big problem with this idea is that so many of the ‘rules’ are, necessarily, very nebulous… But if you try to make every rule as precise as possible, what you get is certainly not self-consistent.”

Obvious as it is, the fundamental difference between works of art and otherwise similar creative products that aren’t intended as art is often forgotten. One result is the well-known quotation of uncertain origin (it has been attributed to people from Clara Schumann to Frank Zappa), “Writing about music is like dancing about architecture.” This is certainly thought-provoking, but it’s not a good analogy because writing about music is generally intended primarily to convey information and only secondarily (if at all) as art. For the same reason, the common variation, “Talking about music is like dancing about architecture”, is an even worse analogy. Actually, writing [or talking] about music is like writing [or talking] about architecture. Of course, this formulation is not too exciting! Using arts throughout, we might put it like this instead: Writing poetry about music is like dancing about architecture. This is a more memorable statement, but really interesting only as a response to the misguided original.

2. Music is fundamentally a performing art, so that performances, symbolic representations of general performances (scores and performance parts), and symbolic representations of specific performances (transcriptions) all exist, and the relationships among them can be extremely subtle. (I say music is “fundamentally” a performing art to distinguish it from forms like poetry, which can be performed, i.e., read aloud by one person for the benefit of others, but rarely is.) To borrow terms from mathematics, the mapping is not one-to-one; it’s often many-to-many. For one thing, what is the correct or most authoritative version of a musical work? For Western classical music, it’s ordinarily a score, but there may be several that are equally authoritative, or it may be a performance—or a set of parts intended for performers. Bach’s B Minor Mass is a well-known example of the problems (Frans Wiering, personal communication, January 2009; see also the Wikipedia article “Mass in B Minor”). And as you go back in time, the correct interpretation of any written notation gets more and more difficult.[2] For most traditions other than Western classical music, the most authoritative version is almost always a recorded performance, but there may be many performances with equally strong credentials. It’s hard to see how one could decide what is the most authoritative version of the folk song Greensleeves. (Cf. Goodman 1976 or Talbot 2000.) Even for recent Western classical music, it’s by no means unknown for the composer to record a performance that disagrees significantly from their own preexisting score.

The closest text analog of these first two characteristics together is the play, especially the play in verse.

3. A tremendous amount of music, especially Western music, has complex synchronization requirements. This applies to most popular and radio and TV music as well as European art-tradition music. To my knowledge, it is not only unique to music among the performing arts and other physical activities of people (sports, etc.), but it may be more demanding than any other presentation of information intended for human consumption.

The text equivalent is something like explicitly synchronized speeches in plays, which indeed occur. But there is simply no comparison: even the few examples of these (e.g., Churchill 1982) are far less demanding in terms of synchronization than the vast majority of Western music of, say, the 14th century or later. (In some ways, a play in verse is more like a piece of music than one in prose, and plays in verse are an important category. But of course they almost never involve explicit synchronization.) True, circus acrobats, jugglers working together, etc., do feats that require split-second timing, but at nowhere near the rate of musicians—and never, as far as I know, involving as many simultaneous coordinated events.

4. Music involves “instruments”, very often in groups. This has several implications. (a) It makes arrangements and transcriptions of a given work possible. Of course, there are many different instruments, and therefore an enormous number of combinations. Enough of them are actually used that the Library of Congress Subject Headings include hundreds, if not thousands, of entries for different ensembles. (b) Versions of many works exist for players of different skill levels, mostly lower but often—for virtuosi who want to “show off”—higher than the original. (c) Music notation may represent the sounds to be produced or it may represent the actions the performers are to take to produce the sounds. Both are widely used. Conventional Western music notation largely represents sounds, though it has features (e,g., notation of artificial harmonics for string instruments) that represent actions. Tablature for guitar and similar instruments represents the players’ actions (“put a finger in this position on that string”).

5. Music is often combined with text, not only via singing but also in the cases of narration and background music, plus several usages in music-notation scores. As a result, the problems of handling music are in several ways a superset of the problems of handling text. For example, to use space efficiently, scores of orchestral works routinely show on a page only the instruments that actually play on that page; the instrument to play each staff is identified by a label at the left end of the staff (“flute 1”, “fagotti 1 e 2”, etc.). Therefore, to convert the score to symbolic form, an optical music recognition program must first perform optical character recognition (OCR) on the label strings.

6. Finally, music is extremely popular; in many cultures including ours, it is among the most popular arts. If this were not the case, perhaps other phenomena would be as complex and challenging. But the great popularity of music overall means that really popular works are likely to exist in many recordings and many scores, in arrangements for several different ensembles of instruments, at multiple levels of technical demands on the performers. As a result, music wins the “most challenging” contest hands down. It also means that handling music’s challenges is important in itself, even on purely economic grounds.

Implications for Computer Applications to Music

One of my own research interests is computer systems for music. For all the reasons above, it would be surprising if a computer system designed for music did not have features of great value to other domains. Of course this does not mean that we should expect every computer system for music actually to be useful in other areas. The trick is to keep general features of the system separate from music-specific features so the former can actually be used in nonmusic contexts, and few systems are designed that way.

Acknowledgement

The ideas herein about the quotation “writing [talking] about music is like dancing about architecture” originated in a 2001 e-mail conversation with Steve Larson. Thanks to him and to Douglas Hofstadter, analogy fanatic par excellence, who clarified some of my thinking about that quotation.

References

Byrd, Donald (1994). Music-Notation Software and Intelligence. Computer Music Journal 18, no. 1, pp. 17–20. Available at file:///Users/donbyrd/Documents/WebSiteDon/Papers/MusNotSoftware+Intelligence.pdf .

Byrd, Donald (2009). Gallery of Interesting Music Notation. Available at http://www.informatics.indiana.edu/donbyrd/InterestingMusicNotation.html .

Churchill, Caryl (1982). Top Girls. London: Methuen.

Collins, Nick (2006, Winter). Composing to Subvert Content Retrieval Engines. ICMA Array, pp. 37–41.

Goodman, Nelson (1976). Languages of Art: An Approach to a Theory of Symbols, 2nd ed. Indianapolis: Hackett Publishing.

Hook, Julian (2008). How to Perform Impossible Rhythms. Talk at Graduate Theory Association Symposium, Indiana University, Bloomington, Indiana, February 2008.

Talbot, Michael, ed. (2000). The Musical Work: Reality or Invention? Liverpool: Liverpool University Press.

---

[1] A particularly interesting one is an example from a Chopin Nocturne of what Julian Hook calls an “impossible rhythm”, where one double-stemmed notehead is a triplet eighth in one voice but a normal eighth in another and both end at the same point (the barline); therefore they must begin at different times in the two voices! Hook (2008) lists dozens of examples in the works of Brahms, Chopin, Rachmaninov, etc.

[2] Along the same lines, Nina Fales observes (personal communication, November 2008) that “among ethnomusicologists, a commonly recognized reality of fieldwork is that the further in time and distance a particular item of information travels from its source, the less likely it is to communicate its original significance.”