2000 character limit reached
Sonification Abstraite/Sonification Concrète: An 'Aesthetic Perspective Space' for Classifying Auditory Displays in the Ars Musica Domain
Published 21 Nov 2013 in cs.HC | (1311.5426v1)
Abstract: This paper discusses {\ae}sthetic issues of sonifications and the relationships between sonification (ars informatica) and music & sound art (ars musica). It is posited that many sonifications have suffered from poor internal ecological validity which makes listening more difficult, thereby resulting in poorer data extraction and inference on the part of the listener. Lessons are drawn from the electroacoustic music and musique concr`ete communities as it is argued that it is not instructive to distinguish between sonifications and music/sound art.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview
This paper is about how to turn data into sound in ways that people can actually understand and enjoy listening to. This is called sonification. The authors argue that many past sonifications were hard to listen to because they didn’t think carefully about aesthetics (how things sound and feel). They suggest we should treat sonifications more like music and learn from musical practices to make data-as-sound clearer and less tiring.
Key Questions
- How can we design sonifications that are easier and more pleasant to listen to?
- Is there really a strict line between “sonification” (data sounds) and “music,” or are they two sides of the same thing?
- Can we classify sonifications and musical pieces together on a single “map” so we can borrow useful ideas from music to improve sonifications?
Approach and Key Ideas
The paper is a theory and design proposal, not a lab experiment. The authors build a simple mental “map” and use examples to show how thinking like a composer can improve sonifications.
What is sonification?
Sonification means turning numbers or events into sound so you can hear patterns. Think of it like turning a line graph into a melody, or converting earthquake waves into audio so you can hear the tremors.
- Example: A scientist “plays” seismograph data by shifting it into the audible range so we can hear quake patterns.
Two ways to map data to sound
- Direct mapping: One thing in the data equals one thing in the sound. For example, higher temperature = higher pitch. This is like a speedometer: the needle position directly shows the speed.
- Metaphorical (or analogical) mapping: You choose a sound that represents the idea, not the exact data shape. For example, a “jug filling” sound stands for a progress bar finishing. These are called auditory icons. Earcons are short musical “motifs” or tones used as symbols.
Ways of listening
We don’t always listen the same way:
- Everyday listening: You focus on what made the sound (big truck, glass breaking).
- Musical listening: You focus on the sound itself (pitch, rhythm, tone). In real life, we slide between these two. Good sonification designs respect this and help the listener get the right information with the right kind of listening.
Acoustic ecology (soundscapes)
A soundscape is like an environment’s “mix” of sounds that feel like they belong together. If a sound doesn’t fit, it sticks out. Sonifications work better when their sounds “live well together” and also fit into the real-world environment where people hear them (so they don’t annoy or overwhelm).
Music styles: musique concrète vs. abstract music
- Musique concrète: Music built from real-world recorded sounds (doors, waves, voices). It’s “concrete” because the sounds strongly remind you of their sources.
- Abstract music (musique abstraite): Music built from notes and instruments in a more “pure” way (melodies, chords), not tied to real-world sources.
This matters because:
- Direct sonifications are like musique concrète (the sound closely follows the data).
- Metaphorical sonifications are like abstract music (the sound is a designed symbol for the data).
The “Aesthetic Perspective Space” map
The authors first imagine a line (a continuum) from:
- Ars informatica (pure sonification) to
- Ars musica (pure music).
Then they suggest a better shape: a circle (or a sphere). The vertical axis is indexicality:
- Concrete/high-indexicality at the top: sounds that clearly resemble their source (or directly reflect the data).
- Abstract/low-indexicality at the bottom: sounds that are symbolic or stylized.
Placing sonifications and musical works on this circle lets you see which ones share similar “sound logic” even if one is called “music” and the other “data.”
The core idea: Switch your perspective, and a sonification can be heard as music; switch again, and a piece of music can be heard as a sonification of some process. This shared space helps designers borrow proven musical techniques to make sonifications clearer.
Main Findings and Why They Matter
- Sonification and music are not opposites. They are deeply related. Whether you hear “data” or “music” often depends on your perspective and purpose.
- Many sonifications fail because their internal “sound world” is messy. Better aesthetics (clearer timbres, good balance, smart spatial placement in stereo or surround) reduce fatigue and make patterns easier to notice.
- Direct vs. metaphorical mappings in sonification line up with concrete vs. abstract approaches in music. Knowing this match-up helps you pick the right style for your data and audience.
- Bringing in musically trained people improves sonification quality. Their skills in organizing sound, avoiding masking, shaping timbre, and pacing can make the data easier to hear and understand.
- User preferences matter. Letting people choose sound sets or “ecologies” that fit their culture or taste can improve comprehension and comfort.
These points matter because sonifications are often used for monitoring or analyzing complex systems (like servers, markets, or scientific data). If the sounds are unpleasant or confusing, people miss important changes or get tired.
Implications and Potential Impact
If designers treat sonifications as part of the musical world, they can:
- Classify a sonification on the Aesthetic Perspective Space, find a musically similar “neighbor,” and borrow its organizing tricks.
- Design soundscapes that are easy to live with over long periods, reducing annoyance and listener fatigue.
- Choose mappings and timbres that reveal the data without distracting “ear candy,” striking a balance between clarity and beauty.
- Collaborate with composers and sound designers to raise the overall quality and usefulness of auditory displays.
- Help more people understand complex data by listening, not just by looking—especially in situations where eyes are busy or screens are overloaded.
In short, the paper encourages a mindset shift: stop asking “Is it music or data?” and start asking “How can we make this sound world clear, meaningful, and pleasant enough that people actually hear what the data is saying?”
Glossary
- Acoustic ecology: The study of relationships between organisms and their sonic environments, and in this context, the internal ecology of sounds within a sonification. "Gaver's work was strongly motivated by R. Murray Schafer's concept of acoustic ecology in soundscapes (see \cite{Schafer:1977})."
- Æsthetic Computing: An approach applying art theory and practice to design computing systems, emphasizing cultural and personal aesthetics. "Fishwick \cite{Fishwick:2002,Fishwick:2006} coined the term Æsthetic Computing to refer to the application of art theory and practice to the design of computing systems."
- Æsthetic interrogation: The analysis of compositional techniques and aesthetic properties to inform sonification design. "This enables an æsthetic interrogation of the different compositional techniques to elicit the fundamental organising principles and æsthetic properties of various ars musica genres."
- Æsthetic Perspective Space: A proposed space mapping sonifications and musical compositions along dimensions such as indexicality and perspective. "Therefore, an æsthetic perspective space onto which sonifications and musical compositions alike can be mapped is proposed."
- Aleatoric compositions: Music where elements are left to chance or performer choice, often associated with John Cage. "John Cage's aleatoric compositions (e.g. Music of Changes (1951))"
- Analogic–symbolic continuum: A classification spectrum for auditory displays from analogic (mimetic) to symbolic representations. "Kramer \cite{Kramer:1994b} uses an analogic--symbolic continuum to classify audtory displays."
- Ars informatica: The domain of sonification and data-to-sound mapping as informational practice. "To move us towards how to think about the relationship between sonification and music æsthetics we may imagine a line, a continuum, with sonification (or, ars informatica) at one end"
- Ars musica: The domain of music and sound art as artistic practice. "To move us towards how to think about the relationship between sonification and music æsthetics we may imagine a line, a continuum, with sonification (or, ars informatica) at one end and music and sound art (ars musica) at the other"
- Auditory display: The field and practice of representing data via sound for analysis or monitoring. "sonification is generally used as a catch-all to describe most work in the area of auditory display."
- Auditory icons: Everyday sounds used to represent events or states in interfaces, leveraging mimetic associations. "Gaver's auditory icons \cite{Gaver:1989} are also interpretive but unlike earcons, auditory icons take a symbolic or representational (mimetic) approach."
- Auditory seismograms: Direct sonifications of seismic data scaled into audible frequencies for listening analysis. "Hayward's auditory seismograms are an example of the former"
- Auralisation: Rendering data or models into sound for analysis; akin to visualization but in the audio domain. "Much current art music would not be appropriate for an auralisation system."
- Automatic gain control (AGC): A signal processing technique that automatically adjusts audio amplitude to maintain consistent levels. "amplitude scaled (using automatic gain control) until they lay in the human audible range."
- Binaural recordings: Audio recorded with two microphones (often in a head-like setup) to reproduce 3D spatial hearing over headphones. "Binaural recordings of all ten works, and the EEG data on which they are based, are available to download at \url{www.icad.org/websiteV2.0/Conferences/ICAD2004/concert.htm}"
- Causal Listening: Listening mode focused on identifying the source or cause of a sound. "His `everyday listening' is very close to Chion's Causal Listening \cite{Chion:1994}"
- Chaotic attractor: A set toward which a dynamical system evolves, used here as data for musical/sonic mapping. "Mayer-Kress et al's \cite{Mayer-Kress:1994} sonification of a chua circuit chaotic attractor function"
- Chua circuit: A simple electronic circuit exhibiting chaotic behavior, often used in nonlinear dynamics studies. "Mayer-Kress et al's \cite{Mayer-Kress:1994} sonification of a chua circuit chaotic attractor function"
- DC removal: Signal processing step that eliminates the direct current (zero-frequency) component from a signal. "pre-processed with amplitude scaling, DC removal, and interpolation"
- Earcons: Structured musical motifs used as abstract auditory messages in interfaces. "Through the use of earcons they represented changes in fluid state which would be hard to spot in a graphical display."
- Electroacoustic music: Music that uses electronic technology to manipulate recorded or synthetic sounds, often beyond traditional tonality. "Lessons are drawn from the electroacoustic music and musique concrète communities"
- Everyday listening: Listening mode focusing on attributes of the sound source rather than the sound’s parameters. "Gaver \cite{Gaver:1989} proposed a theory of `everyday listening'"
- Gesture (in sound): Perceived motion and physical action encoded in sonic events, conveying meaning beyond pitch/harmony. "given their dependence upon the notion of `gesture' encoded into sounds."
- Indexical qualities: Source-indicating characteristics of sounds that link them to their causes or contexts. "everyday sounds, with their indexical qualities, become musical"
- Indexicality: The degree to which a sound resembles or points to the source that produced it. "The continuum has a second dimension, the degree of indexicality, that is, how strongly a sound sounds like the thing that made it."
- Language Grid (Emmerson): A framework for analyzing syntactic abstraction in music and electroacoustic works. "use structures such as Emmerson's {\it Language Grid} \cite{Emmerson:1986}"
- Metaphoric mappings: Non-literal data-to-sound mappings that use interpretive functions to represent information. "Blattner, Greenberg, and Kamegai \cite{Blattner:1992} used metaphoric mappings in their work on the auditory representation of turbulence in fluid flow."
- Mimetic (in sonification): Sound organization that imitates or reflects the qualities of the data’s source. "sonifications organise sound to reflect mimetically the thing being sonified."
- Musical listening: Listening mode focused on sound parameters such as pitch and intensity rather than source attributes. "Gaver's `musical listening' resembles Schaeffer's écoute réduite or Reduced Listening"
- Musique abstraite: Traditional score-based music where composition precedes performance by players, contrasted with concrète. "This traditional way of compusing music is known as musique abstraite."
- Musique concrète: Composition using recorded “found” or environmental sounds assembled into musical pieces. "principles of Pierre Schaeffer's musique concrète"
- Quantisation: Mapping continuous data values into discrete steps for representation or processing. "direct mappings impose a one-to-one relationship between data items and sonic events (possibly involving some scaling and quantisation)"
- Reduced Listening: Schaeffer’s mode of listening that focuses on the sound itself, independent of source or cause. "Gaver's `musical listening' resembles Schaeffer's écoute réduite or Reduced Listening"
- Semantic Listening: Listening mode aimed at extracting the informational content communicated by sounds. "Of particular interest here is Chion's Semantic Listening in which the listener seeks to gain information about what the sound is communicating"
- Serialism (Schoenbergian): Atonal compositional method organizing musical elements into series, often twelve-tone. "atonal (Schoenbergian serialism) systems makes them less well suited as carriers of information."
- Sound art: Artistic practice where sound is the primary medium, overlapping with experimental music and installations. "sonification (ars informatica) and music {paper_content} sound art (ars musica)."
- Soundscape: The acoustic environment regarded as a field of study or composition. "we treat the sonification both as a real-world soundscape in its own right"
- Spectro-morphology: Analytical/creative concept describing sound shapes and spectral evolution over time. "conventional pitched tones are only a subset of spectro-morphologies within a much broader world of spectra \cite{Smalley:1986}."
- Spatialisation: The placement and movement of sounds in space using mono, stereo, or multichannel techniques. "Careful thought needs to be given to the spatialisation of the sounds -- mono, stereo, and multi-channel sound can all be used to good effect but only if used well."
- Timbre: The quality or color of a sound that differentiates sources with the same pitch and loudness. "they ``\ldots let the sonification's timbre and duration represent the information and avoid recognizable musical patterns''"
- Tonal music: Music organized around hierarchical pitch relationships and harmonic centers. "melodies following the rules of western tonal music are easier to learn, organise cognitively, and discriminate than control tone sequences of similar complexity."
- Technological Listening: One of Schaeffer’s four modes, focusing on technical aspects of sound production and recording. "The four being Causal Listening, Reduced Listening, Semantic Listening, and Technological Listening."
Collections
Sign up for free to add this paper to one or more collections.