Skip to main content
×
×
Home

Can Micro-Gestural Inflections Be Used to Improve the Soniculatory Effectiveness of Parameter Mapping Sonifications?

  • David Worrall (a1)
Abstract

Parameter mapping sonification is the most widely used technique for representing multi-dimensional data in sound. However, it is known to be unreliable when used for detecting information in some types of data. This is generally thought to be the result of the co-dependency of the psychoacoustic dimensions used in the mapping.

Positing its perceptual basis in a theory of embodied cognition, the most common approach to overcoming this limitation involves techniques that afford the interactive exploration of the data using gross body gestures. In some circumstances, such exploration is not possible and, even when it is, it may be neither necessary nor sufficient.

This article explores some other possible reasons for the unreliability of parameter mapping sonification and, drawing from the experience of expressive musical performance, suggests that the problem lies not in the parametric approach per se, nor in the lack of interactivity, but in the extent to which the parameters employed contribute to coherent gestalts. A method for how this might be achieved that relies on the use of micro-gestural information is proposed. While this is speculative, the use of such gestural inflections is well known in music performance, is supported by findings in neuroscience and lends itself to empirical testing.

Copyright
Corresponding author
E-mail: david.worrall@iis.fraunhofer.de
References
Hide All
Adorno, T.W. 1991. On the Fetish Character in Music and the Regression of Listening. In The Culture Industry, ed. J.M. Bernstein, London: Routledge.
Beyer, C. 2011. Edmund Husserl. In E.N. Zalta (ed.) The Stanford Encyclopedia of Philosophy. http://plato.stanford.edu/archives/win2011/entries/husserl.
Cary, T.O. 1992. Illustrated Compendium of Musical Technology. London: Faber and Faber.
Diniz, N., Demey, M., Leman, M. 2010. An Interactive Framework for Multilevel Sonification. In Proceedings of ISon 2010, 3rd Interactive Sonification Workshop, KTH, Stockholm, Sweden, 7 April.
Doornbusch, P. 2002. Composers’ Views on Mapping in Algorithmic Composition. Organised Sound 7(2): 145156.
Dreyfus, H. 1992. What Computers Still Can't Do. Cambridge, MA: The MIT Press.
Flowers, J.H. 2005. Thirteen Years of Reflection on Auditory Graphing: Promises, Pitfalls, and Potential New Directions. In Proceedings of the First Symposium on Auditory Graphs, Limerick, Ireland, 10 July.
Flowers, J.H., Buhman, D.C., Turnage, K.D. 1997. Cross-Modal Equivalence of Visual and Auditory Scatterplots for Exploring Bivariate Data Samples. Human Factors 39: 341351.
Frysinger, S.P. 2005. A Brief History of Auditory Data Representation to the 1980s. In Proceedings of the First Symposium on Auditory Graphs, Limerick, Ireland, 10 July.
Fyk, J. 1997. Intonational Protention in the Performance of Melodic Octaves on the Violin. In M. Leman (ed.) Music, Gestalt, and Computing: Studies in Cognitive and Systematic Musicology. Berlin: Springer.
Goehr, L. 1994. The Imaginary Museum of Musical Works. Oxford: Oxford University Press.
Godøy, R.I. 2006. Gestural-Sonorous Objects: Embodied Extensions of Schaeffer's Conceptual Apparatus. Organised Sound 11(2): 149157.
Godøy, R.I. 2010. Images of Sonic Objects. Organised Sound 15(1): 5462.
Godøy, R.I., Leman, M. (eds.) 2010. Musical Gestures: Sound, Movement, and Meaning. New York: Routledge.
Godøy, R.I., Jensenius, A.R., Voldsund, A., Glette, K., Høvin, M., Nymoen, K., Skogstad, S., Tørresen, J. 2012. Classifying Music-Related Actions. In Proceeding of the 12th International Conference on Music Perception and Cognition and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music, Thessaloniki, Greece, 23–28 July.
Grond, F., Berger, J. 2011. Parameter Mapping Sonification. In T. Hermann, A. Hunt and J.G. Neuhoff (eds.) The Sonification Handbook. Berlin: Logos.
Großhauser, T., Großekathöfer, U., Herman, T. 2010. New Sensors and Pattern Recognition Techniques for String Instruments. In Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME 2010), Sydney, Australia.
Hermann, T., Ritter, H. 2005. Model-Based Sonification Revisited: Authors’ Comments on Hermann and Ritter, ICAD 2002. In ACM Transactions on Applied Perception 2(4): 559–63.
Hermann, T., Krause, J., Ritter, H. 2002. Real-Time Control of Sonification Models with an Audio-Haptic Interface. In Proceedings of the International Conference on Auditory Display (ICAD 2002), International Community for Auditory Display, 82–86.
Hunt, A., Hermann, T. 2011. Interactive Sonification. In T. Hermann, A. Hunt and J.G. Neuhoff (eds.) The Sonification Handbook. Berlin: Logos.
Hunt, A, Wanderley, M.M. 2002. Mapping Performer Parameters to Synthesis Engines. Organised Sound 7(2): 97108.
Iazzetta, F. 2000. Meaning in Musical Gesture. In M.M. Wanderley and M. Battier (eds.) Trends in Gestural Control of Music. Paris: IRCAM.
Kane, B. 2007. L'Objet Sonore Maintenant: Pierre Schaeffer, Sound Objects and the Phenomenological Reduction. Organised Sound 12(1): 1524.
Kima, J.H., Demeyb, M., Moelantsb, D., Leman, M. 2010. Performance Micro-Gestures Related to Musical Expressiveness. In Proceedings of the 11th International conference on Music Perception and Cognition, Seattle, WA, USA, 23–27 August.
Kramer, G. 1994. Some Organizing Principles for Representing Data with Sound. In: G. Kramer (ed.), Auditory Display: Sonification, Audification, and Auditory Interfaces, Santa Fe Institute Studies in the Sciences of Complexity Proceedings 18. Reading, MA: Addison-Wesley.
Loui, P., Wessel, D.L., Hudson Kam, C.L. 2010. Humans Rapidly Learn Grammatical Structures in a New Scale. Music Perception 27(5) (June): 377388.
Mathews, M.V. 1969. The Technology of Computer Music. Cambridge, MA: The MIT Press.
McCorduck, P. 2004. Machines Who Think, 2nd ed. Natick, MA: A. K. Peters.
Miller, G.A. 1956. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. Psychological Review 63: 8197.
Moore, F.R. 1990. Elements of Computer Music. Englewood Cliffs, NJ: Prentice-Hall.
Nichols, C. 2002. The vBow: A Virtual Violin Bow Controller for Mapping Gesture to Synthesis with Haptic Feedback. Organised Sound 7(2): 215220.
Paine, G. 2009. Towards Unified Design Guidelines for New Interfaces for Musical Expression. Organised Sound 14(2): 143156.
Palombini, C.V. de L. 1993. Pierre Schaeffer.s typo-morphology of sonic objects. Doctoral thesis, Durham University. Durham E-Theses Online: http://etheses.dur.ac.uk/1191. Accessed on 1 May 2013.
Polansky, L., Childs, E. 2002. Manifestation and Sonification: The Science and Art of Sonification, Tufte's Visualization, and the ‘Slippery Slope’ to Algorithmic Composition. An Informal Response to Ed Childs’ Short Paper on Tufte and Sonification; with Additional Commentary by Childs. http://eamusic.dartmouth.edu/~larry/sonification.html. Accessed on 16 October 2012.
Scaletti, C. 1994. Sound Synthesis Algorithms for Auditory Data Representation. In: G. Kramer (ed.) Auditory Display: Sonification, Audification, and Auditory Interfaces. Santa Fe Institute Studies in the Sciences of Complexity Proceedings 18. Reading, MA: Addison-Wesley.
Schaeffer, P. 1966. Traité des objets musicaux: essai interdisciplines. Paris: Seuil.
Smalley, D. 1986. Spectro-Morphology and Structuring Processes. In S. Emmerson (ed.) The Language of Electroacoustic Music. London: Macmillan.
Smalley, D. 1992. The Listening Imagination: Listening in the Electroacoustic Era. In J. Payner, T. Howell, R. Orton and P. Seymour (eds.) Companion to Contemporary Musical Thought. London and New York: Routledge.
Smalley, D. 1997. Spectromorphology: Explaining Sound-Shapes. Organised Sound 2(2): 107126.
Smalley, D. 2007. Space-Form and the Acousmatic Image. Organised Sound 12(1): 107126.
Thoresen, L., Hedman, A. 2007. Spectromorphological Analysis of Sound Objects: An Adaptation of Pierre Schaeffer's Typomorphology. Organised Sound 12(2): 129141.
Todd, P.M, Loy, G. (eds.) 1991. Music and Connectionism. Cambridge, MA: The MIT Press.
Tuuri, K., Eerola, T. 2012. Formulating a Revised Taxonomy for Modes of Listening. Journal of New Music Research 41(2): 137152.
Tuuri, K., Mustonen, M., Pirhonen, A. 2007. Same Sound – Different Meanings: A Novel Scheme for Modes of Listening. In Proceedings of Audio Mostly, Ilmenau, Germany: Fraunhofer Institute for Digital Media Technology IDMT, 13–18.
Worrall, D. 2009a. An Introduction to Data Sonification. In R.T. Dean (ed.), The Oxford Handbook of Computer Music and Digital Sound Culture. Oxford: Oxford University Press.
Worrall, D. 2009b. Sonification: Concepts, Instruments and Techniques. PhD thesis, University of Canberra. Available at http://erl.canberra.edu.au/public/adt-AUC20090818.142345.
Worrall, D. 2010. Parameter Mapping Sonic Articulation and the Perceiving Body. In Proceeding of the 16th International Conference on Auditory Display, Washington, DC, USA, 9–15 June.
Worrall, D., Bylstra, M., Barrass, S., Dean, R. 2007. SoniPy: The Design of an Extendable Software Framework for Sonification Research and Auditory Display. In Proceedings of the 13th International Conference on Auditory Display, Montreal, Canada, 26–29 June.
Xenakis, I. 1971. Formalized Music: Thought and Mathematics in Music. Bloomington: Indiana University Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Organised Sound
  • ISSN: 1355-7718
  • EISSN: 1469-8153
  • URL: /core/journals/organised-sound
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed