Skip to main content Accessibility help
×
Home
Hostname: page-component-558cb97cc8-zcgq2 Total loading time: 0.644 Render date: 2022-10-07T15:17:06.022Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "displayNetworkTab": true, "displayNetworkMapGraph": false, "useSa": true } hasContentIssue true

Fostering Social Interaction Through Sound Feedback: Sentire

Published online by Cambridge University Press:  09 February 2022

Marta Rizzonelli*
Affiliation:
Department of Musicology and Media Studies, Humboldt University of Berlin, Germany
Jin Hyun Kim*
Affiliation:
Department of Musicology and Media Studies, Humboldt University of Berlin, Germany
Pascal Staudt*
Affiliation:
Department of Musicology and Media Studies, Humboldt University of Berlin, Germany
Marcello Lussana*
Affiliation:
Department of Musicology and Media Studies, Humboldt University of Berlin, Germany

Abstract

Sentire is a body–machine interface that sonifies motor behaviour in real time and a participatory, interactive performance in which two people use their physical movements to collaboratively create sound while constantly being influenced by the results. Based on our informal observation that basal social behaviours emerge during Sentire performances, the present article investigates our principal hypothesis that Sentire can foster basic mechanisms underlying non-verbal social interaction. We illustrate how coordination serves as a crucial basic mechanism for social interaction, and consider how it is addressed by various therapeutic approaches, including therapeutic use of real-time auditory feedback. Then we argue that the implementation of Sentire may be fruitful in healthcare contexts and in promoting general well-being. We describe how the Sentire system has been developed further within the scope of the research project ‘Social interaction through sound feedback–Sentire’ that combines human–computer interaction, sound design and real-world research, against the background of the relationship between sound, sociality and therapy. The question concerning how interaction is facilitated through Sentire is addressed through the first results of behavioural analysis using structured observation, which allows for a quasi-quantitative sequential analysis of interactive behaviour.

Type
Article
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Arfib, D., Couturier, J. M. and Kessous, L. 2005. Expressiveness and Digital Musical Instrument Design. Journal of New Music Research 34(1): 125–36.CrossRefGoogle Scholar
Bakeman, R. and Quera, V. 2011. Sequential Analysis and Observational Methods for the Behavioral Sciences. New York: Cambridge University Press.CrossRefGoogle Scholar
Blaine, T. and Fels, S. [2003] 2017. Contexts of Collaborative Musical Experiences. In A. Jensenius, M. Lyons (eds.) A NIME Reader. Current Research in Systematic Musicology, vol. 3. Cham: Springer, 7187.Google Scholar
Buhmann, J., Desmet, F., Moens, B., Van Dyck, E. and Leman, M. 2016. Spontaneous Velocity Effect of Musical Expression on Self-paced Walking. PLoS ONE 11(5): e0154414. https://doi.org/10.1371/journal.pone.0154414.CrossRefGoogle ScholarPubMed
Burgoon, J. K., Dillman, L. and Stern, L. A. 1993. Adaptation in Dyadic Interaction: Defining and Operationalizing Patterns of Reciprocity and Compensation. Communication Theory 3(4): 295316.CrossRefGoogle Scholar
Casadio, M., Ranganathan, R. and Mussa-Ivaldi, F. A. 2012. The Body-machine Interface: A New Perspective on an Old Theme. Journal of Motor Behavior 44(6): 419–33.CrossRefGoogle Scholar
Clayton, M., Jakubowski, K. and Eerola, T. 2019. Interpersonal Entrainment in Indian Instrumental Music Performance: Synchronization and Movement Coordination Relate to Tempo, Dynamics, Metrical and Cadential structure. Musicae Scientiae 23(3): 304–31.CrossRefGoogle Scholar
Clayton, M., Sager, R. and Will, U. 2005. In Time with the Music: The Concept of Entrainment and Its Significance for Ethnomusicology. European Meetings in Ethnomusicology 11: 3142.Google Scholar
Cohen, J. 1960. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20(1): 3746.CrossRefGoogle Scholar
Cook, P. [2001] 2017. Principles for Designing Computer Music Controllers. In A. Jensenius, M. Lyons (eds.) A NIME Reader. Current Research in Systematic Musicology, vol. 3. Cham: Springer, 113.Google Scholar
Cross, I. 2007. Music and Cognitive Evolution. In Barrett, L. and Dunbar, R. (eds.) The Oxford Handbook of Evolutionary Psychology. Oxford: Oxford University Press, 649–67.Google Scholar
Cross, I. 2009. The Evolutionary Nature of Musical Meaning. Musicae Scientiae 13(2): 179200.CrossRefGoogle Scholar
Cross, I. 2014. Music and Communication in Music Psychology. Psychology of Music 42(6): 809–19.CrossRefGoogle Scholar
Cross, I., Laurence, F. and Rabinowitch, T. 2012. Empathy and Creativity in Group Musical Practices: Towards a Concept of Empathic Creativity. In G. E. McPherson, G. F. Welch (eds.) The Oxford Handbook of Music Educationi, vol. 2, 2nd edn. New York: Oxford University Press, 337–54.Google Scholar
Cross, I. and Woodruff, G. E. 2009. Music as a Communicative Medium. In Botha, R. and Knight, C. (eds.) The Prehistory of Language, vol. 1. Oxford: Oxford University Press, 113–44.Google Scholar
De Jaegher, H. and Di Paolo, E. 2007. Participatory Sense-making: An Enactive Approach to Social Cognition. Phenomenology and the Cognitive Sciences 6(4): 485507.CrossRefGoogle Scholar
Dyer, J., Stapleton, P. and Rodger, M. 2017. Advantages of Melodic over Rhythmic Movement Sonification in Bimanual Motor Skill Learning. Experimental Brain Research 235: 3129–40.CrossRefGoogle ScholarPubMed
Effenberg, A. O., Fehse, U., Schmitz, G., Krueger, B. and Mechling, H. 2016. Movement Sonification: Effects on Motor Learning Beyond Rhythmic Adjustments. Frontiers in Neuroscience 10(219): 118.CrossRefGoogle ScholarPubMed
Evola, V., Skubisz, J. and Fernandes, C. 2015. The Role of Eye Gaze and Body Movements in Turn-Taking during a Contemporary Dance Improvisation. Proceedings from the 3rd European Symposium on Multimodal Communication. Dublin: MMSYM 2015, 2431.Google Scholar
Feltham, F., Loke, L., van den Hoven, E., Hannam, J. and Bongers, B. 2014. The Slow Floor: Increasing Creative Agency While Walking on an Interactive Surface. Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI ’14). Munich: ACM, 105–12.Google Scholar
Françoise, J., Candau, Y., Alaoui, S. and Schiphorst, T. 2017. Designing for Kinesthetic Awareness: Revealing User Experiences through Second-Person Inquiry. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Denver: CHI ’17, 5171–83.Google Scholar
Fuchs, T. and De Jaegher, H. 2009. Enactive Intersubjectivity: Participatory Sense-making and Mutual Incorporation. Phenomenology and the Cognitive Sciences 8: 465–86.CrossRefGoogle Scholar
Gill, S. 2015. Tacit Engagement. Beyond Interaction. Heidelberg: Springer.CrossRefGoogle Scholar
Gurevich, M. and von Muehlen, S. 2001. The Accordiatron: A MIDI Controller for Interactive Music. Proceedings of the 2001 International Conference on New Interfaces for Musical Expression. Seattle: NIME, 27–9.Google Scholar
Hall, E. T. 1966. The Hidden Dimension. New York: Garden City.Google Scholar
Halm, A. 1978. Musikalische Bildung. In Schmalzriedt, S. (ed.) Von Form und Sinn der Musik. Wiesbaden: Breitkopf und Härtel, 211–27.Google Scholar
Hanslick, E. 1854. Vom Musikalisch-Schönen. Ein Beitrag zur Revision der Ästhetik der Tonkunst. Leipzig: Rudolph Weigel.Google Scholar
Hobeika, L. 2017. Interplay between Multisensory Integration and Social Interaction in Auditory Space: Towards an Integrative Neuroscience Approach of Proxemics. Paris: Université Paris Descartes.Google Scholar
Hove, M. J., Suzuki, K., Uchitomi, H., Orimo, S. and Miyake, Y. 2012. Interactive Rhythmic Auditory Stimulation Reinstates Natural 1/f Timing in Gait of Parkinson’s Patients. PLoS ONE 7(3): e32600. https://doi.org/10.1371/journal.pone.0032600 CrossRefGoogle ScholarPubMed
Hunt, A., Wanderley, M. M., Paradis, M. 2002. The Importance of Parameter Mapping in Electronic Instrument Design. Journal of New Music Research 32(4): 429–40.CrossRefGoogle Scholar
Jaffe, J. and Feldstein, S. 1970. Rhythms of Dialogue. New York: Academic Press.Google Scholar
Keller, P., Novembre, G. and Hove, M. J. 2014. Rhythm in Joint Action: Psychological and Neurophysiological Mechanisms for Real-Time Interpersonal Coordination. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences 369(1658): 20130394. https://doi.org/10.1098/rstb.2013.0394 CrossRefGoogle ScholarPubMed
Kestenberg, J. S. and Sossin, K. M. 1973. The Role of Movement Patterns in Development, vol. 1. New York: Dance Notation Bureau Press.Google Scholar
Kim, J. H. 2013. Shaping and Co-Shaping Forms of Vitality in Music: Beyond Cognitivist and Emotivist Approaches to Musical Expressiveness. Empirical Musicology Review 8(3–4): 162–73.CrossRefGoogle Scholar
Kim, J. H., Chang, S. H., Schmidt, L., Otto, J. A., Buch, B., Seifert, U. and Coussement, P. 2010. ‘playing_robot’: An Investigation of Situated Cognition in the Context of (Artistic) Human-Robot Interaction Design. Proceedings of IROS 2010 Workshop on Robotics and Musical Expressions. Taipei: IWRME-2010, 65–72.Google Scholar
Kim, J. H., Reifgerst, A. and Rizzonelli, M. 2019. Musical Social Entrainment. Music & Science 2: 117.CrossRefGoogle Scholar
Koch, S. C. 2014. Rhythm Is It: Effects of Dynamic Body Feedback on Affect and Attitudes. Frontiers in Psychology 5: Article 537. https://doi.org/10.3389/fpsyg.2014.00537 Google Scholar
Kurth, E. 1931. Musikpsychologie. Berlin: M. Hesse.Google Scholar
Livingstone, D. and Miranda, E. 2004. Composition for Ubiquitous Responsive Sound Environments. Proceedings of the 2004 30th Annual International Computer Music Conference. Miami: ICMC.Google Scholar
Magnusson, T. 2010. Designing Constraints: Composing and Performing with Digital Musical Systems. Computer Music Journal 34(4): 6273.CrossRefGoogle Scholar
Malloch, S. and Trevarthen, C. 2009. Musicality: Communicating the Vitality and Interests of life. In Malloch, S. and Trevarthen, C. (eds.) Communicative Musicality: Exploring the Basis of Human Companionship. Oxford: Oxford University Press, 111.Google Scholar
McArthur, J. A. 2016. Digital Proxemics. Bern: Peter Lang.CrossRefGoogle Scholar
McGrath, J. E. and Kelly, J. R. 1986. Time and Human Interaction: Toward a Social Psychology of Time. New York: Guilford Press.Google Scholar
McHugh, M. L. 2012. Interrater Reliability: The Kappa Statistic. Biochemia Medica 22(3): 276–82.CrossRefGoogle ScholarPubMed
Mersmann, H. 1925. Zur Phänomenologie der Musik. Zeitschrift für Aesthetik und allgemeine Kunstwissenschaft 12: 372–97.Google Scholar
Pellecchia, M., Beidas, R. S., Mandell, D. S., Cannuscio, C. C., Dunst, C. J. and Stahmer, A. C. 2020. Parent Empowerment and Coaching in Early Intervention: Study Protocol for a Feasibility Study. Pilot and Feasibility Studies 6(22): 112.CrossRefGoogle ScholarPubMed
Pikovsky, A., Rosenblum, M. and Kurths, J. 2001. Synchronization: A Universal Concept in Nonlinear Sciences. Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
Rabinowitch, T., Cross, I. and Burnard, P. 2012. Long-term Musical Group Interaction Has a Positive Influence on Empathy in Children. Psychology of Music 41(4): 484–98.CrossRefGoogle Scholar
Rizzonelli, M., Kim, J. H., Gladow, T. and Mainka, S. 2017. Musical Stimulation with Feedback in Gait Training for Parkinson’s Disease. Psychomusicology: Music, Mind, and Brain 27(3): 213–18.CrossRefGoogle Scholar
Robson, C. and McCartan, K. 2016. Real World Research. Chichester: Wiley.Google Scholar
Robson, D. 2002. Play!: Sound Toys for Non-Musicians. Computer Music Journal 26(3): 5061.CrossRefGoogle Scholar
Rose, D., Delevoye-Turrell, Y., Ott, L., Annett, L. E. and Lovatt, P. J. 2019. Music and Metronomes Differentially Impact Motor Timing in People with and without Parkinson’s Disease: Effects of Slow, Medium, and Fast Tempi on Entrainment and Synchronization Performances in Finger Tapping, Toe Tapping, and Stepping on the Spot Tasks. Parkinson’s Disease 2019: Article 6530838. https://doi.org/10.1155/2019/6530838 Google ScholarPubMed
Rosenbaum, D. A. 2010. Human Motor Control. Amsterdam and Boston, MA: Elsevier.Google Scholar
Rothfarb, L. 2002. Energetics. In Christensen, T. (ed.) The Cambridge History of Western Music Theory. Cambridge: Cambridge University Press, 927–55.CrossRefGoogle Scholar
Rovan, J. B., Wanderley, M. M., Dubnov, S. and Depalle, P. 1997. Instrumental Gestural Mapping Strategies as Expressivity Determinants in Computer Music Performance. Proceedings of the 1997 AIMI International Workshop. Kansei, The Technology of Emotion. Genoa: AIMI, 68–73.Google Scholar
Samaritter, R. and Payne, H. 2013. Kinaesthetic Intersubjectivity: A Dance Informed Contribution to Self-other Relatedness and Shared Experience in Non-verbal Psychotherapy with an Example from Autism. The Arts in Psychotherapy 40(1): 143–50.CrossRefGoogle Scholar
Searle, J. 1990. Collective Intentions and Actions. In Cohen, P. R., Morgan, J. and Pollack, M. E. (eds.) Intentions in Communication. Cambridge, MA: MIT Press, 401–15.Google Scholar
Searle, J. 2010. Making the Social World: The Structure of Human Civilization. Oxford: Oxford University Press.CrossRefGoogle Scholar
Seifert, U. and Kim, J. H. 2008. Towards a Conceptual Framework and an Empirical Methodology in Research on Artistic Human-Computer and Human-Robot Interaction. In Pavlidis, I. (ed.) Human-Computer Interaction. Vienna: In-Tech, 177–94.Google Scholar
Sigrist, R., Rauter, G., Riener, R. and Wolf, P. 2013. Augmented Visual, Auditory, Haptic, and Multimodal Feedback in Motor Learning: A Review. Psychonomic Bulletin & Review 20(1): 2153.CrossRefGoogle ScholarPubMed
Stern, D. N. 1985. The Interpersonal World of the Infant. A View from Psychoanalysis and Developmental Psychology. London: Karnac Books.Google Scholar
Stern, D. N. 2004. The Present Moment in Psychotherapy and Everyday Life. New York: W. W. Norton.Google Scholar
Stern, D. N. 2010. Forms of Vitality: Exploring Dynamic Experience in Psychology, the Arts, Psychotherapy, and Development. New York: Oxford University Press.CrossRefGoogle Scholar
Stern, D. N., Hofer, L., Haft, W. and Dore, J. 1985. Affect Attunement: The Sharing of Feeling States Between Mother and Infant by Means of Intermodal Fluency. In Field, T. N. and Fox, N. (eds.) Social Perception in Infants. Norwood, NJ: Ablex, 249–68.Google Scholar
Stuart, S. 2012. Enkinaesthesia: The Essential Sensuous Background for Co-agency. In Radman, Z. (ed.) The Background: Knowing Without Thinking. Palgrave Macmillan, 167–86.CrossRefGoogle Scholar
Stupacher, J., Wood, G. and Witte, M. 2017. Synchrony and Sympathy: Social Entrainment with Music Compared to a Metronome. Psychomusicology: Music, Mind, and Brain 27(3): 158–66.CrossRefGoogle Scholar
Styns, F., Van Noorden, L., Moelants, D. and Leman, M. 2007. Walking on Music. Human Movement Science 26(5): 769–85.CrossRefGoogle ScholarPubMed
Tanaka, A. and Knapp, R. B. 2002. Multimodal Interaction in Music Using the Electromyogram and Relative Position Sensing. Proceedings of the 2nd International Conference on New Interfaces for Musical Expression. Dublin: NIME02, 43–8.Google Scholar
Thaut, M. H., McIntosh, G. C., Rice, R. R., Miller, R. A., Rathbun, J. and Brault, J. M. 1996. Rhythmic Auditory Stimulation in Gait Training for Parkinson’s Disease Patients. Movement Disorders 11(2): 193200.CrossRefGoogle ScholarPubMed
Thaut, M. H., Rathbun, J. A. and Miller, R. A. 1997. Music Versus Metronome Timekeeper in a Rhythmic Motor Task. International Journal of Arts Medicine 5(1): 412.Google Scholar
Wittmann, M. and Pöppel, E. 1999/2000. Temporal Mechanisms of the Brain as Fundamentals of Communication – With Special Reference to Music Perception and Performance. Musicae Scientiae 3(1_suppl): 1328.CrossRefGoogle Scholar
Woolhouse, M. H., Tidhar, D. and Cross, I. 2016. Effects on Inter-Personal Memory of Dancing in Time with Others. Frontiers in Psychology 7: Article 167. https://doi.org/10.3389/fpsyg.2016.00167 CrossRefGoogle Scholar
Zappi, V. and McPherson, A. 2015. The D-Box: How to Rethink a Digital Musical Instrument. Proceedings of the 21st International Symposium on Electronic Art. Vancouver: ISEA.Google Scholar
Zuckerkandl, V. 1956. Sound and Symboli, trans. Trask, W. R.. New York: Pantheon Books.Google Scholar

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Fostering Social Interaction Through Sound Feedback: Sentire
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Fostering Social Interaction Through Sound Feedback: Sentire
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Fostering Social Interaction Through Sound Feedback: Sentire
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *