Book contents
- Frontmatter
- Contents
- Contributors
- 1 Introduction: Social Signal Processing
- Part I Conceptual Models of Social Signals
- Part II Machine Analysis of Social Signals
- Part III Machine Synthesis of Social Signals
- 19 Speech Synthesis: State of the Art and Challenges for the Future
- 20 Body Movements Generation for Virtual Characters and Social Robots
- 21 Approach and Dominance as Social Signals for Affective Interfaces
- 22 Virtual Reality and Prosocial Behavior
- 23 Social Signal Processing in Social Robotics
- Part IV Applications of Social Signal Processing
- References
21 - Approach and Dominance as Social Signals for Affective Interfaces
from Part III - Machine Synthesis of Social Signals
Published online by Cambridge University Press: 13 July 2017
Book contents
- Frontmatter
- Contents
- Contributors
- 1 Introduction: Social Signal Processing
- Part I Conceptual Models of Social Signals
- Part II Machine Analysis of Social Signals
- Part III Machine Synthesis of Social Signals
- 19 Speech Synthesis: State of the Art and Challenges for the Future
- 20 Body Movements Generation for Virtual Characters and Social Robots
- 21 Approach and Dominance as Social Signals for Affective Interfaces
- 22 Virtual Reality and Prosocial Behavior
- 23 Social Signal Processing in Social Robotics
- Part IV Applications of Social Signal Processing
- References
Summary
Introduction
Recent years have seen a growing interest in the development of affective interfaces. At the heart of these systems is an ability to capture social signals and analyse them in a way that meets the requirements and characteristics of the application. There has been a concerted effort in devising a principled approach which could benefit from the interdisciplinary nature of the affective computing endeavour. One common strategy has been to seek conceptual models of emotion that could mediate between the social signals to be captured and the knowledge content of the application. Early systems have largely relied on the intrinsic, natural properties of emotional communication with, for instance, an emphasis on facial expressions both on the output and the input side, together with some theoretical foundation for the acceptance of computers as interaction partners (Reeves & Nass, 1996). Without downplaying the importance of these systems in the development of the field or the practical interest of the applications they entailed (Prendinger & Ishizuka, 2005), it soon appeared that not all interactions could be modelled after interhuman communication, in particular when considering interaction with more complex applications. This complexity can be described at two principal levels: the interaction with complex data, knowledge, or task elements and the nature of the emotions themselves (and their departing from universal or primitive emotions to reflect more sophisticated ones). On the other hand, part of the problem rests with various simplifications that were necessary to get early prototypes off the ground. A good introduction to the full complexity of an affective response can be found in Sander and Scherer (2014), in particular its description of the various levels and components to be considered. These have not always been transposed into affective computing systems; however, as is often the case, the original description frameworks may not be computational enough to support direct and complete transposition.
Dimensional models of emotions have been enthusiastically adopted in affective interfaces for their flexibility in terms of representation as well as the possibility of mapping input modalities onto their dimensions to provide a consistent representation of input.
- Type
- Chapter
- Information
- Social Signal Processing , pp. 287 - 303Publisher: Cambridge University PressPrint publication year: 2017
References
, , & (2001).Manipulation of frontal EEG asymmetry through biofeedback alters self-reported emotional responses and facial EMG.Psychophysiology, 38(04), 685–693.Google Scholar
, , , & (2008). Neurocognitive components of the behavioral inhibition and activation systems: Implications for theories of self-regulation.Psychophysiology, 45(1), 11–19.Google Scholar
(2006). Psychophysiology: Human Behavior and Physiological Response. Hoboken, NJ: Routledge.
, , & (2001). Clinical use of an alpha asymmetry neurofeedback protocol in the treatment of mood disorders: Follow-up study one to five years post therapy.Journal of Neurotherapy, 4(4), 11–18.Google Scholar
& (2010). Approaching the bad and avoiding the good: Lateral prefrontal cortical asymmetry distinguishes between action and valence.Journal of Cognitive Neuroscience, 22(9), 1970–1979.Google Scholar
, , , & (2013). Approach the good, withdraw from the bad – a review on frontal alpha asymmetry measures in applied psychological research.Psychology, 4(3), 261–267
Google Scholar
(2012). In defense of dominance: PAD usage in computational representations of affect.International Journal of Synthetic Emotions, 3(1), 33–42.Google Scholar
, , , et al. (2014a). Towards empathic neurofeedback for interactive storytelling.Open Access Series in Informatics, 41, 42–60.Google Scholar
, , , et al. (2014b). Frontal alpha asymmetry neurofeedback for brain–computer interfaces. In Proceedings of the 6th International Brain–Computer Interface Conference, December. Graz, Austria.
, , , et al. (2014). Towards emotional regulation through neurofeedback. In Proceedings of the 5th Augmented Human International Conference (p. 42).
, , & (2007). Causal perception in virtual reality and its implications for presence factors.Presence: Teleoperators and Virtual Environments, 16(6), 623–642.Google Scholar
& (2003). The state and trait nature of frontal EEG asymmetry in emotion. In & (Eds), The Asymmetrical Brain (pp. 565–615), Cambridge, MA: MIT Press.
& (2004). Frontal EEG asymmetry as a moderator and mediator of emotion.Biological psychology, 67(1), 7–50.Google Scholar
(1991). Flow: The Psychology of Optimal Experience (vol. 41). New York: Harper Perennial.
(1992). Anterior cerebral asymmetry and the nature of emotion.Brain and Cognition, 20(1), 125–151.Google Scholar
(1998). Anterior electrophysiological asymmetries, emotion, and depression: Conceptual and methodological conundrums.Psychophysiology, 35(5), 607–614.Google Scholar
(2003). Darwin and the neural bases of emotion and affective style.Annals of the New York Academy of Sciences, 1000(1), 316–336.Google Scholar
(2004). What does the prefrontal cortex “do” in affect: Perspectives on frontal EEG asymmetry research.Biological Psychology, 67(1), 219–234.Google Scholar
, , , , & (1990). Approachwithdrawal and cerebral asymmetry: Emotional expression and brain physiology: I.Journal of Personality and Social Psychology, 58(2), 330.Google Scholar
& (2007). The empathic brain and its dysfunction in psychiatric populations: Implications for intervention across different clinical conditions.BioPsychoSocial Medicine, 1(1), 22.Google Scholar
, , , & (2005). Brain lateralization of emotional processing: Historical roots and a future incorporating “dominance.”
Behavioral and Cognitive Neuroscience Reviews, 4(1), 3–20.Google Scholar
(2008). Putting the altruism back into altruism: The evolution of empathy.Annual Review of Psychology, 59, 279–300.Google Scholar
, , , et al. (2014). From auditory and visual to immersive neurofeedback: Application to diagnosis of Alzheimer's disease. In (Ed.), Neural Computation, Neural Devices, and Neural Prosthesis (pp. 63–97). New York: Springer.
& (1988). Patterns of brain electrical activity during facial signs of emotion in 10-month-old infants.Developmental Psychology, 24(2), 230–236.Google Scholar
& (2008). Relative left frontal activation to appetitive stimuli: Considering the role of individual differences.Psychophysiology, 45(2), 275–278.Google Scholar
, , , & (2013). Clustering approach to characterize haptic expressions of emotions.ACM Transactions on Applied Perception, 10(4), 1–20.Google Scholar
(2009). Automatic nonverbal analysis of social interaction in small groups: A review.Image and Vision Computing, 27(12), 1775–1787.Google Scholar
(2005). ALMA: a layered model of affect. In Proceedings of The Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (pp. 29–36).
, , & (2009). Using affective trajectories to describe states of flow in interactive art. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (pp. 165–172).
, , , et al. (2009). PAD-based multimodal affective fusion. In 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops (pp. 1–8).
, , & (2011). Evaluating multimodal affective fusion using physiological signals. In Proceedings of the 16th International Conference on Intelligent User Interfaces (pp. 53–62).
, , , et al. (2013). A brain-computer interface to a plan-based narrative. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence (pp. 1997–2005).
, , , et al. (2010). Multi-scale entropy analysis of dominance in social creative activities. In Proceedings of the International Conference on Multimedia (pp. 1035–1038).
& (2010). Empathy constrained: Prejudice predicts reduced mental simulation of actions during observation of outgroups.Journal of Experimental Social Psychology, 46(5), 841–845.Google Scholar
(2004). Contributions from research on anger and cognitive dissonance to understanding the motivational functions of asymmetrical frontal brain activity.Biological Psychology, 67(1), 51–76.Google Scholar
, , & (2010). The role of asymmetric frontal cortical activity in emotion-related phenomena: A review and update.Biological Psychology, 84(3), 451–462.Google Scholar
& (2009). Supine body position reduces neural response to anger evocation.Psychological Science, 20(10), 1209–1210.Google Scholar
, , , et al. (2011). Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding.PLoS Computational Biology, 7(4), e1002028.Google Scholar
& (2007). Recognizing affective dimensions from body posture.Lecture Notes in Computer Science, 4738, 48–58.Google Scholar
& (2013). Affective body expression perception and recognition: A survey.IEEE Transactions on Affective Computing, 4(1), 15–33.Google Scholar
, , , & (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions.Psychophysiology, 30(3), 261–273.Google Scholar
, , , et al. (2009). Empathy is associated with dynamic change in prefrontal brain electrical activity during positive emotion in children.Child Development, 80(4), 1210–1231.Google Scholar
, , , , & (2012). The brain basis of emotion: A meta-analytic review.Behavioral and Brain Sciences, 35(3), 121–143.Google Scholar
& (2012). EEG-based dominance level recognition for emotion-enabled interaction. In Proceedings of IEEE International Conference on Multimedia and Expo (pp. 1039–1044).
& (2011). The feeling of action tendencies: On the emotional regulation of goal-directed behavior. Frontiers in Psychology, 2.
(1996). Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament.Current Psychology, 14, 261–292.Google Scholar
& (1974). An Approach to Environmental Psychology. Cambridge, MA: MIT Press.
& (2005). The empathic companion: A character-based interface that addresses users' affective states.Applied Artificial Intelligence, 19(3–4), 267–285.Google Scholar
& (1996). The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. New York: Cambridge University Press.
, , & (2013). Valence, arousal and dominance in the EEG during game play.International Journal of Autonomous and Adaptive Communications Systems, 6(1), 45–62.Google Scholar
, , , & (1995). Operant (biofeedback) control of leftright frontal alpha power differences: Potential neurotherapy for affective disorders.Biofeedback and Self-Regulation, 20(3), 241–258.Google Scholar
& (1977). Evidence for a three-factor theory of emotions.Journal of Research in Personality, 11(3), 273–294.Google Scholar
& (2014). Traité de psychologie des émotions. Paris: Dunod.
& (1997). Prefrontal brain asymmetry: A biological substrate of the behavioral approach and inhibition systems.Psychological Science, 8(3), 204–210.Google Scholar
, , & (2012). Right frontal cortical asymmetry predicts empathic reactions: Support for a link between withdrawal motivation and empathy.Psychophysiology, 49(8), 1145–1153.Google Scholar
, , , & (2014). Individual Classification of Emotions Using EEG. Journal of Biomedical Science and Engineering, 2014.
, , , et al. (2013). EEG-based assessment of video and ingame learning. In Proceedings of CHI'13 Extended Abstracts on Human Factors in Computing Systems (pp. 667–672).
, , , , & (2014). User Research for 3D Display Settings with EEG Frontal Alpha Asymmetry. In CHI Play 2014 Games User
Workshop, Toronto.Google Scholar
& (1998). Measuring presence in virtual environments: A presence questionnaire.Presence: Teleoperators and Virtual Environments, 7(3), 225–240.Google Scholar
, , , , & (2014). Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback.NeuroImage, 85, 985–995.Google Scholar