Skip to main content Accessibility help
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 7
  • Print publication year: 2009
  • Online publication date: December 2009

11 - Sensory control of object manipulation



Series of action phases characterize natural object manipulation tasks where each phase is responsible for satisfying a task subgoal. Subgoal attainment typically corresponds to distinct mechanical contact events, either involving the making or breaking of contact between the digits and an object or between a held object and another object. Subgoals are realized by the brain selecting and sequentially implementing suitable action-phase controllers that use sensory predictions and afferents signals in specific ways to tailor the motor output in anticipation of requirements imposed by objects' physical properties. This chapter discusses the use of tactile and visual sensory information in this context. It highlights the importance of sensory predictions, especially related to the discrete and distinct sensory events associated with contact events linked to subgoal completion, and considers how sensory signals influence and interact with such predictions in the control of manipulation tasks.

Sensory systems supporting object manipulation

In addition to multiple motor systems (arm, hand, posture), most natural object manipulation tasks engage multiple sensory systems. Vision provides critical information for control of task kinematics. In reaching, we use vision to locate objects in the environment and to identify contact sites for the digits that will be stable and advantageous for various actions we want to perform with the grasped object (Goodale et al., 1994; Santello & Soechting, 1998; Cohen & Rosenbaum, 2004; Cuijpers et al., 2004; Lukos et al., 2007).

Ballard, D. H., Hayhoe, M. M., Li, F. & Whitehead, S. D. (1992). Hand-eye coordination during sequential tasks. Philos Trans R Soc Lond B, Biol Sci, 337, 331–338.
Birznieks, I., Burstedt, M. K. O., Edin, B. B. & Johansson, R. S. (1998). Mechanisms for force adjustments to unpredictable frictional changes at individual digits during two-fingered manipulation. J Neurophysiol, 80, 1989–2002.
Birznieks, I., Jenmalm, P., Goodwin, A. & Johansson, R. (2001). Encoding of direction of fingertip forces by human tactile afferents. J Neurosci, 21, 8222–8237.
Bracewell, R. M., Wing, A. M., Soper, H. M. & Clark, K. G. (2003). Predictive and reactive co-ordination of grip and load forces in bimanual lifting in man. Eur J Neurosci, 18, 2396–2402.
Burstedt, M. K. O., Edin, B. B. & Johansson, R. S. (1997). Coordination of fingertip forces during human manipulation can emerge from independent neural networks controlling each engaged digit. Exp Brain Res, 117, 67–79.
Burstedt, M. K. O., Flanagan, R. & Johansson, R. S. (1999). Control of grasp stability in humans under different frictional conditions during multi-digit manipulation. J Neurophysiol, 82, 2393–2405.
Cadoret, G. & Smith, A. M. (1996). Friction, not texture, dictates grip forces used during object manipulation. J Neurophysiol, 75, 1963–1969.
Cohen, R. G. & Rosenbaum, D. A. (2004). Where grasps are made reveals how grasps are planned: generation and recall of motor plans. Exp Brain Res, 157, 486–495.
Cole, K. J., Steyers, C. M. & Graybill, E. K. (2003). The effects of graded compression of the median nerve in the carpal canal on grip force. Exp Brain Res, 148, 150–157.
Cuijpers, R. H., Smeets, J. B. & Brenner, E. (2004). On the relation between object shape and grasping kinematics. J Neurophysiol, 91, 2598–2606.
Edin, B. B., Westling, G. & Johansson, R. S. (1992). Independent control of fingertip forces at individual digits during precision lifting in humans. J Physiol, 450, 547–564.
Eliasson, A. C., Forssberg, H., Ikuta, al. (1995). Development of human precision grip V. Anticipatory and triggered grip actions during sudden loading. Exp Brain Res, 106, 425–433.
Flanagan, J. R. & Tresilian, J. R. (1994). Grip load force coupling: a general control strategy for transporting objects. J Exp Psychol Hum Percept Perform, 20, 944–957.
Flanagan, J. R. & Wing, A. M. (1995). The stability of precision grip forces during cyclic arm movements with a hand-held load. Exp Brain Res, 105, 455–464.
Flanagan, J. R. & Beltzner, M. A. (2000). Independence of perceptual and sensorimotor predictions in the size–weight illusion. Nat Neurosci, 3, 737–41.
Flanagan, J. R. & Johansson, R. S. (2003). Action plans used in action observation. Nature, 424, 769–771.
Flanagan, J. R., Burstedt, M. K. O. & Johansson, R. S. (1999). Control of fingertip forces in multi-digit manipulation. J Neurophysiol, 81, 1706–1717.
Flanagan, J. R., Bowman, M. C. & Johansson, R. S. (2006). Control strategies in object manipulation tasks. Curr Opin Neurobiol, 16, 650–659.
Forssberg, H., Eliasson, A. C., Kinoshita, H., Johansson, R. S. & Westling, G. (1991). Development of human precision grip. I: Basic coordination of force. Exp Brain Res, 85, 451–457.
Forssberg, H., Kinoshita, H., Eliasson, A. al. (1992). Development of human precision grip. 2. Anticipatory control of isometric forces targeted for objects weight. Exp Brain Res, 90, 393–398.
Forssberg, H., Eliasson, A. C., Kinoshita, H., Westling, G. & Johansson, R. S. (1995). Development of human precision grip. IV. Tactile adaptation of isometric finger forces to the frictional condition. Exp Brain Res, 104, 323–330.
Gentilucci, M., Toni, I., Daprati, E. & Gangitano, M. (1997). Tactile input of the hand and the control of reaching to grasp movements. Exp Brain Res, 114, 130–137.
Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. (1986). Neuronal population coding of movement direction. Science, 233, 1416–1419.
Goodale, M. A., Meenan, J. P., Bülthoff, H. al. (1994). Separate neural pathways for the visual analysis of object shape in perception and prehension. Curr Biol, 4, 604–610.
Goodwin, A. W. & Wheat, H. E. (2004). Sensory signals in neural populations underlying tactile perception and manipulation. Ann Rev Neurosci, 27, 53–77.
Goodwin, A. W., Jenmalm, P. & Johansson, R. S. (1998). Control of grip force when tilting objects: effect of curvature of grasped surfaces and of applied tangential torque. J Neurosci, 18, 10724–10734.
Gordon, A. M. & Soechting, J. F. (1995). Use of tactile afferent information in sequential finger movements. Exp Brain Res, 107, 281–292.
Gordon, A. M., Forssberg, H., Johansson, R. S. & Westling, G. (1991). Integration of sensory information during the programming of precision grip: comments on the contributions of size cues. Exp Brain Res, 85, 226–229.
Gordon, A. M., Forssberg, H., Johansson, R. S., Eliasson, A. C. & Westling, G. (1992). Development of human precision grip. 3. Integration of visual size cues during the programming of isometric forces. Exp Brain Res, 90, 399–403.
Gordon, A. M., Westling, G., Cole, K. J. & Johansson, R. S. (1993). Memory representations underlying motor commands used during manipulation of common and novel objects. J Neurophysiol, 69, 1789–1796.
Gysin, P., Kaminski, T. R. & Gordon, A. M. (2003). Coordination of fingertip forces in object transport during locomotion. Exp Brain Res, 149, 371–379.
Häger-Ross, C. & Johansson, R. S. (1996). Non-digital afferent input in reactive control of fingertip forces during precision grip. Exp Brain Res, 110, 131–141.
Jenmalm, P. & Johansson, R. S. (1997). Visual and somatosensory information about object shape control manipulative finger tip forces. J Neurosci, 17, 4486–4499.
Jenmalm, P., Dahlstedt, S. & Johansson, R. S. (2000). Visual and tactile information about object curvature control fingertip forces and grasp kinematics in human dexterous manipulation. J Neurophysiol, 84, 2984–2997.
Jenmalm, P., Birznieks, I., Goodwin, A. W. & Johansson, R. S. (2003). Influences of object shape on responses in human tactile afferents under conditions characteristic for manipulation. Eur J Neurosci, 18, 164–176.
Johansson, R. S. & Vallbo, Å.B. (1976). Skin mechanoreceptors in the human hand: an inference of some population properties. In Zotterman, Y. (Ed.), Sensory Functions of the Skin in Primates, with Special Reference to Man (pp. 171–184). Oxford, UK: Pergamon Press Ltd.
Johansson, R. S. & Vallbo, A. B. (1979). Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. J Physiol, 286, 283–300.
Johansson, R. S. & Vallbo, Å. B. (1983). Tactile sensory coding in the glabrous skin of the human hand. Trends Neurosci, 6, 27–31.
Johansson, R. S. & Westling, G. (1984). Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp Brain Res, 56, 550–564.
Johansson, R. S. & Westling, G. (1987). Signals in tactile afferents from the fingers eliciting adaptive motor responses during precision grip. Exp Brain Res, 66, 141–154.
Johansson, R. S. & Westling, G. (1988a). Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with precision grip. Exp Brain Res, 71, 59–71.
Johansson, R. S. & Westling, G. (1988b). Programmed and triggered actions to rapid load changes during precision grip. Exp Brain Res, 71, 72–86.
Johansson, R. S. & Cole, K. J. (1992). Sensory-motor coordination during grasping and manipulative actions. Curr Opin Neurobiol, 2, 815–823.
Johansson, R. S. & Birznieks, I. (2004). First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nat Neurosci, 7, 170–177.
Johansson, R. S., Backlin, J. L. & Burstedt, M. K. O. (1999). Control of grasp stability during pronation and supination movements. Exp Brain Res, 128, 20–30.
Johansson, R. S., Westling, G., Bäckström, A. & Flanagan, J. R. (2001). Eye-hand coordination in object manipulation. J Neurosci, 21, 6917–6932.
Knibestöl, M. (1973). Stimulus-response functions of rapidly adapting mechanoreceptors in human glabrous skin area. J Physiol, 232, 427–452.
Knibestöl, M. (1975). Stimulus-response functions of slowly adapting mechanoreceptors in the human glabrous skin area. J Physiol, 245, 63–80.
Lackner, J. R. & DiZio, P. A. (2000). Aspects of body self-calibration. Trends Cogn Sci, 4, 279–288.
LaMotte, R. H. (2000). Softness discrimination with a tool. J Neurophysiol, 83, 1777–1786.
Land, M. F. & Furneaux, S. (1997). The knowledge base of the oculomotor system. Philos Trans R Soc Lond B, Biol Sci, 352, 1231–1239.
Land, M., Mennie, N. & Rusted, J. (1999). The roles of vision and eye movements in the control of activities of daily living. Perception, 28, 1311–1328.
Lemon, R. N., Johansson, R. S. & Westling, G. (1995). Corticospinal control during reach, grasp and precision lift in man. J Neurosci, 15, 6145–6156.
Lukos, J., Ansuini, C. & Santello, M. (2007). Choice of contact points during multidigit grasping: effect of predictability of object center of mass location. J Neurosci, 27, 3894–3903.
Macefield, V. G. & Johansson, R. S. (1996). Control of grip force during restraint of an object held between finger and thumb: responses of muscle and joint afferents from the digits. Exp Brain Res, 108, 172–184.
Macefield, V. G., Häger-Ross, C. & Johansson, R. S. (1996). Control of grip force during restraint of an object held between finger and thumb: responses of cutaneous afferents from the digits. Exp Brain Res, 108, 155–171.
Monzée, J., Lamarre, Y. & Smith, A. M. (2003). The effects of digital anesthesia on force control using a precision grip. J Neurophysiol, 89, 672–683.
Niu, X., Latash, M. L. & Zatsiorsky, V. M. (2007). Prehension synergies in the grasps with complex friction patterns: local versus synergic effects and the template control. J Neurophysiol, 98, 16–28.
Nowak, D. A. & Hermsdörfer, J. (2003). Digit cooling influences grasp efficiency during manipulative tasks. Eur J Appl Physiol, 89, 127–133.
Nowak, D. A., Glasauer, S. & Hermsdorfer, J. (2004). How predictive is grip force control in the complete absence of somatosensory feedback?Brain, 127, 182–192.
Paillard, J. (1996). Fast and slow feedback loops for the visual correction of spatial errors in a pointing task: a reappraisal. Can J Physiol Pharmacol, 74, 401–417.
Paré, M. & Dugas, C. (1999). Developmental changes in prehension during childhood. Exp Brain Res, 125, 239–247.
Pawluk, D. T. & Howe, R. D. (1999). Dynamic lumped element response of the human fingerpad. J Biomech Eng, 121, 178–183.
Prablanc, C., Pélisson, D. & Goodale, M. A. (1986). Visual control of reaching movements without vision of the limb. I. Role of retinal feedback of target position in guiding the hand. Exp Brain Res, 62, 293–302.
Prablanc, C., Desmurget, M. & Gréa, H. (2003). Neural control of on-line guidance of hand reaching movements. Progr Brain Res, 142, 155–170.
Quaney, B. M. & Cole, K. J. (2004). Distributing vertical forces between the digits during gripping and lifting: the effects of rotating the hand versus rotating the object. Exp Brain Res, 155, 145–155.
Rabin, E. & Gordon, A. M. (2004). Tactile feedback contributes to consistency of finger movements during typing. Exp Brain Res, 155, 362–369.
Rao, A. K. & Gordon, A. M. (2001). Contribution of tactile information to accuracy in pointing movements. Exp Brain Res, 138, 438–445.
Reilmann, R., Gordon, A. M. & Henningsen, H. (2001). Initiation and development of fingertip forces during whole-hand grasping. Exp Brain Res, 140, 443–452.
Rizzolatti, G., Fogassi, L. & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci, 2, 661–670.
Rotman, G., Troje, N. F., Johansson, R. S. & Flanagan, J. R. (2006). Eye movements when observing predictable and unpredictable actions. J Neurophysiol, 96, 1358–1369.
Säfström, D. & Edin, B. B. (2004). Task requirements influence sensory integration during grasping in humans. Learn Mem, 11, 356–363.
Salimi, I., Hollender, I., Frazier, W. & Gordon, A. M. (2000). Specificity of internal representations underlying grasping. J Neurophysiol, 84, 2390–2397.
Salimi, I., Frazier, W., Reilmann, R. & Gordon, A. M. (2003). Selective use of visual information signaling objects' center of mass for anticipatory control of manipulative fingertip forces. Exp Brain Res, 150, 9–18.
Santello, M. & Soechting, J. F. (1998). Gradual molding of the hand to object contours. J Neurophysiol, 79, 1307–1320.
Santello, M. & Soechting, J. F. (2000). Force synergies for multifingered grasping. Exp Brain Res, 133, 457–467.
Saunders, J. A. & Knill, D. C. (2004). Visual feedback control of hand movements. J Neurosci, 24, 3223–3234.
Schenker, M., Burstedt, M. K., Wiberg, M. & Johansson, R. S. (2006). Precision grip function after hand replantation and digital nerve injury. J Plast Reconstr Aesthet Surg, 59, 706–716.
Vallbo, Å.B. & Hagbarth, K.-E. (1968). Activity from skin mechanoreceptors recorded percutaneously in awake human subjects. Exp Neurol, 21, 270–289.
Vallbo, A. B. & Johansson, R. S. (1984). Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Hum Neurobiol, 3, 3–14.
Westling, G. & Johansson, R. S. (1984). Factors influencing the force control during precision grip. Exp Brain Res, 53, 277–284.
Westling, G. & Johansson, R. S. (1987). Responses in glabrous skin mechanoreceptors during precision grip in humans. Exp Brain Res, 66, 128–140.
Wheat, H. E., Goodwin, A. W. & Browning, A. S. (1995). Tactile resolution: peripheral neural mechanisms underlying the human capacity to determine positions of objects contacting the fingerpad. J Neurosci, 15, 5582–5595.
Wing, A. M. & Lederman, S. J. (1998). Anticipating load torques produced by voluntary movements. J Exp Psychol Hum Percept Perform, 24, 1571–1581.
Witney, A. G. & Wolpert, D. M. (2007). The effect of externally generated loading on predictive grip force modulation. Neurosci Lett, 414, 10–15.
Witney, A. G., Goodbody, S. J. & Wolpert, D. M. (1999). Predictive motor learning of temporal delays. J Neurophysiol, 82, 2039–2048.