Skip to main content Accessibility help
×
Hostname: page-component-848d4c4894-ndmmz Total loading time: 0 Render date: 2024-06-09T22:51:53.568Z Has data issue: false hasContentIssue false

5 - Compromise strategies for action selection

from Part I - Rational and optimal decision making

Published online by Cambridge University Press:  05 November 2011

Anil K. Seth
Affiliation:
University of Sussex
Tony J. Prescott
Affiliation:
University of Sheffield
Joanna J. Bryson
Affiliation:
University of Bath
Get access

Summary

Summary

Among many properties suggested for action selection mechanisms, a prominent one is the ability to select compromise actions, i.e., actions that are not the best to satisfy any active goal in isolation, but rather compromise between the multiple goals. This chapter briefly reviews the history of compromise behaviour and performs experimental analyses of it in an attempt to determine how much compromise behaviour aids an agent. It concludes that optimal compromise behaviour has a surprisingly small benefit over non-compromise behaviour in the experiments performed, it presents some reasons why this may be true, and hypothesises cases where compromise behaviour is truly useful. In particular, it hypothesises that a crucial factor is the level at which an action is taken (low level actions are specific, such as ‘move left leg’; high level actions are vague, such as ‘forage for food’). The chapter hypothesises that compromise behaviour is more beneficial for high-level actions than low-level actions.

Introduction

Agents act. An agent, be it a robot, animal, or piece of software, must repeatedly select actions from a set of candidates. A controller is the mechanism within an agent that selects the action. The question of how to design controllers for such agents is the action selection problem. Researchers who consider the action selection problem have identified potential properties of these controllers. One such property is the ability to exhibit compromise behaviour. A controller exhibits compromise behaviour when the agent has multiple conflicting goals, yet the action selected is not the optimal action for achieving any single one of those goals, but is good for achieving several of those goals in conjunction. For example, a predator stalking two prey might not move directly toward one of the prey, but in between the two, in case one flees (Hutchinson, 1999). The action would not be optimal for individual goals to catch either prey, instead being a compromise between them.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Antonelli, GArrichiello, FChiaverini, S 2008 The null-space-based behavioral control for autonomous robotic systemsIntell. Serv. Rob 1 27CrossRefGoogle Scholar
Arkin, R 1998 Behavior-based RoboticsCambridge, MAMIT PressGoogle Scholar
Avila-Garcia, OCanamero, L 2004 Using hormonal feedback to modulate action selection in a competitive scenarioFrom Animals to Animats 8: Proceedings of the Eighth International Conference on Simulation of Adaptive BehaviorSchaal, SIjspeert, A. JBillard, ACambridge, MAMIT Press,243Google Scholar
Bailey, W. JCunningham, R. JLebel, L 1990 Song power, spectral distribution and female phonotaxis in the bushcricket (Tettigoniiddae: Orthoptera): active female choice or passive attractionAnim. Behav 40 33CrossRefGoogle Scholar
Bartumeus, FCatalan, J 2009 Optimal search behavior and classic foraging theoryJ. Phys. A-Math. Theor 42 434002CrossRefGoogle Scholar
Bertsekas, D. P 2005 Dynamic Programming and Optimal ControlBelmont, MAAthena ScientificGoogle Scholar
Blumberg, B. M 1994 Action-selection in Hamsterdam: lessons from ethologyFrom Animals to Animats 3: Proceedings of the Third International Conference on the Simulation of Adaptive BehaviorCliff, DHusbands, PMeyer, J. AWilson, SCambridge, MAMIT Press108Google Scholar
Blumberg, B. MTodd, P. MMaes, P 1996 No bad dogs: ethological lessons for learning in HamsterdamFrom Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive BehaviorMaes, PMataric, MMeyer, J. APollack, JWilson, WCambridge, MAMIT Press295Google Scholar
Bonasso, R. PFirby, R. JGat, EKortenkamp, DMiller, D 1997 Experiences with an architecture for intelligent, reactive agentsJ. Exp. Theor. Artif. In 9 237CrossRefGoogle Scholar
Bridle, S. LLahav, OOstriker, J. PSteinhardt, P. J 2003 Precision cosmology? Not just yetScience 299 1532CrossRefGoogle Scholar
Brigant, I 2005 The instinct concept of the early Konrad LorenzJ. Hist. Biol 38 571CrossRefGoogle Scholar
Brooks, R 1986 A robust layered control system for a mobile robotIEEE J. Robotic. Autom RA-2 14CrossRefGoogle Scholar
Brooks, R 1997 From earwigs to humansRobot. Auton. Syst 20 291CrossRefGoogle Scholar
Brown, J. SKotler, B 2004 Hazardous duty pay and the foraging cost of predationEcol. Lett 7 999CrossRefGoogle Scholar
Bryson, J 2000 Hierarchy and sequence vs. full parallelism in action selectionFrom Animals to Animats 6: Proceedings of the Sixth International Conference on the Simulation of Adaptive BehaviorMeyer, J. ABerthoz, AFloreano, DRoitblat, HWilson, SCambridge, MAMIT Press,147Google Scholar
Burkhardt, R. W 2004 Patterns of Behavior: Konrad Lorenz, Niko Tinbergen, and the Founding of EthologyChicagoChicago University PressGoogle Scholar
Caldwell, G 1986 Predation as a selective force on foraging herons: effects of plumage color and flockingAuk 103 494Google Scholar
Cannings, CCruz Orive, L. M 1975 On the adjustment of the sex ratio and the gregarious behavior of animal populationsJ. Theor. Biol 55 115CrossRefGoogle ScholarPubMed
Carruthers, P 2004 Practical reasoning in a modular mindMind Lang 19 259CrossRefGoogle Scholar
Choset, HLynch, K. MHutchinson, S 2005 Principles of Robot Motion: Theory, Algorithms, and ImplementationsCambridge, MAMIT PressGoogle Scholar
Clemen, R 1996 Making Hard Decisions: An Introduction to Decision AnalysisBelmont, CADuxbury PressGoogle Scholar
Crabbe, F. L 2002 Compromise Candidates in Positive Goal ScenariosFrom Animals to Animats 7: Proceedings of the Seventh International Conference on the Simulation of Adaptive BehaviorHallam, BFloreano, DHallam, JHeyes, GMeyer, J. ACambridge, MAMIT Press105Google Scholar
Crabbe, F. L 2004 Optimal and non-optimal compromise strategies in action selectionFrom Animals to Animats 8: Proceedings of the Eighth International Conference on Simulation of Adaptive BehaviorSchaal, SIjspeert, A. JBillard, ACambridge, MAMIT Press,pp233Google Scholar
Crabbe, F. L 2007 Compromise strategies for action selectionPhil. Trans. R. Soc. Lond. B Biol. Sci 362 1559CrossRefGoogle ScholarPubMed
Crabbe, F. LDyer, M. G 1999 Second-order networks for wall-building agentsProceedings of the International Joint Conference on Neural NetworksBrown, DWashington, DC, USAIEEEGoogle Scholar
de Sevin, EKallmann, MThalmann, D 2001 Towards real time virtual human life simulationsComputer Graphics InternationalHong KongIEEE Computer Society Press, pp31Google Scholar
Decugis, VFerber, J 1998 An extension of Maes’ action selection mechanism for animatsFrom Animals to Animats 5: Proceedings of the Fifth International Conference on Simulation of Adaptive BehaviorPfeifer, RBlumberg, BMeyer, J-AWilson, S. WCambridge, MAMIT Press153Google Scholar
Dewsbury, D. A 1978 fixed action pattern?Anim. Behav 26 310CrossRefGoogle Scholar
Dewsbury, D. A 1992 Comparative psychology and ethology: a reassessmentAm. Psychol 47 208CrossRefGoogle Scholar
Evans, JPatron, PSmith, BLane, M 2008 Design and evaluation of a reactive and deliberative collision avoidance and escape architecture for autonomous robotsAuton. Rob 24 247CrossRefGoogle Scholar
Favilla, M 2002 Reaching movements: mode of motor programming influences programming by time itselfExp Brain Res 144 414CrossRefGoogle Scholar
Fikes, R. ENilsson, N. J 1971 STRIPS: a new approach to the application of theorem proving to problem solvingArtif. Intell 2 189CrossRefGoogle Scholar
Fraenkel, G. SGunn, D. L 1961 The Orientation of AnimalsNew YorkDoverGoogle Scholar
Fraser, D. FCerri, R. D 1982 Experimental evaluation of predator–prey relationships in a patchy environment: consequences for habitat use patterns in minnowsEcology 63 307CrossRefGoogle Scholar
Gat, E 1991 Reliable goal-directed reactive control for real-world autonomous mobile robotsVirginia Polytechnic Institute and State UniversityBlacksburg, VAGoogle Scholar
Gerevini, A. ESaetti, ASerina, I 2008 An approach to efficient planning with numerical fluents and multi-criteria plan qualityArtif. Intell 172 8CrossRefGoogle Scholar
Ghallab, MNau, DTraverso, P 2004 Automated Planning: Theory and PracticeSan Francisco, CAMorgan Kaufmann Publishers IncGoogle Scholar
Ghez, CFavilla, MGhilardi, M. F 1997 Discrete and continuous planning of hand movements and isometric force trajectoriesExp. Brain Res 115 217CrossRefGoogle ScholarPubMed
Girard, BCuzin, VGuillot, AGurney, K. NPrescott, T. J 2002 Comparing a brain-inspired robot action selection mechanism with ‘winner-takes-all’From Animals to Animats 7: Proceedings of the Seventh International Conference on the Simulation of Adaptive BehaviorHallam, BFloreano, DHallam, JHeyes, GMeyer, J. ACambridge, MAMIT Press75Google Scholar
Grubb, T. CGreenwald, L 1982 Sparrows and a brushpile: foraging responses to different combinations of predation risk and energy costAnim. Behav 30 637CrossRefGoogle Scholar
Hillier, F. SLieberman, G. J 2002 Introduction to Operations ResearchNew YorkMcGraw-HillGoogle Scholar
Hinde, R. A 1966 Animal BehaviourNew YorkMcGraw-HillGoogle Scholar
Hoffmann, J 2003 The Metric-FF planning system: translating ignoring delete lists to numeric state variablesJ. Artif. Intell. Res 20 291Google Scholar
Houston, A. IMcNamara, J. MSteer, S 2007 Do we expect natural selection to produce rational behaviorPhil. Trans. Roy. Soc. B 362 1531CrossRefGoogle Scholar
Howard, R. A 1977 Risk preferenceReadings in Decision AnalysisMenlo Park, CASRI InternationalGoogle Scholar
Humphrys, M 1996 Action Selection Methods Using Reinforcement LearningUniversity of CambridgeCambridge, UKGoogle Scholar
Hurdus, J. GHong, D. W 2009 Behavioral programming with hierarchy and parallelism in the DARPA Urban Challenge and RoboCupMultisensor Fusion and Integration for Intelligent SystemsHahn, HKo, HLee, SBerlinSpringer255CrossRefGoogle Scholar
Hutchinson, J. M. C 1999 Bet-hedging when targets may disappear: optimal mate-seeking or prey-catching trajectories and the stability of leks and herdsJ. Theor. Biol 196 33CrossRefGoogle ScholarPubMed
Hutchinson, J. M. CGigerenzer, G 2005 Simple heuristics and rules of thumb: where psychologists and behavioural biologists might meetBehav. Process 69 97CrossRefGoogle ScholarPubMed
Iglesias, ALuengo, F 2005 New goal selection scheme for behavioral animation of intelligent virtual agentsIEICE Trans. Inf. and Syst E88-D 865CrossRefGoogle Scholar
Jaafar, JMcKenzie, E 2008 A fuzzy action selection method for virtual agent navigation in unknown virtual environmentsJ. Uncertain Syst 2 144Google Scholar
Jones, R. MLaird, J. ENielsen, P. E 1999 Automated intelligent pilots for combat flight simulationAI Magazine 20 27Google Scholar
Krebs, J. RDavies, N 1997 Behavioural Ecology: An Evolutionary ApproachOxfordBlackwell PublishersGoogle Scholar
Krebs, J. RErichsen, J. TWebber, M. ICharnov, E. L 1977 Optimal prey selection in the great tit ()Anim. Behav 25 30CrossRefGoogle Scholar
Latimer, WSippel, M 1987 Acoustic cues for female choice and male competition in Anim. Behav 35 887CrossRefGoogle Scholar
Lima, S. L 1998 Stress and decision making under the risk of predation: recent developments from behavioral, reproductive, and ecological perspectivesAdv. Stud. Behav 27 215CrossRefGoogle Scholar
Lorenz, K. Z 1981 The Foundations of EthologyNew YorkSpringer-VerlagCrossRefGoogle Scholar
Lund, H. HWebb, BHallam, J 1997 A robot attracted to the cricket species Fourth European Conference on Artificial LifeHusbands, PHarvey, IBrighton, UKMIT Press/Bradford Books,246Google Scholar
Luo, XJennings, N. R 2007 A spectrum of compromise aggregation operators for multi-attribute decision makingArtif. Intell 171 161CrossRefGoogle Scholar
Maes, P 1990 How to do the right thingConnect. Sci., Special Issue on Hybrid Systems 1 291Google Scholar
McNamara, J. MHouston, A. I 1980 The application of statistical decision theory to animal behaviourJ. Theor. Biol 85 673CrossRefGoogle ScholarPubMed
McNamara, J. MHouston, A. I 1994 The effect of a change in foraging options on intake rate and predation rateAm. Nat 144 978CrossRefGoogle Scholar
Mesterton-Gibbons, M 1989 On compromise in foraging and an experiment by Krebs . (1977)J. Math. Biol 27 273CrossRefGoogle Scholar
Milinksi, M 1986 Constraints places by predators on feeding behaviorThe behavior of teleost fishesPitcher, T. JLondonCroom Helm, pp.236Google Scholar
Mitchell, T. M 1997 Machine LearningBostonMcGraw-HillGoogle Scholar
Montes-Gonzales, FPrescott, T. JGurney, KHumphrys, MRedgrave, P 2000 An embodied model of action selection mechanisms in the vertebrate brainFrom Animals to Animats 6: Proceedings of the Sixth International Conference on the Simulation of Adaptive BehaviorMeyer, J. ABerthoz, AFloreano, DRoitblat, HWilson, SCambridge, MAMIT Press157Google Scholar
Morris, G. KKerr, G. EFullard, J. H 1978 Phonotactic preferences of female meadow katydidsCan. J. Zool 56 1479CrossRefGoogle Scholar
Müller, A 1925 Über Lichtreaktionen von LandasselnZ. vergl. Physiol 3 113CrossRefGoogle Scholar
Newell, ASimon, H. A 1976 Computer science as empirical inquiry: symbols and searchComm. ACM 19 113CrossRefGoogle Scholar
Pfeifer, RScheier, C 1999 Understanding IntelligenceCambridge, MAMIT PressGoogle Scholar
Pirjanian, P 2000 Multiple objective behavior-based controlJ. Robot. Auton. Syst 31 53CrossRefGoogle Scholar
Pirjanian, PChristensen, H. IFayman, J. A 1998 Application of voting to fusion of purposive modules: an experimental investigationJ. Robot. Auton. Syst 23 253CrossRefGoogle Scholar
Quinlan, J. R 1983 Learning efficient classification procedures and their application to chess and gamesMachine Learning: an artificial intelligence approachMichalski, R. SCarbonell, J. GMitchell, T. MSan Mateo, CAMorgan Kaufmann, pp. 463–82Google Scholar
Quinlan, J. R 1993 C4.5: Programs for Machine LearningSan Mateo, CAMorgan KaufmannGoogle Scholar
Refanidis, IVlahavas, I 2003 Multiobjective heuristic state-space planningArtif. Intell 145 1CrossRefGoogle Scholar
Römer, H 1993 Environmental and biological constraints for the evolution of long-range signalling and hearing in acoustic insectsPhil. Trans. Roy. Soc. B 340 179CrossRefGoogle Scholar
Russell, SNorvig, P 2010 Artificial Intelligence. A Modern ApproachCambridge, MAMIT PressGoogle Scholar
Seth, A. K 2007 The ecology of action selectionPhil. Trans. Roy. Soc. B 362 1545CrossRefGoogle ScholarPubMed
Stephens, WKrebs, J. R 1986 Foraging TheoryPrinceton, NJPrinceton University PressGoogle Scholar
Thorisson, K. R 1996 Communicative humanoids: a computational model of psychosocial dialogue skillsMassachusetts Institute of TechnologyCambridge, MAGoogle Scholar
Thorpe, W. H 1979 The Origins and Rise of EthologyNew YorkPreagerGoogle Scholar
Tinbergen, N 1950 The hierarchical organisation of nervous mechanisms underlying instinctive behaviorSympos. Soc. Exper. Biol 4 305Google Scholar
Todd, P. M 1992 Machine intelligence – the animat path to intelligent adaptive behaviourIEEE Computer 25 78CrossRefGoogle Scholar
Tu, X 1996 Artificial animals for computer animation: biomechanics, locomotion, perception, and behaviorUniversity of TorontoDepartment of Computer ScienceGoogle Scholar
Tyrrell, T 1993 Computational mechanism for action selectionUniversity of EdinburghGoogle Scholar
von Holst, E 1935 Über den Lichtruchenreflex bei FischenPubl. Staz. Zool 15 143Google Scholar
Werner, G. M 1994 Using second order neural connection for motivation of behavioral choicesProceedings of the Third International Conference on Simulation of Adaptive BehaviorCliff, DHusbands, PMeyer, J-AWilson, S. WBrighton, UKMIT Press,154Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×