Skip to main content Accessibility help
×
Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-09T05:51:44.201Z Has data issue: false hasContentIssue false

7 - Bayesian approaches to modelling action selection

from Part I - Rational and optimal decision making

Published online by Cambridge University Press:  05 November 2011

Anil K. Seth
Affiliation:
University of Sussex
Tony J. Prescott
Affiliation:
University of Sheffield
Joanna J. Bryson
Affiliation:
University of Bath
Get access

Summary

Summary

We live in an uncertain world, and each decision may have many possible outcomes; choosing the best decision is thus complicated. This chapter describes recent research in Bayesian decision theory, which formalises the problem of decision making in the presence of uncertainty and often provides compact models that predict observed behaviour. With its elegant formalisation of the problems faced by the nervous system, it promises to become a major inspiration for studies in neuroscience.

Introduction

Choosing the right action relies on our having the right information. The more information we have, the more capable we become at making intelligent decisions. Ideally, we want to know what the current state of the world is, what possible actions we can take in response to it, and what the outcomes of these actions will be. When we choose actions that will most clearly bring about our desired results, we are said to be behaving rationally (see Chapter 2). Equivalently, we could say that rational behaviour is optimal, in that this behaviour executes the best actions for achieving our desired results (see Chapters 3 and 4). Thus behaving rationally is equivalent to solving an optimality problem: what actions should we select to best achieve our goals?

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alais, DBurr, D 2004 The ventriloquist effect results from near-optimal bimodal integrationCurr. Biol 14 257CrossRefGoogle ScholarPubMed
Angelaki, D. EShaikh, A. GGreen, A. M 2004 Neurons compute internal models of the physical laws of motionNature 430 560CrossRefGoogle ScholarPubMed
Berniker, MKörding, K 2008 Estimating the sources of motor errors for adaptation and generalizationNat. Neurosci 11 1454CrossRefGoogle ScholarPubMed
Berniker, MVoss, MKörding, K 2010 Learning priors for Bayesian computations in the nervous systemPLoS One 5CrossRefGoogle Scholar
Bishop, C 2006 Pattern Recognition and Machine LearningBerlinSpringerGoogle Scholar
Brashers-Krug, TShadmehr, RBizzi, E 1996 Consolidation in human motor memoryNature 382 252CrossRefGoogle ScholarPubMed
Bryson, A. EHo, Y. C 1975 Applied Optimal Control: Optimization, Estimation, and ControlLondonTaylor and Francis GroupGoogle Scholar
Chater, NTenenbaum, J. BYuille, A 2006 Probabilistic models of cognition: where next?Trends Cogn. Sci 10 292CrossRefGoogle ScholarPubMed
Cisek, PKalaska, J. F 2005 Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of actionNeuron 45 801CrossRefGoogle Scholar
Deneve, S 2008 Bayesian spiking neurons I: inferenceNeural Comput 20 91CrossRefGoogle ScholarPubMed
Ernst, M. OBanks, M. S 2002 Humans integrate visual and haptic information in a statistically optimal fashionNature 415 429CrossRefGoogle Scholar
Ernst, M. OBulthoff, H. H 2004 Merging the senses into a robust perceptTrends Cogn. Sci 8 162CrossRefGoogle ScholarPubMed
Flanagan, J. RBittner, J. PJohansson, R. S 2008 Experience can change distinct size-weight priors engaged in lifting objects and judging their weightsCurr. Biol 18 1742CrossRefGoogle ScholarPubMed
Friston, K 2009 The free-energy principle: a rough guide to the brain?Trends Cogn. Sci 13 293CrossRefGoogle Scholar
Friston, K. JDaunizeau, JKiebel, S. J 2009 Reinforcement learning or active inference?PLoS One 4CrossRefGoogle ScholarPubMed
Geisler, W. SKersten, D 2002 Illusions, perception and BayesNat. Neurosci 5 508CrossRefGoogle ScholarPubMed
Gold, J. IShadlen, M. N 2001 Neural computations that underlie decisions about sensory stimuliTrends Cogn. Sci 5 10CrossRefGoogle ScholarPubMed
Gold, J. IShadlen, M. N 2003 The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commandsJ. Neurosci 23 632CrossRefGoogle ScholarPubMed
Gopnik, AGlymour, CSobel, D 2004 A theory of causal learning in children: causal maps and Bayes netsPsychol. Rev 111 3CrossRefGoogle ScholarPubMed
Griffiths, T. LTenenbaum, J. B 2005 Structure and strength in causal inductionCognit. Psychol 51 334CrossRefGoogle ScholarPubMed
Hoyer, P. OHyvärinen, A 2003 Interpreting neural response variability as Monte Carlo sampling of the posteriorNeural Information Processing SystemsCambridge, MAMIT PressGoogle Scholar
Kagel, J. HRoth, A. E 1995 The Handbook of Experimental EconomicsPrinceton, NJPrinceton University PressGoogle Scholar
Kahneman, DTversky, A 1979 Prospect theory: an analysis of decision under riskEconometrica XVLII 263CrossRefGoogle Scholar
Kersten, DYuille, A 2003 Bayesian models of object perceptionCurr. Opin. Neurobiol 13 150CrossRefGoogle ScholarPubMed
Kiani, RShadlen, M. N 2009 Representation of confidence associated with a decision by neurons in the parietal cortexScience 324 759CrossRefGoogle ScholarPubMed
Knill, D. C 2007 Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slantJ. Vis 7 1CrossRefGoogle ScholarPubMed
Knill, DRichards, W 1996 Perception as Bayesian InferenceCambridgeCambridge University PressCrossRefGoogle Scholar
Körding, K 2007 Decision theory: what ‘should’ the nervous system do?Science 318 606CrossRefGoogle Scholar
Körding, KTenenbaum, J 2006 A generative model based approach to motor adaptationNeural Information Processing SystemsCambridge, MAMIT PressGoogle Scholar
Körding, K. PFukunaga, IHoward, I 2004 A neuroeconomics approach to inferring utility functions in sensorimotor controlPLoS Biol 2 e330CrossRefGoogle ScholarPubMed
Körding, K. PKu, S. PWolpert, D. M 2004 Bayesian integration in force estimationJ. Neurophys 92 3161CrossRefGoogle ScholarPubMed
Körding, K. PWolpert, D. M 2004 Bayesian integration in sensorimotor learningNature 427 244CrossRefGoogle ScholarPubMed
Körding, K. PWolpert, D. M 2004 The loss function of sensorimotor learningProc. Natl. Acad. Sci. USA 101 9839CrossRefGoogle ScholarPubMed
Körding, K. PWolpert, D. M 2006 Bayesian decision theory in sensorimotor controlTrends Cogn Sci 10 319CrossRefGoogle ScholarPubMed
Ma, W. JBeck, J. MLatham, P. EPouget, A 2006 Bayesian inference with probabilistic population codesNat. Neurosci 9 1432CrossRefGoogle ScholarPubMed
MacKay, D. J. C 2003 Information Theory, Inference, and Learning AlgorithmsCambridgeCambridge University PressGoogle Scholar
Maloney, L. TTrommershäuser, J 2006 Questions without words: A comparison between decision making under risk and movement planning under riskIntegrated Models of Cognitive SystemsNew York, NYOxford University Press, pp. 297–314Google Scholar
Mehta, BSchaal, S 2002 Forward models in visuomotor controlJ. Neurophysiol 88 942CrossRefGoogle ScholarPubMed
Michotte, A 1963 The Perception of CausalityLondonMethuenGoogle Scholar
Miyazaki, MNozaki, DNakajima, Y 2005 Testing Bayesian models of human coincidence timingJ. Neurophysiol 94 395CrossRefGoogle ScholarPubMed
Miyazaki, MYamamoto, SUchida, SKitazawa, S 2006 Bayesian calibration of simultaneity in tactile temporal order judgmentNat. Neurosci 9 875CrossRefGoogle ScholarPubMed
Najemnik, JGeisler, W. S 2005 Optimal eye movement strategies in visual searchNature 434 387CrossRefGoogle ScholarPubMed
Najemnik, JGeisler, W. S 2008 Eye movement statistics in humans are consistent with an optimal search strategyJ. Vis 8 1CrossRefGoogle ScholarPubMed
Najemnik, JGeisler, W. S 2009 Simple summation rule for optimal fixation selection in visual searchVision Res 49 1286CrossRefGoogle ScholarPubMed
Peterka, R. JLoughlin, P. J 2004 Dynamic regulation of sensorimotor integration in human postural controlJ. Neurophysiol 91 410CrossRefGoogle ScholarPubMed
Pouget, ADayan, PZemel, R. S 2003 Inference and computation with population codesAn. Rev. Neurosci. 26 381CrossRefGoogle ScholarPubMed
Rosas, PWagemans, JErnst, M. OWichmann, F. A 2005 Texture and haptic cues in slant discrimination: reliability-based cue weighting without statistically optimal cue combinationJ. Opt. Soc. Am. A 22 801CrossRefGoogle ScholarPubMed
Shams, LMa, W. JTanaka, S 2005 Sound-induced flash illusion as an optimal perceptNeuroreport 16 1923CrossRefGoogle ScholarPubMed
Sperber, DPremack, D 1995 Causal Cognition: A Multidisciplinary DebateOxfordOxford University PressGoogle Scholar
Stengel, R. F 1994 Optimal Control and EstimationNew YorkDover PublicationsGoogle Scholar
Stocker, A. ASimoncelli, E. P 2006 Noise characteristics and prior expectations in human visual speed perceptionNat. Neurosci 9 578CrossRefGoogle ScholarPubMed
Stuart, J. RPeter, N 2003 Artificial Intelligence: A Modern ApproachHarlow, UKPearson EducationGoogle Scholar
Sutton, R. SBarto, A. G 1998 Reinforcement LearningCambridge, MAMIT PressGoogle Scholar
Tassinari, HHudson, T. ELandy, M. S 2006 Combining priors and noisy visual cues in a rapid pointing taskJ. Neurosci 26 10154CrossRefGoogle Scholar
Tenenbaum, J. BGriffiths, T. LKemp, C 2006 Theory-based Bayesian models of inductive learning and reasoningTrends Cogn. Sci 10 309CrossRefGoogle ScholarPubMed
Todorov, E 2004 Optimality principles in sensorimotor controlNat. Neurosci 7 907CrossRefGoogle ScholarPubMed
Todorov, E 2006 Optimal Control TheoryBayesian BrainDoya, KCambridge, MAMIT Press269Google Scholar
Todorov, E 2008
Todorov, E 2009 Efficient computation of optimal actionsProc. Natl. Acad. Sci. USA 106 11478CrossRefGoogle ScholarPubMed
Toussaint, M 2009
Toussaint, M 2010 A Bayesian View on Motor Control and PlanningBerlinSpringerCrossRefGoogle Scholar
Trommershäuser, JGepshtein, SMaloney, L. TLandy, M. SBanks, M. S 2005 Optimal compensation for changes in task-relevant movement variabilityJ. Neurosci 25 7169CrossRefGoogle ScholarPubMed
Trommershäuser, JMaloney, L. TLandy, M. S 2003 Statistical decision theory and the selection of rapid, goal-directed movementsJ. Opt. Soc. Am. A 20 1419CrossRefGoogle ScholarPubMed
van Ee, RAdams, W. JMamassian, P 2003 Bayesian modeling of cue interaction: bistability in stereoscopic slant perceptionJ. Opt. Soc. Am. A 20 1398CrossRefGoogle ScholarPubMed
Wei, KKörding, K. P 2008 Relevance of error: what drives motor adaptation?J. Neurophys 90545Google Scholar
Weiss, YSimoncelli, E. PAdelson, E. H 2002 Motion illusions as optimal perceptsNat. Neurosci 5 598CrossRefGoogle ScholarPubMed
Wolpert, D. MGhahramani, ZJordan, M. I 1995 An internal model for sensorimotor integrationScience 269 1880CrossRefGoogle ScholarPubMed
Wozny, D. RBeierholm, U. RShams, L 2008 Human trimodal perception follows optimal statistical inferenceJ. Vis 8 1CrossRefGoogle ScholarPubMed
Zemel, R. SDayan, PPouget, A 1998 Probabilistic interpretation of population codesNeural Comput 10 403CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×