Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bf8c574d5-86b6f Total loading time: 0 Render date: 2025-03-09T22:37:02.742Z Has data issue: false hasContentIssue false

23 - An Integrated Reasoning Approach to Moral Decision Making

from PART IV - APPROACHES TO MACHINE ETHICS

Published online by Cambridge University Press:  01 June 2011

By
Morteza Dehghani ,
Ken Forbus ,
Emmett Tomai  and
Matthew Klenk
Edited by
Michael Anderson  and
Susan Leigh Anderson
Michael Anderson
Affiliation:
University of Hartford, Connecticut
Susan Leigh Anderson
Affiliation:
University of Connecticut
Get access

Summary

Introduction

Although traditional models of decision making in ai have focused on utilitarian theories, there is considerable psychological evidence that these theories fail to capture the full spectrum of human decision making (e.g. Kahneman and Tversky 1979; Ritov and Baron 1999). Current theories of moral decision making extend beyond pure utilitarian models by relying on contextual factors that vary with culture. In particular, research on moral reasoning has uncovered a conflict between normative outcomes and intuitive judgments. This has led some researchers to propose the existence of deontological moral rules; that is, some actions are immoral regardless of consequences, which could block utilitarian motives. Consider the starvation scenario (from Ritov and Baron [1999]) that follows:

A convoy of food trucks is on its way to a refugee camp during a famine in Africa. (Airplanes cannot be used.) You find that a second camp has even more refugees. If you tell the convoy to go to the second camp instead of the first, you will save one thousand people from death, but one hundred people in the first camp will die as a result.

Would you send the convoy to the second camp?

The utilitarian decision would send the convoy to the second camp, but 63 percent of participants did not divert the truck.

Making these types of decisions automatically requires an integrated approach, including natural language understanding, qualitative reasoning, analogical reasoning, and first-principles reasoning.

Type
Chapter
Information
Machine Ethics , pp. 422 - 441
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allen, J F. Natural Language Understanding. 2nd ed. Redwood City, CA: Benjamin/Cummings, 1995.Google Scholar
Anderson, M, S Anderson, and Armen, C. “An Approach to Computing Ethics.” IEEE Intelligent Systems 21, no. 4 (2006): 56–63.CrossRefGoogle Scholar
Atran, S, Axelrod, R, and Davis, R. “Sacred Barriers to Conflict Resolution.” Science 317 (2007): 1039–1040.CrossRefGoogle ScholarPubMed
Bang, M, J Townsend, S J Unsworth, and Medin, D L. “Cultural Models of Nature and Their Relevance to Science Education.” 2005.
Baron, J, and Spranca, M. “Protected Values.” Organizational Behavior and Human Decision Processes 70 (1997): 1–16.CrossRefGoogle ScholarPubMed
Bartels, Daniel M, and Medin, Douglas L. “Are Morally Motivated Decision Makers Insensitive to the Consequences of Their Choices?” Psychol Sci 18, no. 1 (Jan 2007): 24–28.CrossRefGoogle ScholarPubMed
Bennis, W M, Medin, D L, and Bartels, D M. “The Costs and Benefits of Calculation and Moral Rules.” Perspectives in Psychological Science 5 (2010): 187–202.CrossRefGoogle ScholarPubMed
Dague, P. “Model-based Diagnosis of Analog Electronic Circuits.” Annals of Mathematics and Artificial Intelligence 11 (1994): 439–492.CrossRefGoogle Scholar
Dague, P. “Numeric Reasoning with Relative Orders of Magnitude.” Proceedings of the 7th International Workshop on Qualitative Reasoning. 1993a.Google Scholar
Dague, P. “Symbolic Reasoning with Relative Orders of Magnitude.” Proceedings of the 13th International Joint Conference of Artificial Intelligence (IJCAI-93). 1993b.Google Scholar
Dague, P, Deves, P, and Raiman, O. “Troubleshooting: When Modeling is the Trouble.” Proceedings AAAI-87 Sixth National Conference on Artificial Intelligence. 1987.Google Scholar
Dehghani, M, D Gentner, K Forbus, Ekhtiari, H, and Sachdeva, S. “Analogy and Moral Decision Making.” Analogy09. 2009.Google Scholar
Dehghani, M, Sachdeva, S, Ekhtiari, H, Gentner, D, and Forbus, K. “The Role of Cultural Narratives in Moral Decision Making.” In Proceedings of the 31st Annual Conference of the Cognitive Science Society (CogSci-09). 2009.Google Scholar
Falkenhainer, B, Forbus, K, and Gentner, D. “The Structure-Mapping Engine: Algorithms and Examples.” Artificial Intelligence 41 (1989): 1–63.CrossRefGoogle Scholar
Fillmore, C. “Frame Semantics.” Edited by D Geeraerts. Berlin, New York: Mouton de Gruyter, 2006.
Forbus, K, and Oblinger, D. “Making SME Greedy and Pragmatic.” Proceedings of the Eighth Annual Conference of the Cognitive Science Society (CogSci-1990). 1990.Google Scholar
Forbus, K, Gentner, D, and Law, K. “MAC/FAC: A Model of Similarity-Based Retrieval.” Cognitive Science 19, no. 2 (1995): 141–205.CrossRefGoogle Scholar
Forbus, K, Usher, J, and Tomai, E. “Analogical Learning of Visual/Conceptual Relationships in Sketches.” Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05). 2005.Google Scholar
Forbus, K, Ferguson, R, and Gentner, D. “Incremental Structure-Mapping.” Proceedings of the 16th Annual Conference of the Cognitive Science Society (CogSci-1994). 1994.Google Scholar
Friedman, S E, and Forbus, K D. “Learning Causal Models via Progressive Alignment & Qualitative Modeling: A Simulation.” In the Proceedings of the 30th Annual Conference of the Cognitive Science Society (CogSci), Washington, D.C. 2008.Google Scholar
Frijda, N.The Emotions: Studies in Emotion and Social Interaction. New York, NY: Cambridge University Press, 1986.Google Scholar
Gentner, D. “Structure-Mapping: A Theoretical Framework for Analogy.” Cognitive Science 7, no. 2 (Apr.–Jun. 1983): 155–170.CrossRefGoogle Scholar
Gentner, D, and Markman, A B. “Structure Mapping in Analogy and Similarity.” American Psychologist 52 (1997): 45–56.CrossRefGoogle Scholar
Gentner, D, M J Rattermann, and Forbus, K. “The Roles of Similarity in Transfer: Separating Retrievability from Inferential Soundness.” Cognitive Psychology 25 (1993): 524–575.CrossRefGoogle ScholarPubMed
Gentner, D, Brem, S, Ferguson, R W, Wolff, P, Markman, A B, and Forbus, K. “Analogy and Creativity in the Works of Johannes Kepler.” Edited by Ward, T B, Smith, S M and Vaid, J, 403–459. Washington, DC: American Psychological Association, 1997.
Gratch, J, and Marsella, S. “A Domain Independent Framework For Modeling Emotion.” Journal of Cognitive Systems Research 5 (2004a): 269–306.CrossRefGoogle Scholar
Gratch, J, and Marsella, S. “Evaluating the Modeling and Use of Emotion in Virtual Humans.” Edited by Jennings, N. R., Sierra, C., Sonenberg, L., and Tambe, M.. Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems (AAMAS 2004). New York: ACM Press, 2004b. 320–327.Google Scholar
Haidt, J, Koller, S, and Dias, M. “Affect, Culture and Morality, or, is it Wrong to Eat Your Dog?” Journal of Personality and Social Psychology 65 (1993): 613–628.CrossRefGoogle ScholarPubMed
Hsee, C, and Weber, E. “Cross-National Differences in Risk Preference and Lay Predictions.” Journal of Behavioral Decision Making 12 (1999): 165–179.3.0.CO;2-N>CrossRefGoogle Scholar
Kahneman, D, and Tversky, A. “Prospect Theory: An Analysis of Decision under Risk.” American Psychologist 39, no. 4 (1979): 341–350.CrossRefGoogle Scholar
Kamp, H, and Reyle, U. From Discourse to Logic: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Vol. 42. Springer, 1993.
Kuehne, S, and Forbus, K. “Capturing QP-Relevant Information from Natural Language Text.” In Proceedings of the 18th International Workshop on Qualitative Reasoning (QR04). 2004.Google Scholar
Lazarus, R S. Emotion and Adaptation. New York, NY: Oxford University Press, 1991.Google Scholar
Lenat, D, and Gupta, R V. Building Large Knowledge Based Systems. Reading, MA: Addison Wesley, 1990.Google Scholar
Lim, C S, and Baron, J. Protected values in Malaysia Singapore, and the United States. Department of Psychology, University of Pennsylvania, 1997.Google Scholar
Macleod, C, R Grishman, and Meyers, A. “COMLEX Syntax Reference Manual, Version 3.0.” 1998.
Markman, A B, and Medin, D L. Decision Making. Vol. 2, edited by Medin, D L and Pashler, H, 413–466. New York: John Wiley and Sons, 2002.CrossRefGoogle Scholar
Mavrovouniotis, M L, and Stephanopoulos, G. “Formal Order-of-Magnitude Reasoning in Process Engineering.” Edited by D S Weld and J de Kleer, 323–336. Morgan Kaufmann Publishers Inc., 1990.
Mavrovouniotis, M L, and Stephanopoulos, G. “Order of Magnitude Reasoning in Process Engineering.” Computers & Chemical Engineering 12 (1988).CrossRefGoogle Scholar
Mavrovouniotis, M L, and Stephanopoulos, G. “Reasoning with Orders of Magnitude and Approximate Relations.” Proceedings of the Sixth National Conference on Artificial Intelligence. 1987.Google Scholar
McLaren, B M. “Computational Models of Ethical Reasoning: Challenges, Initial Steps, and Future Directions.” IEEE Intelligent Systems July/August (2006): 29–37.CrossRefGoogle Scholar
Ortony, A, Clore, G L, and Collins, A. The Cognitive Structure of Emotions. Cambridge: Cambridge University Press, 1988.CrossRefGoogle Scholar
Raiman, O. “Order of Magnitude Reasoning.” Artificial Intelligence 51 (1991): 11–38.CrossRefGoogle Scholar
Ritov, I, and Baron, J. “Protected Values and Omission Bias.” Organizational Behavior and Human Decision Processes 79, no. 2 (1999): 79–94.CrossRefGoogle ScholarPubMed
Shweder, R A, Much, N C, Mahapatra, M, and Park, L. “The ‘Big Three’ of Morality (Autonomy, Community, Divinity) and the ‘Big Three’ Explanations of Suffering.” Edited by Brandt, A. M., 119–172. Routledge Press, 1997.
Subrahmanian, V S, et al. “CARA: A Cultural Adversarial Reasoning Architecture.” IEEE Intelligent Systems 22, no. 2 (2007): 12–16.CrossRefGoogle Scholar
Tetlock, P E. “Cognitive Biases and Organizational Correctives: Do Both Disease and Cure Depend on the Ideological Beholder?” Administrative Science Quarterly 45, no. 2 (2000): 293–326.CrossRefGoogle Scholar
Tetlock, P E. “Thinking the Unthinkable: Sacred Values and Taboo Cognitions.” Trends in Cognitive Sciences 7 (2003): 320–324.CrossRefGoogle ScholarPubMed
Tomai, E, and Forbus, K D. “EA NLU: Practical Language Understanding for Cognitive Modeling.” In Proceedings of the Twenty-Second International FLAIRS Conference. 2009.
Tomai, E, and Forbus, K D. “Using Qualitative Reasoning for the Attribution of Moral Responsibility.” In the Proceedings of the 30th Annual Conference of the Cognitive Science Society (CogSci), Washington, D.C. 2008.
Waldmann, M R, and Dieterich, J. “Throwing a Bomb on a Person versus Throwing a Person on a Bomb: Intervention Myopia in Moral Intuitions.” Psychological Science 18, no. 3 (2007): 247–253.CrossRefGoogle ScholarPubMed
Weber, E U, Ames, D, and Blais, A R. “‘How Do I Choose Thee? Let Me Count the Ways’: A Textual Analysis of Similarities and Differences in Modes of Decision Making in China and the United States.” Management and Organization Review 1 (2005): 87–118.CrossRefGoogle Scholar
Weber, E, and Hsee, C. “Cross-Cultural Differences in Risk Perception, but Cross-Cultural Similarities in Attitudes towards Perceived Risk.” Management Science 44(9) (1998): 1205–1217.CrossRefGoogle Scholar
Winston, P H. “Learning New Principles from Precedents and Exercises.” Artificial Intelligence 19(3) (1982): 321–350.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×