This chapter deals with how microeconomics can provide insights into the key challenge that artificial intelligence (AI) scientists face. This challenge is to create intelligent, autonomous agents that can make rational decisions. In this challenge, they confront two questions: what decision theory to follow and how to implement it in AI systems. This chapter provides answers to these questions and makes three contributions. The first is to discuss how economic decision theory – expected utility theory (EUT) – can help AI systems with utility functions to deal with the problem of instrumental goals, the possibility of utility function instability, and coordination challenges in multiactor and human–agent collective settings. The second contribution is to show that using EUT restricts AI systems to narrow applications, which are “small worlds” where concerns about AI alignment may lose urgency and be better labeled as safety issues. The chapter’s third contribution points to several areas where economists may learn from AI scientists as they implement EUT.

Belief is often formalized using tools of probability theory. However, probability theory often focuses on simple examples – like coin flips or basic parametric distributions – and these do not describe much about actual human thinking. I highlight some basic examples of the complexity and richness of human mental representations and review some work which attempts to marry plausible types of representations with probabilistic models of belief, one of the most exciting current directions in psychology and machine learning.

This chapter introduces Bayesian probability and the rules of probability theory, emphasizing that Bayesian probability emerges as the uniquely consistent extension of deductive (Boolean) logic to contexts of uncertainty and incomplete information