The challenges facing those working on machine ethics can be divided into two main categories: philosophical concerns about the feasibility of computing ethics and challenges from the AI perspective. In the first category, we need to ask first whether ethics is the sort of thing that can be computed. One well-known ethical theory that supports an affirmative answer to this question is Act Utilitarianism. According to this teleological theory (a theory that maintains that the rightness and wrongness of actions is determined entirely by the consequences of the actions), the right act is the one, of all the actions open to the agent, which is likely to result in the greatest net good consequences, taking all those affected by the action equally into account. Essentially, as Jeremy Bentham (1781) long ago pointed out, the theory involves performing “moral arithmetic.”

Of course, before doing the arithmetic, one needs to know what counts as “good” and “bad” consequences. The most popular version of Act Utilitarianism – Hedonistic Act Utilitarianism – would have us consider the pleasure and displeasure that those affected by each possible action are likely to receive. As Bentham pointed out, we would probably need some sort of scale to account for such things as the intensity and duration of the pleasure or displeasure that each individual affected is likely to receive. This is information that a human being would need to have, as well, in order to follow the theory.