“Theory of Mind” (ToM) is the capacity to deduce other persons’ cognitive and emotional states. Studies investigating affective ToM in healthy older adults and in persons with Alzheimer’s disease have reported contradictory results, although evidence indicates that advanced age (Ruffman, Henry, Livingstone, & Phillips, 2008) and Alzheimer’s disease (Elferink, van Tilborg, & Kessels, 2015) do not affect the ability to identify or infer different emotions to the same extent. To evaluate affective ToM abilities in these populations, we asked 63 individuals (17 with Alzheimer’s disease) to infer the emotional states of characters presented without facial details in emotional situations. We observed similar results in healthy younger and older adults, but poorer performance in persons with Alzheimer’s disease for disgust, sadness, and surprise, but not for anger, fear, and joy. Results suggest that persons with Alzheimer’s disease have difficulties in inferring several emotional states from contextual information without facial cues.