Many decisions are curated, incentivised or nudged by a third party. Despite this, only a handful of studies have looked at paternalistic decision-makers and the psychological processes by which they arrive at their decisions. The role of affect, in particular, has been ignored so far, and yet restricting agency on a potentially large group of people might be highly unpleasant. We are the first to propose a conceptual framework of affective paternalism which explicitly accounts for the role of affect in paternalistic decision-making, identifying all entry points through which affect may create systematic deviations in decision outcomes. We shed light on some of these phenomena by using a novel survey experiment in which we let participants make paternalistic decisions whilst also asking them about their motivations behind their choices, including cognitive reasons and affect. Our findings suggest that affect may play a significant role in paternalistic decision-making and lead to systematically different decision outcomes. To the extent that these that could result in inefficient, undesirable or unfair consequences, our framework may help more accurately predict a paternalist's decision and suggest entry points for where and possibly how to intervene in the paternalistic decision-making process.