It is unclear whether lifestyle modifications, such as dietary changes, should be advocated to prevent olfactory dysfunction. We investigated the association between dietary intakes of fats (saturated, mono-unsaturated and polyunsaturated fats, and cholesterol) and related food groups (nuts, fish, butter, margarine) with olfactory impairment. There were 1331 and 667 participants (older than 60 years) at baseline and 5-year follow-up, respectively, with complete olfaction and dietary data. Dietary data were collected using a validated semi-quantitative FFQ. Olfaction was measured using the San Diego Odor Identification Test. In a cross-sectional analysis of baseline data, those in the highest v. lowest quartile of n-6 PUFA intake had reduced odds of having any olfactory impairment, multivariable-adjusted OR 0·66 (95 % CI 0·44, 0·97), P for trend = 0·06. Participants in the highest v. lowest quartile of margarine consumption had a 65 % reduced odds of having moderate/severe olfactory impairment (P for trend = 0·02). Participants in the highest quartile compared to the lowest quartile (reference) of nut consumption had a 46 % (P for trend = 0·01) and 58 % (P for trend = 0·001) reduced odds of having any or mild olfactory impairment, respectively. Older adults in the highest v. lowest quartile of fish consumption had 35 % (P for trend = 0·03) and 50 % (P for trend = 0·01) reduced likelihood of having any or mild olfactory impairment, respectively. In longitudinal analyses, a marginally significant association was observed between nut consumption and incidence of any olfactory impairment, highest v. lowest quartile of nut consumption: OR 0·61 (95 % CI 0·37, 1·00). Older adults with the highest consumption of nuts and fish had reduced odds of olfactory impairment, independent of potential confounding variables.