Socio-economic gradients in diet quality are well established. However, the influence of material socio-economic conditions particularly in childhood, and the use of multiple disaggregated socio-economic measures on diet quality have been little studied in the elderly. In the present study, we examined childhood and adult socio-economic measures, and social relationships, as determinants of diet quality cross-sectionally in 4252 older British men (aged 60–79 years). A FFQ provided data on daily fruit and vegetable consumption and the Elderly Dietary Index (EDI), with higher scores indicating better diet quality. Adult and childhood socio-economic measures included occupation/father's occupation, education and household amenities, which combined to create composite scores. Social relationships included social contact, living arrangements and marital status. Both childhood and adult socio-economic factors were independently associated with diet quality. Compared with non-manual social class, men of childhood manual social class were less likely to consume fruit and vegetables daily (OR 0·80, 95 % CI 0·66, 0·97), as were men of adult manual social class (OR 0·65, 95 % CI 0·54, 0·79), and less likely to be in the top EDI quartile (OR 0·73, 95 % CI 0·61, 0·88), similar to men of adult manual social class (OR 0·66, 95 % CI 0·55, 0·79). Diet quality decreased with increasing adverse adult socio-economic scores; however, the association with adverse childhood socio-economic scores diminished with adult social class adjustment. A combined adverse childhood and adulthood socio-economic score was associated with poor diet quality. Diet quality was most favourable in married men and those not living alone, but was not associated with social contact. Diet quality in older men is influenced by childhood and adulthood socio-economic factors, marital status and living arrangements.