This perceptual-dialectology study investigates how listener-related social factors impact the geographical recognition of regional accents. In contrast to much prior research on English, our focus is on Dutch, which lends itself well to our study, allowing for notable regional accents within a well-defined standard language. Using a map-based recognition task in which 1,578 listeners placed forty representative speakers on a map based on fragments of their speech, we investigated the regional biases in accent recognition and the extent to which each listener’s awareness of these depends on their familiarity and proximity. Education, geographical knowledge, and distance to listeners’ own regions significantly predicted their accent-recognition accuracy. Moreover, we found a curvilinear age effect, which we interpret in terms of age-related changes in geographical and social mobility. We show how these effects in our design lead to meaningful accent-recognition patterns in groups of listeners.