Identification and characterization of the farmers’ named crop varieties cultivated around the world is a major issue for conservation and sustainable use of plant genetic resources. Intraspecific diversity is strongly determined by farmers’ socio-cultural environment, but this has little been documented. In this paper, we tested, on a contact zone among three ethnolinguistic groups located on the Mount Kenya region, whether farmers’ socio-cultural differences have an impact on the morphological characteristics of the farmers’ named sorghum varieties. Eighteen qualitative morphological traits of the panicles were measured. We first compared the morphological diversity of the named varieties among ethnolinguistic groups using multivariate analysis of homogeneity of groups’ dispersion and tested their differentiation using permutational multivariate analysis of variance. Discriminant analysis of principal components was then used to categorize the morphological diversity without a priori, and this classification was compared with farmers’ local taxonomy (vernacular names) in the three ethnolinguistic groups. Our results show that some morphotypes are peculiar to some ethnolinguistic groups and that a morphotype can bear different variety names while the same variety name can be used to identify different morphotypes. Morphological differentiation that was explained by ethnolinguistic groups was higher for local landraces than for improved varieties. Our findings imply that socio-cultural diversity of farmers and the criteria they use to identify and maintain landraces need to be considered in studying and sampling crop diversity for in situ as well as for ex situ conservation.