People tend to see faces from non-face objects or meaningless patterns. Such illusory face perception is called face pareidolia. Previous studies have revealed an interesting fact that there are huge individual differences in face pareidolia experience among the population. Here, we review previous findings on individual differences in face pareidolia experience from four categories: sex differences, developmental factors, personality traits and neurodevelopmental factors. We further discuss underlying cognitive or neural mechanisms to explain why some perceive the objects as faces while others do not. The individual differences in face pareidolia could not only offer scientific insights on how the brain works to process face information, but also suggest potential clinical applications.