What undermines cooperation in ethnically diverse communities? Scholars have focused on factors that explain the lack of inter-ethnic cooperation, such as prejudice or the difficulty to communicate and sanction across group boundaries. We direct attention to the fact that diverse communities are also often poor and ask whether poverty, rather than diversity, reduces cooperation. We developed a strategic cooperation game where we vary the income and racial identity of the interaction partner. We find that beliefs about how poor people behave have clear detrimental effects on cooperation: cooperation is lower when people are paired with low-income partners, and the effect is particularly strong when low-income people interact among themselves. We observe additional discrimination along racial lines when the interaction partner is poor. These findings imply that poverty and rising inequality may be a serious threat to social cohesion, especially under conditions of high socioeconomic segregation.