The evolution of agriculture improved food security and enabled significant increases in the size and complexity of human groups. Despite these positive effects, some societies never adopted these practices, became only partially reliant on them, or even reverted to foraging after temporarily adopting them. Given the critical importance of climate and biotic interactions for modern agriculture, it seems likely that ecological conditions could have played a major role in determining the degree to which different societies adopted farming. However, this seemingly simple proposition has been surprisingly difficult to prove and is currently controversial. Here, we investigate how recent agricultural practices relate both to contemporary ecological opportunities and the suitability of local environments for the first species domesticated by humans. Leveraging a globally distributed dataset on 1,291 traditional societies, we show that after accounting for the effects of cultural transmission and more current ecological opportunities, levels of reliance on farming continue to be predicted by the opportunities local ecologies provided to the first human domesticates even after centuries of cultural evolution. Based on the details of our models, we conclude that ecology probably helped shape the geography of agriculture by biasing both human movement and the human-assisted dispersal of domesticates.