The Glomeromycota is the newest fungal phylum. These are the arbuscular mycorrhizal (AM) fungi that form symbioses with the majority of land plant species. Fossil and molecular evidence suggest this is an ancient symbiosis, that may have been instrumental in enabling plants to colonise terrestrial habitats. The AM fungi gain carbon from their plant host, and the primary benefit to plants is thought to be the acquisition of phosphate, a highly immobile ion in soil. However, they are thought to have many other ecological benefits. Host specificity has been considered to be low in this group, as most fungi in culture will colonise most plants, but recent evidence suggests that some AM fungi show a degree of specificity. We review the evidence revealing the genetic structure of this ancient group of fungi. Although a consensus has not yet been reached, we suggest that understanding the evolution and genetic structure of this enigmatic group may be the key to understanding how they function in ecosystems.