Group selection has historically been an important and controversial subject in evolutionary biology. There is now a compelling body of evidence, both theoretical and experimental, that group selection not only can be effective, but can be effective in situations when individual selection is not. However, experiments in which true population-level traits have been shown to evolve in response to group selection are currently limited to two species of flour beetle in the genus Tribolium and RNA viruses. Here we report the results of an experiment wherein we imposed group selection via differential extinction for increased and decreased population size at 6-week intervals, a true population-level trait, in the poeciliid fish Heterandria formosa. In contrast to most other group selection experiments, we observed no evolutionary response after six rounds of group selection in either the up- or down-selected lines. Populational heritability for population size was low, if not actually negative. Our results suggest that group selection via differential extinction may be effective only if population sizes are very small and/or migration rates are low.