Precise targeting of retinal projections is required for the normal development of topographic maps in the mammalian primary visual system. During development, retinal axons project to and occupy topographically appropriate positions in the dorsal lateral geniculate nucleus (dLGN) and superior colliculus (SC). Phr1 retinal mutant mice, which display mislocalization of the ipsilateral retinogeniculate projection independent of activity and ephrin-A signaling, were found to have a more global disruption of topographic specificity of retinofugal inputs. The retinocollicular projection lacks local refinement of terminal zones and multiple ectopic termination zones originate from the dorsal–nasal (DN) retinal quadrant. Similarly, in the dLGN, the inputs originating from the contralateral DN retina are poorly refined in the Phr1 mutant. These results show that Phr1 is an essential regulator of retinal ganglion cell projection during both dLGN and SC topographic map development.