Moduli spaces of pointed curves with some level structure are studied. We prove that for so-called geometric level structures, the levels encountered in the boundary are smooth if the ambient variety is smooth, and in some cases we can describe them explicitly. The smoothness implies that the moduli space of pointed curves (over any field) admits a smooth finite Galois cover. Finally, we prove that some of these moduli spaces are simply connected.