Spectroscopic observations of galaxies at high redshift has recently been revolutionised by the Multi Unit Spectroscopic Explorer (MUSE) instrument in operation at the VLT since 2014. Thanks to its unrivalled capabilities, MUSE has been able to increase by an order of magnitude the number of spectroscopic redshifts in these fields. The most spectacular increase is at high redshift (z > 3), where MUSE was able to detect thousands of Lyman-alpha emitters. In the deepest exposures, MUSE is even able to goes beyond the limiting magnitude of the deepest HST exposures. These observations have led to a breakthrough in our understanding of the high redshift universe: e.g. the discovery of Lyman-alpha emission from the circumgalactic medium around individual galaxies, the role and property of low mass galaxies. In this talk I will present the latest results obtained with the MUSE observations of the Hubble deep and ultra-deep fields.