Scanning confocal electron microscopy (SCEM) is a new imaging technique that is capable of depth sectioning with nanometer-scale depth resolution. However, the depth resolution in the optical axis direction (Z) is worse than might be expected on the basis of the vertical electron probe size calculated with the existence of spherical aberration. To investigate the origin of the degradation, the effects of electron energy loss and chromatic aberration on the depth resolution of annular dark-field SCEM were studied through both experiments and computational simulations. The simulation results obtained by taking these two factors into consideration coincided well with those obtained by experiments, which proved that electron energy loss and chromatic aberration cause blurs at the overfocus sides of the Z-direction intensity profiles rather than degrade the depth resolution much. In addition, a deconvolution method using a simulated point spread function, which combined two Gaussian functions, was adopted to process the XZ-slice images obtained both from experiments and simulations. As a result, the blurs induced by energy loss and chromatic aberration were successfully removed, and there was also about 30% improvement in the depth resolution in deconvoluting the experimental XZ-slice image.