Several recent papers have studied lensing of the CMB by large-scale structures, which probes the projected matter distribution from $z=10^3$ to $z\simeq 0$. This interest is motivated in part by upcoming high resolution, high sensitivity CMB experiments, such as APEX/SZ, ACT, SPT or Planck, which should be sensitive to lensing. In this paper we examine the reconstruction of the large-scale dark matter distribution from lensed CMB temperature anisotropies. We go beyond previous work in using numerical simulations to include higher order, non-Gaussian effects and find that the convergence and its power spectrum are biased, with the bias increasing with the angular resolution. We also study the contamination by the kinetic Sunyaev-Zel'dovich signal, which is spectrally indistinguishable from lensed CMB anisotropies, and find that it leads to an overestimate of the convergence. We finish by estimating the sensitivity of the previously cited experiments and find that all of them could detect the lensing effect, but would be biased at around the 10% level.To search for other articles by the author(s) go to: http://adsabs.harvard.edu/abstract_service.html