Classically, the simulation of primary recrystallization and grain growth starts from a theoretical microstructure defined numerically and characterized by a random texture distribution. The results obtained in this way represent well the overall behaviour of the material during annealing, but do not allow to obtain a local description, consistent with reality. In such procedures, a perfect homogeneity is assumed both for microstructure and for texture, which is generally not observed experimentally. For this reason, the authors have proposed in recent years to combine simulation and experimental observations; the calculation starts from the real microstructure characterized by TEM or by OIMTM. The first simulations of this kind carried out have shown that such an approach is feasible and has a great potential. These results are summarized and discussed in this paper. There are limits to this approach, related in particular to the lack of experimental data on the strain energies stored in the grains after deformation and on the energies and mobilities of grain boundaries.