Nulling interferometry is one of the promising techniques for the study of extra terrestrial planets. This technique will be applied in the future space missions Darwin and TPF-I, and from the ground with GENIE. The nulling interferometry techniques require high symmetry of the interfering beams, to obtain the required contrast (typically $10^6$ to detect terrestrial exo-planets in the thermal infrared). In this paper we consider the polarization symmetry issue, such as polarization rotation and polarization phase shifts occurring on slightly misaligned optics. We study the consequences of these symmetry requirements on a nulling interferometer design. We find the relation between the misalignment tolerances and the achievable nulling, and we show that this tolerance is highly dependent on the interferometer configuration (the way beams turn right, left, up or down in the interferometer arms). It is typically of the order of the arcminute (not the arcsecond) for a $10^6$ contrast. We present a analytical and numerical analyses.