The future PRIMA facility at the Very Large Telescope Interferometer (VLTI) in astrometric mode offers the possibility to perform relative narrow-angle astrometry with 10 micro-arcsecond accuracy. This is achieved with a dual-beam interferometer concept, where a reference star and the scientific target, confined in a 60 arcsecond field, are observed simultaneously. The angular separation of the two stellar objects gives rise to an optical delay in the interferometer, which is measured by the Fringe Sensor Unit (FSU) and an internal laser metrology. PRIMA is using two FSU fringe detectors, each observing the interference of stellar beams coming from one of the two objects and measuring the corresponding phase and group delay. The astrometric observable, yielding the angular separation, is deduced from the group delay difference observed between the two objects. In addition, the FSU phase delay estimate is used as error signal for the fringe stabilisation loop of the VLTI. Both functions of the FSU require high precision fringe phase measurements with a goal of 1 nm rms (corresponding to λ/2000). These can only be achieved by applying a calibration procedure prior to the observing run. We discuss the FSU measurement principle and the applied algorithms. The calibration strategy and the methods used to derive the calibration parameters are presented. Special attention is given to the achieved measurement linearity and repeatability. The quality of the FSU calibration is crucial in order to achieve the ultimate accuracy and to fulfill the primary objective of PRIMA astrometry: the detection and characterisation of extrasolar planetary systems.