The Position and Orientation System (POS) is a special Strapdown Inertial Navigation System (SINS)/Global Positioning System (GPS) integrated system, widely employed in airborne remote sensing. In-Flight Alignment (IFA) is an effective way to improve the accuracy and speed of initial alignment for an airborne POS. IFA is normally accomplished with references from the position and velocity of GPS for SINS, so that unstable GPS measurements will result in poor alignment accuracy. To improve alignment accuracy under unstable GPS conditions, an adaptive filtering algorithm of the Second-order Divided Difference filter (DD2) based on adaptive innovation estimation is proposed, which introduces calculated innovation covariance directly into computation of the filter gain matrix. Then, the adaptive DD2 algorithm is used for the IFA of the POS with a large initial heading error. To validate the proposed algorithm, simulations are undertaken, followed by IFA experiments for the prototype of the airborne POS (TX-F30) under a turning manoeuvre in a car-mounted experiment, and under an “8” manoeuvre in-flight. The simulations and experimental results show that the proposed algorithm can reach better alignment accuracy under unknown statistical characteristic of GPS measurement noises.