Micro-Electro-Mechanical System (MEMS)-based inertial technology has recently evolved. It holds remarkable potential as the future technology for various navigation related applications. This is mainly due to the significant reduction in size, cost, and weight of MEMS sensors. A major drawback of low-cost MEMS-based inertial sensors, however, is that their output signals are contaminated by high-level noise. Unless the high frequency noise component is suppressed, optimizing the pre-filtering methodology cannot be achieved. This paper proposes a neural network-based de-noising model for MEMS-based inertial data. A modular, three-layer feedforward neural network trained using the back-propagation algorithm is used for this purpose. Simulated and real MEMS-based inertial data sets are used to validate the model. It is shown that the model is capable of reducing the noise of the Crossbow's AHRS300CA IMU data by over one order of magnitude without altering the stochastic nature of the original signal. This is of utmost importance in developing a generic stochastic model for MEMS-based inertial data. A comparison between the developed neural network model and the wavelet de-noising method is made to further validate the model. It is shown that achieving the same level of noise suppression with wavelet-based de-noising model changes the stochastic characteristics of original signal.