This paper studies the parameters contained in the truncated Fourier series (TFS) formulation for bipedal walking balance control. Using the TFS generated lateral motion reference, 3D bipedal walking can be directly achieved without any parameter adjustment. Furthermore, the potential of this TFS formulation for motion balance control has also been investigated. One more motion balance strategy is developed through the reinforcement learning, which adjusts the motion's reference trajectory according to the selected dynamic feedback in real time. Dynamic simulation results of the presented balance control method show that the resulting motion can be constrained periodical and long-distance 3D bipedal walking motions are achievable.