A Pneumatic Muscle Actuator (PMA) is a new pneumatic component sharing similar characteristics with biological muscles, and the flexible manipulator actuated by PMAs can better reflect the flexibility of the mechanism. First and foremost, based on the study of the characteristics of human shoulder joints, the configuration design of the flexible manipulator is analyzed, and its kinematics and dynamics models are established. Furthermore, with regard to the nonlinearity, time-invariance and uncertainty of the control system, three aspects of improvement are proposed, which are based on the Radial Basis Function (RBF) network torque control algorithm. The Genetic Algorithm is used to optimize the initial values of RBF network parameters; RBF network parameters are adjusted dynamically by using the additional momentum method; the Levenberg--Marquardt (LM) algorithm, instead of the gradient descent method, is adopted to adjust Proportion Integration Differentiation (PID) parameters online in real time. At last, to test the effects that the improved algorithm exerts on the flexible manipulator control system, some physical platform experiments are carried out. It turns out that the control accuracy and robustness of the improved algorithm are well improved, and the mechanism can be controlled better to track the circular arc trajectory. It lays fundamental importance to the practical application for the working environment.