In recent years, autonomous control based on contact inspections in unknown environments is a new hot and difficult point in robotics research. This paper presents a new control law for unmanned aerial manipulator (UAM) to perform contact inspection tasks on vertical surfaces. The selected circular image feature decouples the position and attitude of UAM, so an image-based impedance control is proposed to control the position and track the contact force. The developed controller uses geometric methods to control the attitude. In addition, the designed aerial manipulator decouples the roll and pitch of the UAV from the UAV, which improves the system’s stability. Experiments have been carried out to demonstrate the feasibility of this method.