This paper aims to develop a complete algorithm to determine the robot motion and the scene structure using a monocular vision system. It is based on straight lines and significant points extracted on them. In this way, an agreement between the problems to extract or to match points and the limitations of infinite lines to compute structure and motion is established. Many geometrical relations of the lines in the scene are exploited to clear up the coupling between the rotation and the translation of the camera. Several real images have been used to validate the proposed method. The algorithm has been considered for navigation of a mobile robot moving in man-made environments.