This paper presents an approach to visual obstacle avoidance and reactive robot navigation for outdoor and indoor environments. The obstacle detection algorithm includes an image feature tracking procedure followed by a feature classification process based on the IPT (Inverse Perspective Transformation). The classifier discriminates obstacle points from ground points. Obstacle features permit to draw out the obstacle boundaries which are used to construct a local and qualitative polar occupancy grid, analogously to a visual sonar. The navigation task is completed with a robocentric localization algorithm to compute the robot pose by means of an EKF (Extended Kalman Filter). The filter integrates the world coordinates of the ground points and the robot position in its state vector. The visual pose estimation process is intended to correct possible drifts on the dead-reckoning data provided by the proprioceptive robot sensors. The experiments, conducted indoors and outdoors, illustrate the range of scenarios where our proposal has proved to be useful, and show, both qualitatively and quantitatively, the benefits it provides.