This paper describes an algorithm for surface reconstruction from a set of scattered three-dimensional points extracted from an image sequence. In this process, additional information (such as location of the viewpoints and the points visible from a particular viewpoint) is available and can be exploited for accurately recovering the shape of the objects portrayed in the images. Initially, the set of points is subjected to a Delaunay triangulation that fills the convex hull of the set of points with disjoint tetrahedra. The key idea of the shape recovery algorithm is to eliminate triangles that obstruct the visibility of points from certain viewpoints, whose locations are known from the image acquisition process. The major contribution of this paper is that we have been able to design an algorithm for surface reconstruction that handles a wide variety of shapes, as opposed to currently existing techniques.