The maturity of current 3D rendering software in combination with recent developments in computer vision techniques enable an exciting range of applications for the visualisation, measurement and interactive manipulation of volumetric data, relevant both for diagnostic imaging and for anatomy. This paper reviews recent work in this area from the Image Sciences Institute at Utrecht University. The processes that yield a useful visual presentation are sequential. After acquisition and before any visualisation, an essential step is to prepare the data properly: this field is known as ‘image processing’ or ‘computer vision’ in analogy with the processing in human vision. Examples will be discussed of modern image enhancement and denoising techniques, and the complex process of automatically finding the objects or regions of interest, i.e. segmentation. One of the newer and promising methodologies for image analysis is based on a mathematical analysis of the human (cortical) visual processing: multiscale image analysis. After preprocessing the 3D rendering can be acquired by simulating the ‘ray casting’ in the computer. New possibilities are presented, such as the integrated visualisation in one image of (accurately registered) datasets of the same patient acquired in different modality scanners. Other examples include colour coding of functional data such as SPECT brain perfusion or functional magnetic resonance (MR) data and even metric data such as skull thickness on the rendered 3D anatomy from MR or computed tomography (CT). Optimal use and perception of 3D visualisation in radiology requires fast display and truly interactive manipulation facilities. Modern and increasingly cheaper workstations (<$10000) allow this to be a reality. It is now possible to manipulate 3D images of 2563 at 15 frames per second interactively, placing virtual reality within reach. The possibilities of modern workstations become increasingly more sophisticated and versatile. Examples presented include the automatic detection of the optimal viewing angle of the neck of aneurysms and the simulation of the design and placement procedure of intra-abdominal aortic stents. Such developments, together with the availability of high-resolution datasets of modern scanners and data such as from the NIH Visible Human project, have a dramatic impact on interactive 3D anatomical atlases.