Introduction

The first competence required of a motion analysis system is the accurate and robust measurement of image motion. This chapter addresses the problem of tracking independently–moving (and possibly non–rigid) objects in a long, monocular image sequence. “Corner features” are automatically identified in the images and tracked through successive frames, generating image trajectories. This system forms the low–level front–end of our architecture (cf. Figure 1.1), making reliable trajectory computation of the utmost importance, for these trajectories underpin all subsequent segmentation and motion estimation processes.

We build largely on the work of Wang and Brady [156, 157], and extend their successful corner–based stereo algorithm to the motion domain. Their key idea was to base correspondence on both similarity of local image structure and geometric proximity. There are, however, several ways in which motion correspondence is more complex than stereo correspondence [90]. For one thing, objects can change between temporal viewpoints in ways that they cannot between spatial viewpoints, e.g. their shape and reflectance can alter. For another, the epipolar constraint is no longer hard–wired by once–off calibration of a stereo–rig; motion induces variable epipolar geometry which has to be continuously updated (if the constraint is to be used). Furthermore, motion leads to arbitrarily long image sequences (instead of frame–pairs), which requires additional tracking machinery. The benefits are that temporal integration facilitates noise resistance, resolves ambiguities over time, and speeds up matching (via prediction).

Our framework has two parts: the matcher performs two–frame correspondence while the tracker maintains the multi-frame trajectories. Each corner is treated as an independent feature at this level (i.e. assigned an individual tracker as in [26]), and is tracked purely within the image plane. Section 2.2 justifies this feature–based approach and establishes the utility of corners as correspondence tokens.