Objectives: White matter (WM) integrity within the mesial temporal lobe (MTL) is important for episodic memory (EM) functioning. The current study investigated the ability of diffusion tensor imaging (DTI) in MTL WM tracts to predict 3-year changes in EM performance in healthy elders at disproportionately higher genetic risk for Alzheimer’s disease (AD). Methods: Fifty-one cognitively intact elders (52% with family history (FH) of dementia and 33% possessing an Apolipoprotein E ε4 allelle) were administered the Rey Auditory Verbal Learning Test (RAVLT) at study entry and at 3-year follow-up. DTI scanning, conducted at study entry, examined fractional anisotropy and mean, radial and axial diffusion within three MTL WM tracts: uncinate fasciculus (UNC), cingulate-hippocampal (CHG), and fornix-stria terminalis (FxS). Correlations were performed between residualized change scores computed from RAVLT trials 1–5, immediate recall, and delayed recall scores and baseline DTI measures; MTL gray matter (GM) and WM volumes; demographics; and AD genetic and metabolic risk factors. Results: Higher MTL mean and axial diffusivity at baseline significantly predicted 3-year changes in EM, whereas baseline MTL GM and WM volumes, FH, and metabolic risk factors did not. Both ε4 status and DTI correlated with change in immediate recall. Conclusions: Longitudinal EM changes in cognitively intact, healthy elders can be predicted by disruption of the MTL WM microstructure. These results are derived from a sample with a disproportionately higher genetic risk for AD, suggesting that the observed WM disruption in MTL pathways may be related to early neuropathological changes associated with the preclinical stage of AD. (JINS, 2016, 22, 1005–1015)