The current study examined independent and interactive effects between Apolipoprotein E (APOE) genotype and two types of cognitively-stimulating lifestyle activities (CSLA)—integrated information processing (CSLA-II) and novel information processing (CSLA-NI)—on concurrent and longitudinal changes in cognition. Three-wave data across 6 years of follow-up from the Victoria Longitudinal Study (n=278; ages 55–94) and linear mixed model analyses were used to characterize the effects of APOE genotype and participation in CSLA-II and CSLA-NI in four cognitive domains. Significant CSLA effects on cognition were observed. More frequent participation in challenging activities (i.e., CSLA-NI) was associated with higher baseline scores on word recall, fact recall, vocabulary and verbal fluency. Conversely, higher participation in less cognitively-challenging activities (i.e., CSLA-II) was associated with lower scores on fact recall and verbal fluency. No longitudinal CSLA-cognition effects were found. Two significant genetic effects were observed. First, APOE moderated CSLA-II and CSLA-NI associations with baseline verbal fluency and fact recall scores. Second, APOE non-ɛ4 carriers’ baseline performance were more likely to be moderated by CSLA participation, compared to APOE-ɛ4 carriers. Our findings suggest APOE may be a “plasticity” gene that makes individuals more or less amenable to the influence of protective factors such as CSLA. (JINS, 2014, 20, 1–9)