This study examined factors affecting adherence in a 16-week tai chi program among multi-ethnic middle-aged and older adults living in a low socioeconomic environment in Toronto. Analysis was based on data collected from three tai chi program cohorts that took place from August 2009 to March 2012. The main outcome variable, adherence, was measured by the total number of sessions attended by each of the participants. Total sample size was 210 participants with a mean age of 68.1 ± 8.6. Based on the regression model, greater adherence was significantly associated with older age, greater perceived stress, higher education, and higher mental and physical scores of Short Form-36 components. Conversely, lower adherence was significantly associated with higher baseline weekly physical activity. Our findings suggest that we target less-educated individuals with poor mental and physical health to optimize adherence for future community-based tai chi programs.