Mycobacterial bone marrow (BM) infection is the most common diagnosis established by BM examinations for fever of unknown origin. In this study, clinical features and outcomes of patients who fulfilled the criteria for BM infection due to Mycobacterium tuberculosis (MTB) and non-tuberculous mycobacteria (NTM) at a medical centre in Taiwan from 2001 to 2009 were investigated. The BM histopathological findings were also analysed. A total of 24 patients (16 men, eight women) with mycobacterial BM infections were found. Of these, nine (38%) were positive for human immunodeficiency virus (HIV) and six (25%) had no pre-existing immunocompromised conditions. MTB isolates were obtained from 11 (46%) patients and NTM species were isolated from 10 (42%) patients, including M. avium complex (MAC, n = 7) and M. kansasii (n = 3). Patients with MTB infections were significantly older than those with NTM infections (60·5 vs. 47·7 years, P = 0·043) and were less likely to have a positive BM culture (45% vs. 100%, P = 0·012). The 90-day survival rates for MTB and NTM BM infections were 68% and 60%, respectively (P = 0·61). In addition, the presence of BM granulomas was significantly more common in patients with MTB BM infections than in those with NTM infections (82% vs. 30%, P = 0·030). In Taiwan, the importance of NTM was not inferior to MTB and besides MAC, M. kansasii might be an important pathogen in non-HIV-infected patients. The presence of BM granulomas and caseation provides valuable information regarding early treatment pending culture results.