The present study is, to our knowledge, the first self-paced reading experiment to investigate the effects of syntactic violation and semantic ambiguity on processing English modal auxiliaries. Forty undergraduate students, native speakers of English, took part in the study and read 36 target sentences, each containing a modal verb in context. Two of the most frequent English modals, can and may, were used in three distinct categories of modal expression: agent-oriented/ability, epistemic possibility, and speaker-oriented/permission. The two modal auxiliaries were manipulated such that they were either congruent or incongruent with the context, or in the case of permission felicitous or infelicitous relative to the context. We found that incongruent modal use in an agent-oriented context resulted in a reading penalty that was observed in a spillover on the segments following the modal and the lexical verb. Incongruent modal use to express epistemic possibility significantly affected reading times immediately after the modal auxiliary, and also spilt over to the following segments. Reading times in sentences expressing speaker-oriented modality were not affected by inconsistency in the use of the modal verb unmarked for formality in a formal context. The substantive and methodological implications of findings are discussed.