In the latter half of the twentieth century, language theory adopted a formalist approach that focused on sentence generation. While generative models were originally intended as computational accounts of sentence grammaticality, they were later taken to reflect an inborn competence and a “language learning device.” Yet formal grammars were being worked out from analyses of script using orthographic units and categories. Several problems arose in bootstrapping grammatical categories in the mind to units like words and phrases, which entailed a shoehorning of concepts of syntax to children's speech. While theory-driven perception research sought to confirm children’s recognition of words, studies of speech corpora indicate that young children are not manipulating units like words as separate units but chunked clusters of items. In documenting these problems, recent evidence is discussed showing that syntactic category information may not be represented as properties of words in the brain, undermining the hypothesis of a language acquisition device.