Natural language syntax has previously been thought to reflect abstract processing rules independent of meaning construction. However, grammatical categories may serve a functional role by allocating attention towards recurrent topics in discourse. Here, we show that listeners incorporate grammatical category into imagery when producing stick figure drawings from heard sentences, supporting the latter view. Participants listened to sentences with transitive verbs that independently varied whether a male or a female character (1) was mentioned first, (2) was the agent or recipient of an action, and (3) was the grammatical subject or object of the sentence. Replicating previous findings, we show that the first named character as well as the agent of the sentence tends to be drawn to the left in the image, probably reflecting left-to-right reading direction. But we also find that the grammatical subject of the sentence has a propensity to be drawn to the left of the object. We interpret this to suggest that grammatical category carries discursive meaning as an attention allocator. Our findings also highlight how language influences processes hitherto thought to be non-linguistic.