More able individuals with autism and Asperger syndrome (AS) have been shown to pass relatively high level theory of mind (ToM) tasks without displaying commensurate levels of social adaptation in naturalistic settings. This paper presents a social cognitive procedure—the Social Attribution Task (SAT)—that reduces factors thought to facilitate ToM task performance without facilitating real-life social functioning. Sixty participants with autism (N = 20), AS (N = 20), and normally developing adolescents and adults (N = 20) with normative IQs were asked to provide narratives describing Heider and Simmel's (1944) silent cartoon animation in which geometric shapes enact a social plot. These narratives were coded in terms of the participants' abilities to attribute social meaning to the geometric cartoon. The SAT provides reliable and quantified scores on seven indices of social cognition. Results revealed marked deficits in both clinical groups across all indices. These deficits were not related to verbal IQ or level of metalinguistic skills. Individuals with autism and AS identified about a quarter of the social elements in the story, a third of their attributions were irrelevant to the social plot, and they used pertinent ToM terms very infrequently. They were also unable to derive psychologically based personality features from the shapes' movements. When provided with more explicit verbal information on the nature of the cartoon, individuals with AS improved their performance slightly more than those with autism, but not significantly so.