1.1 The Dual-Route Access to Meaning

The traditional view of language understanding relies on compositionality: The meaning of a sentence (or a discourse) is a function of the meaning of its constituents and the way they are syntactically combined (Partee, Reference Partee, Gleitman and Liberman1995). More precisely, the associated theoretical strategy sustained in the generative tradition (Chomsky, Reference Chomsky1965; Pinker, Reference Pinker1999) is that syntax provides the primary mode of meaning combination in language: Syntactic rules do no more than determine which symbol sequence works as units for syntactic purposes, while meaning derives from the lexical conceptual structure. In other words, reading a sentence consists of linearly accessing the meaning of the words stored in the lexicon and then integrating them within the abstract hierarchical structure. This stance encourages a bottom-up, or building block, model of meaning where the interpretation mechanism is incremental: The meanings of the words are composed into syntactic units and aggregated until reaching a complete interpretation.

Nevertheless, there is extensive experimental evidence from psycholinguistic and neurolinguistic research against traditional compositionality (Baggio, Reference Baggio2021; Baggio, Van Lambalgen, & Hagoort, Reference Baggio, Van Lambalgen, Hagoort, Hinzen, Werning and Machery2012; Ferreira, Bailey, & Ferraro, Reference Ferreira, Bailey and Ferraro2002; Ferreira & Patson, Reference Ferreira and Patson2007; Mollica et al., Reference Mollica, Siegelman and Diachek2020, among others). These findings can be synthesized into two fundamental observations (Baggio, Reference Baggio2018, p. 19):

• A comprehender generates interpretations based on the semantic relations between words, not necessarily encoded or reflected by the grammar, and

• A comprehender tends to generate semantic representations that could bypass or collide with syntactic analyses, resulting in superficial and even inaccurate interpretations.

These observations argue for a more top-down model, in which our semantic expectations drive comprehension: The linguistic elements function as cues to activate our extensive linguistic knowledge stored in the long-term memory. This knowledge encompasses various facets, such as the frequency of use of certain expressions, common associations, and event schemas. Consequently, linguistic knowledge transcends mere lexical components and transforms into the “cognitive organization of one’s experience with language” (Bybee, Reference Bybee2006, p. 771). Moreover, experimental studies have revealed a consistent pattern: When the comprehender accesses information congruent with their preexisting or pre-activated knowledge, facilitation effects in processing occur (faster reading times, low N400, etc.). The linguistic theories rooted in these assumptions endorse a noncompositional mechanism of meaning interpretation. They advocate a model of language where linguistic knowledge comes from direct linguistic experience and sentence processing is constraint-based, probabilistic, and reliant on expectations.

Although the ongoing debate has opened the possibility that these two strategies (compositional and noncompositional access to meaning) are not mutually incompatible, few linguistic theories have thus far bothered to elucidate how the two mechanisms could be integrated within a unified framework of sentence processing. Notably, theories within the fields of psycholinguistics, theoretical linguistics, and computational linguistics have offered different perspectives on uncovering the cognitive systems behind language, describing their characteristics, and modeling them. On one end, experimental results from different research areas have shed light on the mechanisms that could underlie human language comprehension. However, our knowledge about the language system remains scattered: Psycholinguistic studies usually focus on language processing subtasks (e.g., lexical access) or modules (e.g., morphology, syntax) without being aggregated into a unified framework. Conversely, linguistic theories provide a rigorous formalism for language description, but they still have difficulty integrating the variability of language productions observed in behavioral experiments. To this day, it remains challenging to find in the existing literature a comprehensive model that unifies the different observations on language understanding into a unique architecture (Blache, Reference Blache, Sharp, Sèdes and Lubaszewski2017). However, there is a family of linguistic theories whose fundamental assumption could make it possible to integrate findings from multiple research fields.

1.2 A Constructionist View of Language

The behavioral evidence surrounding noncompositionality points to a model of linguistic representation that is in line with the assumptions of the usage-based models of language (Bybee, Reference Bybee2010; Croft, Reference Croft1991, Reference Croft2001; Langacker, Reference Langacker1987; Tomasello, Reference Tomasello2009) and the Construction Grammar (CxG) paradigm (Hilpert, Reference Hilpert2019; Hoffmann, Reference Hoffmann2022b; Hoffmann & Trousdale, Reference Hoffmann and Trousdale2013; Ungerer & Hartmann, Reference Ungerer and Hartmann2023). CxG refers to a family of models based on the assumption that grammar is more than simply a formal system consisting of stable but arbitrary rules for defining well-formed sequences. Besides their specificities, all constructionist theories agree on a fundamental claim: Grammar consists of meaningful and symbolic form–meaning mappings, called constructions (Goldberg, Reference Goldberg1995, Reference Goldberg2006, Reference Goldberg2019). The definition and operationalization of “construction” are still under debate, and each formalism proposes a slightly different criterion (see Ungerer & Hartmann, Reference Ungerer and Hartmann2023 for a recent discussion on the definition of constructions). In the more general sense, constructions are processing units or chunks, from morphemes or words, to partially and fully lexicalized expressions, to schematic and productive patterns of language – such as Passive or Ditransitive constructions (Goldberg, Reference Goldberg2003), up to even genres and text types (Hoffman & Bergs, Reference Hoffman and Bergs2018). For instance, the comparative correlative construction (Hoffmann, Brunner, & Horsch, Reference Hoffmann, Brunner and Horsch2020; also “Covariational Conditional”; cf. Culicover & Jackendoff, Reference Culicover and Jackendoff1999) “The Xer, the Yer,” such as

1. (2) The more you know, the less you understand.

has specific syntactic and semantic properties. First, both clauses are introduced by an element that “resembles” the English definite article the, which instead is followed by comparative phrases. Semantically, English speakers identify the cause–effect relationship between the two clauses, which is not marked at any syntactic level. Thus, both the syntax and the meaning of the construction are not entirely predictable by any abstract rule.

At the same time, a syntactic pattern like the Double Object construction has a meaning independently of the words that compose the construction. People reading a sentence like

1. (3) She mooped him something.

interpret the made-up word moop as “to give” because the abstract pattern V Obj ${}_1$ Obj ${}_2$ communicates itself the concept of transfer between two persons (Goldberg, Reference Goldberg2019, p. 29).

Constructions form a structured inventory of a speaker’s knowledge of the conventions of their language, called the constructicon (Diessel, Reference Diessel2023): Each construction constitutes a node in the taxonomic network of constructions, and taxonomic relations allow us to distinguish different types of grammatical knowledge. However, there is no complete agreement about how such taxonomies emerge. Formal models such as Sign-Based CxG (Sag, Reference Sag, Boas and Sag2012) assume that only “idiosyncratic morphological, syntactic, lexical, semantic, pragmatic or discourse-functional properties must be represented as an independent node in the constructional network in order to capture a speaker’s knowledge of their language” (Croft & Cruse, Reference Croft and Cruse2004, p. 263). Conversely, usage-based approaches – that is, Radical CxG (Croft, Reference Croft2001) or Cognitive CxG (Goldberg, Reference Goldberg2003) – advocate that constructions can be of any linguistic pattern used enough to be memorized (or entrenched; cf. Blumenthal-Dramé, Reference Blumenthal-Dramé2012) in the long-term memory (Goldberg, Reference Goldberg2006). Specifically, the assumption is that linguistic units that are more frequently encountered become more accessible and are preferred. According to this thesis, the most entrenched linguistic units tend to shape the language system in terms of patterns of use, at the expense of less frequent and thus less well-entrenched words or phrases. This account opens to a more redundant view of the lexicon: Although we do not technically need to memorize the word form cats because, in principle, it can be formed with a productive rule cat+s, it could be memorized in the lexicon because we have encountered it thousands of times in everyday language (Hilpert, Reference Hilpert2021, p. 21). The same observation could be done for phrases. For instance, read a book is a semantic transparent chunk; that is, we can identify the meanings of its parts and how they were combined to generate the final interpretation. Even though we could build the meaning of the expression “on the fly” using a compositional, incremental mechanism, this sequence was heard and used so many times to be stored as a whole; thus, interpretation becomes the act of retrieving the stored meaning. Using Goldberg’s words, “memory is cheap. There is a good deal of evidence that we retain an enormous amount of information about the language(s) we witness” (Goldberg, Reference Goldberg2019, p. 54).

However, that does not mean that people retain frequently observed word combinations as atomic units (as this would quickly result in a combinatorial explosion), but memory traces have an internal structure. Thus, representations of related memories overlap neurally, mitigating the concern about a combinatorial explosion (Goldberg, Reference Goldberg2019). Moreover, the cognitive capacity of pattern detection and schematization (Bybee, Reference Bybee2010) allows the storage of more abstract construction from specific instances. Going back to the previous example, speakers redundantly store frequent plural forms in addition to a general plural construction, or an entire expression together with the abstract transitive pattern.

Placing constructions as the fundamental unit of language has the consequence of blurring the distinction between words of the lexicon and the rules of grammar. Contrary to generative theories, CxG argues that the architecture of the grammar is not layered in distinct modules, but different properties (morphological, prosodic, syntactic, semantic) together constitute the form that allows the construction to be identified, and when it is recognized, it is possible to access the associated meaning directly. This holistic view emphasizes the importance of surface structure, that is, the concrete utterances that a hearer is exposed to, as opposed to mainstream generative grammar, which primarily focuses on hidden syntactic processes not directly observable in the final output (Goldberg, Reference Goldberg, Hoffmann and Trousdale2013). As a joint representation of syntax and semantics, constructions provide a powerful mechanism for investigating many different linguistic phenomena (Diessel, Reference Diessel2019).

Despite the vast possibilities this framework offers for linguistic description and language modeling, some issues are still to be addressed. For instance, while we agree with the assumption that the lexicon is a repository of constructions, it is unclear which factors drive the memorization of specific chunks. The question about which constructions are stored in long-term memory and which aspects can be constructed online in working memory is yet to be fully answered and has consequences on the mechanisms governing sentence processing. In that regard, one more issue has to be figured out: What is the most appropriate and acceptable representation of constructional meaning? Toward a complete model of language comprehension, a further challenge is to give a semantic representation that could be coherent with the usage-based perspective and could account for the evidence that lexical knowledge is quite detailed, often idiosyncratic and verb specific, and often accessible at the earliest possible stage in sentence processing. Finally, while the majority of approaches have focused on grammatical description, only a few efforts have been carried on to operationalize CxG as computational modeling, with the exception of Fluid CxG (Steels, Reference Steels and Steels2011) and Embodied CxG (Bergen & Chang, Reference Bergen, Chang, Hoffmann and Trousdale2013.

To conclude, the core of this Element is investigating the relationship between constructions and compositionality, as the title of this work suggests. Indeed, constructionist approaches do not refute compositionally per se, but reformulate this paradigm in terms of the combination of constructions (“weak compositionality”; cf. Michel, Reference Michel2023, p. 566):

Consequently, the CxG framework turns out to be the best way to unify the different compositional and noncompositional mechanisms observed in language due to its key assumptions. Nevertheless, there is no consensus regarding the manner in which constructions interact with each other to license a specific utterance (Boas, Reference Boas, John and Xu2021, p. 64). Overall, this Element is focused on reviewing the different insights about language processing, which aspects are already depicted in CxG, and which ones still need to be addressed by future research.

1.3 Compositionality, Productivity, Creativity

As introduced in the first paragraph of this Element, the surprising fact about language is that people can constantly generate (and understand) never-ever-produced utterances. Chomsky defined this property as the “creative aspect” of language: “[A]n essential property of language is that it provides the means for expressing indefinitely many thoughts and for reaching appropriately in an indefinite range of new situations” (Chomsky, Reference Chomsky1965, p. 6). More recently, Adger affirmed:

Chomsky and the generative tradition thus seem to suggest that linguistic creativity is “combinatorial” and “productive” (Bergs, Reference Bergs2018, p. 278): It involves creating something entirely new using existing rules in almost infinite ways. The success of the principle of compositionality thus relies on its ability to explain the most attractive property of language: creativity. However, before introducing compositionality in the next section, it is essential to step back and understand how linguistic creativity is defined (especially in the realm of CxG). Let us start with some creative expressions:

1. 1. a. She smiled him in the door (Goldberg, Reference Goldberg2019, p. 61).

   2. b. The mother of all battles (Hartmann & Ungerer, Reference Hartmann and Ungerer2023, p. 5).

   3. c. Messi is the Mozart of football (Hoffmann, Reference Hoffmann2019, p. 5).

   4. d. Weapons of mass distraction (Giora et al., Reference Giora, Fein and Kronrod2004).

The following expressions are likely to be unfamiliar to most readers and, therefore, by the previous definition can be considered creative utterances. However, according to Sampson (Reference Sampson and Hinton2016), there are two distinct types of creativity: F-creativity (fixed creativity), which produces examples drawn from a predetermined and established inventory, and E-creativity (extending/enlarging creativity), which goes beyond the system rules. According to this dichotomy, many linguistic phenomena traditionally assumed as “creative” are, in fact, examples of F-creativity, as new sentences are the result of grammatical rules (Hoffmann, Reference Hoffmann2018).

The term productivity is used in linguistics to refer to the “original use of established possibilities of the language” (Leech, Reference Leech2014, p. 24). For instance, syntactic productivity concerns “the range of lexical items that may fill the slots of constructions” (Perek, Reference Perek2016, p. 66). In accordance with CxG’s assumptions, one uses and extends preexisting constructions to generate novel utterances. This is exemplified in (4a), where the use of a typical intransitive verb (e.g., smiled) as transitive in a caused-motion construction forces a creative new meaning, such as “She caused him to move the door by smiling.” Mismatches between the typical environments in which a verb is used and its occurrence in a new and creative way are widely discussed as valency coercion (Goldberg, Reference Goldberg1995). Several studies in CxG have investigated this construction productivity, and in particular Goldberg (Reference Goldberg2019) offers an extensive review focusing on explaining “the partial productivity of grammatical constructions.”

Even though many new expressions arise from productivity (of F-creativity), the question of “how do speakers use their grammar to create E-creative utterances” remains a topic of debate (Hoffmann, Reference Hoffmann2022a, p. 280). According to Bergs (Reference Bergs2018), a source of E-creativity relies on the “intentional manipulation of linguistic structure” (p. 281), usually exemplified by linguistic extravagance, that is, to talk in such a way that you are noticed (Haspelmath, Reference Haspelmath1999; Ungerer & Hartmann, Reference Ungerer and Hartmann2020). The use of formulaic patterns drawn from a fixed template, like in (4b) (namely, snowclones; cf. Hartmann & Ungerer, Reference Hartmann and Ungerer2023) can be considered creative. They represent an interesting case because, even if they transmit a hyperbolic meaning fulfilling a specific pragmatic function, they still derive from a partially fixed construction. As such, these expressions illustrate the complex interplay between creativity and productivity (Ungerer & Hartmann, Reference Ungerer and Hartmann2023).

Other examples of proper creative constructs are metaphorical expressions, like the one in (4c), which are governed by the general cognitive process of Conceptual Blending (Fauconnier & Turner, Reference Fauconnier and Turner2002; Turner, Reference Turner2018). This mental operation constructs a partial match between two input mental spaces (FOOTBALL and CLASSIC MUSIC, in this example) and selectively projects from those inputs into a novel “blended” mental space, resulting in a new meaning (Messi is a genius on the football pitch, just as Mozart was a musical genius; cf. Hoffman, Reference Hoffmann2019). However, even apparently, rule-breaking phenomena like the production of a novel metaphor rely on established patterns (i.e., entrenched construction X-is-the-Y-of-Z; cf. Fauconnier & Turner, Reference Fauconnier and Turner2002) and on established mechanisms. As Bergs and Kompa (Reference Bergs and Kompa2020, p. 14) observes: “Still, even the most creative metaphor has to use established means (analogy) and comply with most of the rules governing language use and linguistic interaction. Thus, metaphors are actually also examples of F-creativity in the widest sense; they do not expand the rules of language as such.”

Other research domains propose alternative models for linguistic creativity. One such theory is the Optimal Innovation hypothesis, which posits that the aesthetics of creative productions are best explained by variations of familiar material (Giora et al., Reference Giora, Fein and Kronrod2004). According to this theory, specific minimal modifications of familiar expressions can be more pleasurable than entirely novel creations. For instance, the neologism in (4d) is optimally innovative because it induces a novel response while enabling the retrieval of a salient stimulus, the familiar expression “weapons of mass destruction.” The question is: Are utterances of this type examples of F-creativity (as they relate to the familiar and use a specific mechanism) or of pure E-creativity?

Despite the considerable research on linguistic creativity, the examples above reveal that a consensus on what constitutes a creative expression has yet to be reached. As Maybin (Reference Maybin and Jones2015, p. 34) stated: “While everyday language creativity is now an established area of ongoing linguistic research, there is a continuing lack of clear agreement about the precise definition and scope of creativity itself.” Generally, the complex relation between productivity and creativity is far from being defined: Given that language is a complex system, it is challenging to define expressions entirely unconstrained by any rules (“All use of natural human language ultimately is F-creative”; cf. Bergs & Kompa, Reference Bergs and Kompa2020, p. 18). In a broader sense, creativity can be viewed as a gradient phenomenon ranging from systematic productivity to extravagant stimuli that generalize from existing schemata. This Element focuses more on the F-creativity aspect of language, a “constrained” form of creativity (Goldberg, Reference Goldberg2019), focusing on how the generation and comprehension of new expressions relate to the familiar and the mechanisms we can exploit to generate novel (but not necessarily creative) utterances apart from compositionality.

1.4 Roadmap

What is compositionality’s role in today’s models of language and sentence processing? How do we process both familiar and novel expressions? How can observations from experimental data be transposed into a formal theory of language representation and processing? This Element connects various linguistic theories and behavioral observations about processes governing semantic interpretation, arguing that CxG provides a more suitable linguistic formalism to explain language comprehension.

This Element is organized as follows. First, Section 2 introduces one of the two protagonists of the title: compositionality. Specifically, it discusses the notion of Fregean compositionality, traditionally believed to be the sole explanation for our ability to understand and create new sentences, and illustrates how this principle was used for describing the mechanism of meaning composition in traditional formalisms, constructionist approaches, and distributional models of meaning. Complementary, Section 3 examines studies in psycholinguistics and neurolinguistics that challenge this traditional view. The behavioral outcomes suggest a model of linguistic representation consistent with usage-based constructionist approaches, blurring the distinction between stored and nonstored sequences and productive and nonproductive patterns. Furthermore, Section 4 introduces the main claim of this Element: Systematic processes of language productivity are mainly explainable by analogical inferences rather than sequential compositional operations. Novel expressions are produced and understood “on the fly” by analogy with familiar ones. The section delves into the characteristics of analogical reasoning and explores the nature of linguistic analogy to support the proposal that analogical processing forms the basis of the human capability to generate new utterances. Finally, Section 5 proposes a redefinition of the role of compositionality as a property of natural language and as the only mechanism in sentence comprehension, suggesting that compositionality is only one of the possible explanations for the human ability to comprehend and produce an endless number of novel utterances.

In the end, readers will realize the complexity of rethinking a linguistic theory that formalizes the coexistence of different mechanisms to interpret any expression, from the most common to the never-encountered-before ones.

2.1 The Principle of Compositionality: Definitions and Main Assumptions

The traditional presumption in philosophy and linguistics is that language and thought are compositional (Martin & Baggio, Reference Martin and Baggio2020): The meaning of a complex expression is entirely determined by its structure and the meanings of its constituents – once we specify what the parts mean and how they are put together, there is no more leeway regarding the meaning of the whole. This view is referred to as the Principle of Compositionality, also called Frege’s Principle by the name of Gottlob Frege, credited with having first formulated this notion – although there are problems with this attribution (Pelletier, Reference Pelletier1994, p. 24). The principle of compositionality was first introduced as a constraint on the relation between the syntax and the semantics of languages, and it was later postulated as an adequacy condition for other representational systems such as structures of mental concepts (Hinzen, Werning, & Machery, Reference Hinzen, Werning, Machery, Hinzen, Werning and Machery2012a).

Broadly, the principle of compositionality is typically defined as follows:

These definitions are just two of the most cited, although several variants have been formulated (Hinzen, Werning, & Machery, Reference Hinzen, Werning and Machery2012b). The common aspect of both versions is that the notions of content and structure are admittedly vague: The nature of the principle can be made precise only with an explicit theory of meaning and syntax, together with a full specification of what is required by the relation “is a function of” (Pelletier, Reference Pelletier and Aronoff2016). In this sense, compositionality is highly theory-dependent (Partee, Reference Partee2004, p. 154).

Linguistic theories adopting the compositional hypothesis diverge on several points depending on the theoretical assumptions regarding the representation of words’ meanings (i.e., the building blocks of the sentence), the syntactic rules governing sentence structure, and the processes implicated in meaning construction. Among others, they diverge on whether syntactic analysis recedes and supplies its output prior to semantic analysis, or whether syntactic and semantic analyses are combined (as exemplified in the Head-driven Phrase Structure Grammar, or HPSG; Sag & Pollard, Reference Sag and Pollard1994), with syntactic rules integrating the semantic information obtained from the elements they combine. Additionally, these theories differ in the form of the semantic output: a logical formula in first-order logic, a lambda expression, or a typed-feature representation, inter alia.

Although several variants of the principle have been formulated, the core idea is that the principle of compositionality advocates for a bottom-up, or building block model of meaning: The meaning of the whole expression is incrementally built from the meanings of its constituent parts (Goldberg, Reference Goldberg and Riemer2015). The principle also entails a modular vision of the interpretation process; that is, there is a clear separation between syntax and semantics. This so-called division of labor works in most formal approaches in the following way: Lexical semantics represents the meanings of words, and syntax governs the combination of words into larger units of meaning and ascribes the relationships between words within these larger units.

Furthermore, the principle presupposes the concepts of localism and incrementality. In the first place, localism pertains to whether compositional operations are local or global in nature. As outlined by Pagin and Westerståhl (Reference Pagin and Westerståhl2010a), the meaning of a complex term can be derived from (1) the meaning of its immediate “children” within the syntactic structure (considered in a tree-like fashion) regardless of the process by which their meaning was built up (strong compositionality), or (2) from its total (global) structure and the meanings of its constituent atomic parts (weak compositionality). In the latter interpretation, complex terms may exhibit different meanings depending on the larger expression of which they are part. Hupkes et al. (Reference Hupkes, Dankers, Mul and Bruni2020) exemplified the problem in arithmetic terms: The outcome of 14 - (2 + 3) does not change when the subsequence (2 + 3) is replaced by 5, a sequence with the same (local) meaning, but a different structure (strong version). However, the strong hypothesis is controversial in natural language, especially in the case of disambiguation of a phrase or word in context. Conversely, incrementality assumes that the interpretation process follows rigidly the same order in which the constituents are combined to form complex expressions, step by step, in a deterministic fashion. At each step, the interpretative operation builds the semantic value of the current node and makes it available for further steps. Importantly, once a semantic value has been ascribed to a particular utterance, it cannot be changed. These two properties introduce a perspective on meaning composition that presents certain challenges. First, localism yields that the assignment of a semantic value to a node must not rely on external factors beyond the current segment of the sentence under analysis. Second, incrementality asserts that once a semantic representation has been assigned, the meaning remains unchanged, regardless of any subsequent constituents, whether they be phrases, sentences, or discourse (Gayral, Kayser, & Lévy, Reference Gayral, Kayser, Lévy, Machery, Werning and Schurz2005). Nonetheless, as will be argued in the subsequent section, the context of use assumes a pivotal role in human interpretation, challenging a strong and incremental view of compositionality.

The principle of compositionality has been compelling for many reasons, even if it has been widely criticized (in fact, approximately 318 arguments against it can be found in the literature, cf. Pelletier, Reference Pelletier1994). Without delving into specific details, the following section sketches the traditional arguments favoring compositionality together with their main criticisms (cf. Pagin & Westerståhl, Reference Pagin and Westerståhl2010b for a comprehensive review of arguments both in favor of and in opposition to compositionality).

2.2 Modeling Compositionality

2.2.1 Compositionality in Formal Semantics

The principle of compositionality stands as a foundational claim in Formal Semantics, a well-established approach in linguistic theory (Groenendijk & Stokhof, Reference Groenendijk, Stokhof, Carlson and Pelletier2005; Partee, Reference Partee, Aloni and Dekker2016; Partee, ter Meulen, & Wall, Reference Partee, ter Meulen and Wall1990). Formal Semantics encompasses a range of semantic theories, all employing standard methodologies grounded in symbolic logic, mathematics, and mathematical logic to rigorously formulate well-defined theories concerning the semantics of natural languages (King, Reference King and Smith2006).

The philosopher and logician Richard Montague (e.g., Reference Montague1970b; Reference Montague, Hintikka, Moravcsik and Suppes1973) was one of the first to argue that the relation between syntax and semantics in natural language could be regarded as not essentially different from the relation between syntax and semantics in a formal language, such as the language of first-order logic. He articulated this idea in the following words:

There is in my opinion no important theoretical difference between natural languages and the artificial languages of logicians; indeed, I consider it possible to comprehend the syntax and semantics of both kinds of languages within a single natural and mathematically precise theory. On this point I differ from a number of philosophers, but agree, I believe, with Chomsky and his associates.

(Montague, Reference Montague1970b)

Accordingly, natural language could be “translated” into the metalanguage of logic, as, for instance, the language of predicate calculus. Within the Montague Grammar tradition, the principle of compositionality assumes a pivotal role in articulating the relation of semantics to syntax. It states that the semantic interpretation for a language is defined as some homomorphism (a structure-preserving mapping) from syntax to semantics (a gentle introduction to this concept is provided by Janssen and Partee Reference Janssen, Partee, van Benthem and ter Meulen1997, p. 448–450). In other words, the syntactic operations that combine syntactic expressions must match the meaning operations, forming complex meanings from simpler ones. Goldberg (Reference Goldberg1995, p. 13) illustrates this claim as follows:

$\sigma \left(x{+}_{syn-comp}y\right)=\sigma \left(x\right){+}_{sem-comp}\sigma \left(y\right),$(1)

where $\sigma$ is a function from syntax to semantics, + ${}_{syn-comp}$ is a rule of syntactic composition, and + ${}_{sem-comp}$ is a rule of semantic composition. The formula formally conveys that the interpretation of the entire expression results from applying the meanings of the immediate constituents (and only by those meanings) via a semantic operation that aligns directly with the corresponding syntactic operation. It is worth noticing that having a compositional interpretation structured in this manner represents a straightforward way of ensuring that each of the infinitely potential syntactic structures within a language will receive a clearly defined interpretation (Groenendijk & Stokhof, Reference Groenendijk, Stokhof, Carlson and Pelletier2005).

The perspective and the technical apparatus Montague offered have significantly impacted the study of natural language semantics, paving the way for a wide range of Formal Semantic approaches, from model-theoretic to proof-theoretic semantics. Besides their specificities, any compositional formal semantic framework provides

1. 1. a knowledge or a semantic representation of linguistic expressions in a logic,

2. 2. some mechanisms for combining them in the form of formal rules.

Classically, semantic information is depicted in terms of feature-value structures, and logic is used both as a description language and calculus for constructing the meaning. In these approaches, the meaning is assembled starting from atomic objects (typically the meaning of words) and incrementally combined into larger structures. This mechanism constitutes the basis of compositionality. It is noticeable that this formalization requires an explicit representation of both types of information: the information associated with the constituents (typically lexical semantics) and the meaning composition mechanisms. The standard approach is to use first-order logic and model meaning composition as function application (Montague, Reference Montague and Visentini1970a; Partee, ter Meulen, & Wall, Reference Partee, ter Meulen and Wall1990). The semantic representation of the phrase Alex smiled can be expressed as in Equation 2. In this representation, the proper noun “Alex” is denoted by a constant (a), while the predicate “smiled” is expressed as a lambda term, signifying a function that can be applied to arguments of the appropriate type. The application of “smiled” function to the argument “Alex” (as illustrated in the third row) results in the final expression, where the bound variable (x), found within the lambda term, is replaced with the argument expression a (the example is adapted from Martin & Baggio, Reference Martin and Baggio2020).

$\begin{array}{cc}& \text{Alex:}a\\ & \text{smiled:}\lambda x.smile\left(x\right)\\ & \text{Alex smiled:}[\lambda x.smile(x\left)\right]\left(a\right)\\ & ⟶smile\left(a\right)\end{array}$(2)

In lambda calculus, the process of function application allows the identification of the arguments and the predicates to be gathered into formulae, thanks to a mapping function from syntax to semantics. The integration of quantifiers and modalities completes the logical model, employing specific mechanisms based on more intricate calculi. This mechanism, which remains relatively consistent across various theories, relies on two foundational premises: first, that meaning can be dissected into fundamental, atomic components; and second, that a linear and incremental mechanism exists for assembling these components into abstract structures.

This approach constitutes the basis of numerous semantic formal frameworks, particularly those focused on the interface between syntax, semantics, and discourse. Noteworthy examples include Discourse Representation Theory (Kamp & Reyle, Reference Kamp and Reyle1993) and Combinatory Categorial Grammar (Steedman, Reference Steedman2001). From a computational perspective as well, the Montagovian perspective of compositionality has long been a cornerstone in natural language understanding approaches. Extensive work has been carried out in this direction within the logic programming paradigm (Colmerauer, Reference Colmerauer, Clark and Tärnlund1982; Shieber & Pereira Reference Shieber and Pereira1987), and more recently, within the theoretical framework of Categorial Grammars (Bos et al., Reference Bos, Clark, Steedman, Curran and Hockenmaier2004; Moot, Reference Moot2012). Additionally, a more recent framework exploring semantic representation with minimal structures has been proposed in the Head-Driven Phrase Structure Grammar paradigm, known as Minimal Recursion Semantics (Copestake et al., Reference Copestake, Flickinger, Pollard and Sag2005). This framework introduces key notions, such as under-specification and a generalization of the interface between semantics and other domains.

However, the traditional formal analysis of meaning composition is acknowledged to exhibit certain limitations in terms of its power and expressive capacity. On the representation side, it is unclear how lambda terms precisely capture the intricate nuances of the meanings of constituent expressions. Formal approaches struggle to encapsulate content words in all their richness – and, by extension, the array of inferences drawn from lexical information (Boleda & Herbelot, Reference Boleda and Herbelot2016). For instance, while man and dude would have the same ontological representation (they both refer to male humans), they are not equivalent, as they have different connotations (Boleda & Herbelot, Reference Boleda and Herbelot2016). Different formalisms have been explored for modeling lexical meaning, employing richer data structures than lambda terms, with significant implications for theories concerning the process of semantic composition. On the composition side, a critical question emerges: whether meaning arises solely from the process of function application or from the interpretation of formulas within predicate logic. This dilemma becomes particularly pronounced when composition and interpretation do not mirror each other, giving rise to scenarios where a strong version of compositionality falls short of delivering comprehensive explanations (Martin & Baggio, Reference Martin and Baggio2020). Furthermore, it is essential to acknowledge that Formal Semantics does not encode the full spectrum of human linguistic experiences. Notably, analogical reasoning, a fundamental facet of human predictive cognition, lies beyond the scope of Formal Semantics (Boleda & Herbelot, Reference Boleda and Herbelot2016). Additionally, this formalism encounters difficulties in adequately describing numerous linguistic phenomena, including but not limited to co-compositionality (Pustejovsky, Reference Pustejovsky, Hinzen, Machery and Werning2012) and coercion.

2.2.3 Compositionality in Constructionist Approaches

It is frequently contended that constructional approaches either lack compositionality or explicitly deny semantic composition (Kay & Michaelis, Reference Kay and Michaelis2012). However, compositional operations and a construction-based formalism to syntax are not inherently contradictory, and some constructionist approaches have been developed to formally integrate them into a unique representation. Sign-Based Construction Grammar (SBCG) (Michaelis, Reference Michaelis, Hoffmann and Trousdale2013; Sag, Reference Sag, Boas and Sag2012; Sag, Boas, & Kay, Reference Sag, Boas, Kay, Boas and Sag2012) has been the one more extensively focused on formally explaining syntactic and semantic composition through construction representation (Michaelis, Reference Michaelis, Heine and Narrog2015). Sign-Based Construction Grammar proposes a highly structured and taxonomically organized lexicon, based on two fundamental units, namely signs and constructions. Signs are feature structures that specify both syntactic and semantic properties and are formally represented as attribute-value matrices (AVMs; cf. Figure 1). This representation regards each expression of a language as a sign, as words, lexemes, and even phrases (Michaelis, Reference Michaelis, Heine and Narrog2015). Conversely, constructions are described as the means to derive more complex sign descriptions from simpler ones. Specifically, they are type constraints that specify (i) the properties that define a class of constructs (i.e., feature structures equivalent to local trees with signs at the nodes) and (ii) the way to construct a mother sign from one or more daughter signs (Michaelis, Reference Michaelis, Hoffmann and Trousdale2013). Constructions are descriptions of either classes of constructs (combinatoric constructions) or of lexemes (lexical class constructions). An example of the subject–predicate construction is provided in Figure 2.

Figure 1 A sign in SBCG (from Michaelis, Reference Michaelis, Heine and Narrog2015, p. 152). The lexeme drink is represented by syntactic (SYN) and semantic (SEM) constraints.

Figure 2 A construction in SBCG (from Michaelis, Reference Michaelis, Heine and Narrog2015, p. 153). The subject–predicate cxn describes the mother sign of a basic clause. It contains a mother (MTR) feature with an empty valence list, a daughters (DTRS) feature with two items on its valence list, and a head daughter (H) that is a finite verb and has one item on its valence list (X, i.e., the subject of the clause).

Sign-Based Construction Grammar offers a robust formalism to construction-based syntax by relying on the mechanism of unification. This operation involves matching and merging the corresponding features from each linguistic structure (signs), ensuring that the resulting representation captures the combined form and meaning of the components. These constraints ensure that the unified feature structure is well-formed and conforms to the grammatical and semantic constraints of the language. Constraints may include syntactic rules, semantic roles, selectional restrictions, and other linguistic principles (Sag, Reference Sag, Boas and Sag2012). For example, the feature structure of the construct subject–predicate is unified with those of the sign of the lexemes for the verb and the subject to create a representation of the entire fragment. Such a combination is possible because there is no conflicting attribute-value information between the two constructions (i.e., the AVMs are “unifiable”; cf. Chaves, Reference Chaves, Kertész, Moravcsik and Rákosi2019).

Therefore, while traditional syntactic approaches affirm that the interpretation of an expression is licensed by (i) a rule of syntactic composition and (ii) a rule of semantic composition (Equation (1)), SBCG proposes a unique linguistic object (a construction) that serves a similar function (Michaelis, Reference Michaelis, Sinha and Wenin press). Overall, SBCG offers a formalism for construction-based syntax that is declarative and constraint-based (Michaelis, Reference Michaelis, Hoffmann and Trousdale2013).

This approach has the advantage of incorporating the CxG organization into a formalized framework. Specifically, it provides a means to explicitly specify the relationships between the various components of a construction. However, criticism has been leveled against SBCG, suggesting a tendency to prioritize formal-syntactic aspects over semantic considerations. For instance, Sag (Reference Sag2010) proposes a comprehensive construction for filler-gap phenomena that completely lacks semantic content (defective constructions). This contrasts with a shared constructionist view (Goldberg, Reference Goldberg2006; Hilpert, Reference Hilpert2019) for which each construction, even the more abstract one, has associated a specific meaning (Ungerer & Hartmann, Reference Ungerer and Hartmann2023).

Moreover, SBCG is a process-neutral approach that makes no predictions about the actual online parsing or production of constructions (Hoffmann, Reference Hoffmann and Dancygier2017). By contrast, there are two constructionist frameworks based on the unification of AVMs developed for the computational implementation of sentence processing, namely, Fluid Construction Grammar (FCG; Steels, Reference Steels, Hoffmann and Trousdale2013, Reference Steels2017) and Embodied Construction Grammar (ECG; Bergen & Chang, Reference Bergen and Chang2005, Reference Bergen, Chang, Hoffmann and Trousdale2013). Besides their formalisms, both are specifically developed for computational implementation (Hoffmann & Trousdale, Reference Hoffmann and Trousdale2013; Ungerer & Hartmann, Reference Ungerer and Hartmann2023. In particular, FCG utilizes truth-conditional first-order predicate calculus, whereas ECG relies on mental simulation models and embodied schemas. Additionally, while FCG formalism accepts defective constructions, ECG constructions are always form-meaning pairings, though it does not deny the existence of purely form or meaning schemas (Hoffmann, Reference Hoffmann and Dancygier2017).

2.3 Distributional Approaches to Compositionality

Formal approaches assume that the meaning of words (constants that replace symbols into logical formulas) is one and just one, determined a priori, that is to say, “a lexical item must make approximately the same semantic contribution to each expression in which it occurs” (Fodor & Pylyshyn, Reference Fodor and Pylyshyn1988, p. 42). However, this assumption contrasts with the evidence that lexical meanings are context-sensitive, that is, they can “adapt” their meaning to fit a specific context and communicative situation, and generally, their use in contexts defines their semantic representation. That is to say, the distribution of the words constitutes one of the essential sources of information for accessing their meaning.

In this respect, Distributional Semantics (Boleda, Reference Boleda2020; Lenci, Reference Lenci2018; Lenci & Sahlgren, Reference Lenci and Sahlgren2023) have provided a solid alternative framework for denoting word meaning in the past decades. Posing a radically different stance, Distributional Semantics aims at representing the word meaning as the contexts in which it occurs, rising from the so-called Distributional Hypothesis of lexical meaning (Firth, Reference Firth and Firth1957; Harris, Reference Harris1954; Sahlgren, Reference Sahlgren2008). Concretely, a Distributional Semantic Model represents the lexicon in terms of a vector space, where a lexical target is described as a numeric vector (also known as embedding) built by identifying its syntactic and lexical contexts in a corpus (Lenci, Reference Lenci2018). This computational implementation makes it possible to quantify the similarity between words using algebraic formulas while allowing room for semantic changes (Perek, Reference Perek2016) and meaning shifts (Busso, Pannitto, & Lenci, Reference Busso, Pannitto and Lenci2018). Compared to formal representation, this approach has some advantages: (1) It provides a continuous representation that can easily tackle language’s gradience and fuzziness; (2) it does not assume a priori semantic primitives, that is, it is not stipulative; and (3) representations are also explainable in terms of how we can cognitively build these (it is plausible with respect to learnability, cf. Miller & Charles, Reference Miller and Charles1991).

Initial distributional approaches have been designed to represent word meaning by assigning each word to a single vector, produced as an abstraction over all its contexts of use. The logical next step involved understanding how to combine these representations to obtain vectors for phrases, sentences, and even larger pieces of text. Research in the last decade led, first and foremost, to methods integrating DSMs with formal symbolic theories of language: Semantic composition depends on an algebraic function that combines words, which are no longer described as symbolic representations but as distributional ones.

The approaches partaking this stance fall under the name of Compositional Distributional Semantics Models (CDSMs) ((Baroni, Bernardi, & Zamparelli, Reference Baroni, Bernardi and Zamparelli2014; J. Mitchell et al., Reference Mitchell, Lapata, Demberg and Keller2010) and aim to explicitly apply the principle of compositionality to compute distributional vectors for phrases. Compositional Distributional Semantics Models produce representations of phrases by composing distributional vectors of words comprised in these phrases. As in classic Distributional Semantics for words, these models generate similar vectors for semantically similar sentences, regardless of length or structure. For example, require attention and need treatment should have a similar distributional signature, and they should be dissimilar to, that is, attend a conference.

Various strategies to compose word embeddings have been suggested (cf. Table 1). In the most influential papers on the topic, J. Mitchell and Lapata (Reference Mitchell and Lapata2008, Reference Mitchell and Lapata2010) introduced several arithmetic operations for vector composition, operationalized as additive and multiplicative functions. For instance, the expression fluffy cat can be represented as

$\overrightarrow{\text{fluffy}}+\overrightarrow{\text{cat}}=\overrightarrow{\text{fluffy cat}},$

in which the meaning of the phrase is a new embedding derived from the addition of word vectors. Compositional Distributional Semantics Models are usually evaluated on a phrase similarity task: For pairs of phrases, the similarity between their respective combined vectors is computed and these scores are compared with similarity ratings elicited from English speakers.

Table 1 Vector composition functions

Model	Function
Weighted additive	$\alpha \text{fluffy}+\beta \text{cat}$
Multiplicative	$\text{fluffy}\odot \text{cat}$
Full additive	$X\overrightarrow{\text{fluffy}}+Y\text{cat}$
Lexical Function	$A_{\text{fluffy}}\overrightarrow{\text{cat}}$
Fullex	$tanh\left(\right[W_1,W_2]\left(\begin{array}{ccc}{A}_{\text{fluffy}}& \text{cat}& \\ A_{\text{cat}}& \text{fluffy}\end{array}\right)$

Note: $\odot$ stands for pointwise multiplication. $\alpha$ and $\beta$ are scalar parameters, matrices X and Y represent syntactic relation slots (e.g., Adjective and Noun), matrix A represents a functional word (e.g., the adjective in an adjective–noun construction).

Source: References to the models, in order: J. Mitchell and Lapata (Reference Mitchell and Lapata2008); J. Mitchell and Lapata (Reference Mitchell and Lapata2008); Zanzotto et al. (Reference Zanzotto, Korkontzelos, Fallucchi and Manandhar2010); Baroni and Zamparelli (Reference Baroni and Zamparelli2010); Socher et al. (Reference Socher, Perelygin and Wu2013)

More complex models characterize composition by representing lexemes and phrases with matrices and tensors rather than with vectors alone (Socher et al., Reference Socher, Perelygin and Wu2013; Zanzotto et al., Reference Zanzotto, Korkontzelos, Fallucchi and Manandhar2010). For instance, the Lexical Function model denotes predicates (verbs and adjectives specifically) as functions mapping one noun meaning to another, coherently with the Montagovian view. Concretely, predicates are matrices, their nominal arguments are represented as vectors, and their multiplication results in the phrase vector (Baroni & Zamparelli, Reference Baroni and Zamparelli2010). The Lexical Function model was one of the first attempts to represent formal semantic operations with DSMs and turned out to be very influential in the research area. However, the main limitation of this approach is the difficulty in scaling up to multi-argument sentences. Estimating the matrices and tensors for complex functional types such as transitive verbs can be very complex and may encounter data-sparseness problems. Paperno, Pham, and Baroni (Reference Paperno, Pham and Baroni2014) proposed a practical approximation of the Lexical Function model to address these limits, but it is hardly competitive with the much simpler additive models (Rimell et al., Reference Rimell, Maillard, Polajnar and Clark2016).

Some authors also exploited neural networks to learn composition functions explicitly. An example is the recursive neural network (RNN) of Socher, Manning, and Ng (Reference Socher, Manning and Ng2010), in which representations for larger chunks are computed recursively obeying a predefined syntactic parse tree of the sentence. Specifically, the neural network induces a score for each pair of neighboring words, which measures how likely these two words are to be combined into a phrase, and simultaneously, it collapses the two words into an $n$-dimensional representation of the resulting phrase. This new phrase embedding replaces the words in the sequence and possibly becomes a child of another phrase spanning more words. This bottom-up process continues until the whole input sentence is mapped to the embedding space. An alternative approach proposed by Socher (Reference Socher, Huval, Manning and Ng2012) was the matrix-vector RNN, which consists in representing each word by a vector and a matrix encoding its interaction with the syntactic sisters. Compositional representations for phrases and sentences are learned by a RNN in a supervised setting.

Finally, an alternative approach to compositional DSMs assumes that the representation of a sentence is not a vector but rather a logical form containing distributional vectors of the content words (Asher et al., Reference Asher, Van de Cruys, Bride and Abrusán2016; Beltagy et al., Reference Beltagy, Roller, Cheng, Erk and Mooney2016; Coecke, Sadrzadeh, & Clark, Reference Coecke, Sadrzadeh and Clark2010; Garrette, Erk, & Mooney, Reference Garrette, Erk, Mooney, Bunt, Bos and Pulman2014).

Among all compositional functions proposed here, vector addition still shows remarkable performances on various tasks, such as phrase similarity or paraphrase detection (Asher et al., Reference Asher, Van de Cruys, Bride and Abrusán2016; Rimell et al., Reference Rimell, Maillard, Polajnar and Clark2016), outperforming more complex methods, such as the Lexical Function model. However, vector addition is theoretically and cognitively unsatisfactory: The meaning of a complex expression is not simply the sum of the meaning of its parts but it also depends on the syntactic content. Without being able to discriminate between the different syntactic realizations of semantic roles, sentences like

1. 1. a. The dog chases the cat.

   2. b. The cat chases the dog.

are modeled in the same way. Moreover, while vectors are suitable to capture the semantic relatedness among lexemes, this representation might not be adequate for more complex linguistic expressions because of the limited and fixed amount of information that can be encoded (Erk & Padó, Reference Erk and Padó2008).

In summary, how distributional representations can be projected from the lexical level to the sentence or discourse level poses an ongoing challenge. Currently, compositionality is still considered the real bottleneck for Distributional Semantics (Lenci, Reference Lenci2018). It is worth highlighting that all the studies discussed earlier, in one way or another, adhere to the conventional principle of Fregean compositionality: The representation of a complex unit is derived from the representation of its immediate constituents.

A final note concerns the last generation of language models (foundational models, cf. Bommasani et al., Reference Bommasani, Hudson and Adeli2021) built by relying on deep learning artificial neural networks and trained on massive amounts of text using a word-in-context prediction task. Numerous empirical studies have explored the compositional capabilities of neural models using various approaches (Gulordava et al., Reference Gulordava, Bojanowski, Grave, Linzen, Baroni, Walker, Ji and Stent2018; Lake & Baroni, Reference Lake, Baroni, Dy and Krause2018; Linzen, Dupoux, & Goldberg, Reference Linzen, Dupoux and Goldberg2016; Loula, Baroni, & Lake, Reference Loula, Baroni and Lake2018, among others). However, there is still an incomplete understanding of the strategies learned by these networks and their capacity to generalize. Hupkes et al. (Reference Hupkes, Dankers, Mul and Bruni2020) have identified five aspects of compositionality from theoretical literature (cf. Table 2) and translated them into five grounded tests for these models. This evaluation framework underscores the necessity for a more comprehensive and valid set of evaluation criteria and improved analytical tools for assessing the compositional abilities of neural networks.

2.4 Summary

This section has introduced the concept of compositionality from two distinct perspectives. First, compositionality has been outlined as a processing principle, where the fundamental assumptions and the supporting and opposing arguments have been presented. Additionally, the application of compositionality as a representation component within linguistic theory was discussed, examining its various formalizations in Formal Semantics, generative semantics, CxG, and computer science, along with their central assumptions and primary limitations.

Overall, the accounts supporting compositionality, primarily the generative approaches, propose a view of language that can be outlined in two primary components: (i) words, which are stored in a lexicon, and (ii) rules, which govern how words can be combined into meaningful, coherent sentences (the grammar component of language). The rules of grammar mostly obey the principle of compositionality: There is an inventory of rules that dictates the construction of syntactic representations, which are subsequently mapped to principles responsible for the composition of word meanings into more complex expressions. On the contrary, constructionist theories assume that there are no boundaries between lexicon and syntax, that is, between what is regular and what is irregular, or what is productive and what is unproductive. Constructions, as the basic units of language, are productive linguistic constructs that account for syntactic processes. In other words, CxG “handles ‘normal syntax’ in a way that necessitates a shift of perspective away from the common view of words, word classes, and phrase structure rules” (Hilpert, Reference Hilpert2019, p. 70). While formal CxGs (e.g., SBCG, FCG, ECG) employ rigorous unification-based formalism to elucidate the emergence of well-formed structures from feature matching among their constituent parts, these approaches formalize the aggregation of constructions but do not encode other mechanisms that occur concomitantly in sentence processing.

The following section will provide behavioral evidence demonstrating that several factors affect comprehension and are against a “strong” version of compositionality centered solely on syntax-semantics homomorphism.

3.1 Online Processing of Multiunits Sequences

3.2 The Predictive and Shallow Nature of Processing

3.3 Summary

All the experimental literature summarized in this section poses against a serial account of sentence processing where syntax proposes a structural interpretation for semantics to cash out subsequently.

First, the evidence for the psychological reality of multiword linguistic units points to a linguistic model in which there is no clear boundary between expressions stored in our memory and expressions generated by compositional mechanisms, as claimed by constructionist approaches: The large number of facilitation effects observed in language processing make us doubt a clear way to distinguish different processes for idiomatic, formulaic, and novel expressions. While this is problematic for a strong view of compositionality, the notion of constructions and the importance of frequency in modeling semantic memory implies a redefinition of what linguistic units are combined and how they are combined to provide the final representation of a sentence.

Secondly, language processing can be seen as a process predominantly driven by sequence matching and pattern identification and incorporates probabilistic cues, including, importantly, the frequency and predictability of the sentence and its component. Compositionality, devised as a mechanism to aggregate meaning, should be redefined as a complex mechanism able to integrate a network of activated linguistic and contextual information.

Altogether, a linguistic theory of language processing should provide a formalization that (i) integrates formal, semantic, and contextual knowledge, and (ii) implements the predictive nature of comprehension and quantitatively models the basic processing of the good-enough approach, that is, the “interpret whenever possible” principle. Among others, Blache (Reference Blache, Sharp, Sèdes and Lubaszewski2017) proposed that, instead of building a syntactic structure serving as support of the comprehension of a sentence, the processing mechanism is delayed until enough information becomes available (i.e., the density of information – or the cohesion – reaches a certain threshold). This general parsing mechanism offers the possibility to integrate different sources of information when they become available by delaying the evaluation, waiting until a certain threshold of cohesion can be identified.

4.1 Analogical Reasoning and Its Role in Cognition

Analogy is a domain-general cognitive process that enables a structure mapping between two situations or objects (Gentner, Reference Gentner1983). In the most typical case, a familiar concrete domain, referred to as the base or source, functions as a template by which one can understand and draw new inferences about a less familiar or abstract domain, namely the target (Gentner & Smith, Reference Gentner and Smith2013).

Analogical thinking is pervasive in human thought and speech. People draw on experiential analogies to form mental models of phenomena in the world every day. One example is the often-cited “Rutherford analogy”: A sentence like The atom is like the solar system is acceptable because some aspects of the structure of the atom (notably the fact that electrons orbit the nucleus) can be understood from prior knowledge of the structure of the solar system (i.e., planets orbit the sun).

Over the last three decades, Gentner and colleagues have conducted an extensive investigation into analogy, developing one of the most influential frameworks in cognitive research. The Structure-Mapping Theory of analogy (Gentner, Reference Gentner1983, Reference Gentner1988; Gentner & Markman, Reference Gentner and Markman1997) delineates the set of implicit constraints by which people interpret analogy and similarity. At its core, the theory posits that analogy is characterized by mapping relations between objects, rather than attributes of objects, from base to target. Accordingly, the theory assesses analogies on purely structural grounds, as defined by Falkenhainer, Forbus, and Gentner (Reference Falkenhainer, Forbus and Gentner1989, p. 3): “This structural view of analogy is based on the intuition that analogies are about relations, rather than simple features. No matter what kind of knowledge (causal models, plans, stories, etc.), it is the structural properties (i.e., the interrelationships between the facts) that determine the content of an analogy.”

Going back to the Rutherford analogy, the sentence The atom is like the solar system is interpretable as “The electron revolves around the nucleus, just as the planets revolve around the sun.” However, the atom and the sun do not share the same features; that is, the analogy does not imply that “The nucleus is yellow, massive, etc., like the sun”). If that were the case, we would have a literal similarity statement instead, in which a large number of both object attributes and relational predicates are mapped from base to target relative to the number (Gentner, Reference Gentner1983). In other words, the two situations are analogous because they share the complex relationship known as “to revolve around”; however, the target object does not have to resemble its corresponding base.

Analogical processes, hence, rely on a structure-mapping engine (SME) that identifies relations between representations rather than the mere similarity between their attributes. In this sense, mapping of one entity to another depends on the “syntactic properties of the knowledge representation [describing the entities], and not on the specific content of the domains” (Gentner, Reference Gentner1983, p. 1). To clarify, this process is not triggered simply by surface similarity but requires a great deal of relational or structural alignment knowledge:

Figure 3 provides a concrete illustration of these differences. On the one side, (perceptual) similarity occurs when an observer perceives the resemblance between two objects, such as the medium-sized triangle in (b) and the smallest triangle in (a), which are identical in size. In this case, the two objects match because they share identical perceptible attributes. In contrast, the analogy between the smallest triangle in (a) and the small one in (b) is based on a relational resemblance: While these objects differ in size, they both hold the distinction of being the smallest within their respective sets. However, it is crucial to bear in mind that the demarcation between analogy and similarity is not a strict dichotomy; rather, it exists along a continuum.

Figure 3 Perceptual similarity and relational analogy

Note: The smallest triangle in (a) could match either with the middle triangle in (b) – applying an object match, or with the rightmost triangle – following a relational match.

Source: Adapted from Gentner and Smith (Reference Gentner, Smith and Ramachandran2012, p. 131).

Since the late 1980s, cognitive works have converged to delineate the general characteristics of analogical thinking across domains. Most theorists agree that analogies can be decomposed into several basic component processes. Specifically, Gentner and Smith (Reference Gentner and Smith2013, p. 670) identify three key stages involved in analogical reasoning.

• Retrieval: In this stage, the individual retrieves information from their long-term memory (base) based on a current topic or situation they are dealing with in the working memory (target). The goal is to find a prior analogous situation or case from their memory that is similar in some way to the current situation. This retrieval of past experiences or knowledge is essential for drawing parallels and making analogies.

• Mapping: Once a relevant analogous situation has been retrieved, the mapping stage involves aligning and comparing the representations of the base and the target. This alignment process helps identify similarities, differences, and relationships between the elements or components of the two cases. It also allows for projecting inferences from one situation to the other. This mapping process is systematic and structure-consistent, meaning it considers the larger relational systems within the cases.

• Evaluation: After the analogical mapping is complete, the individual evaluates the analogy and its associated inferences. This evaluation can involve assessing the validity and relevance of the analogy to the current situation. It also includes judging the quality and reliability of the inferences drawn from the analogy. Effective evaluation is crucial for making informed decisions or solving problems based on the analogical reasoning process.

Holyoak (Reference Holyoak2012) offers an accurate summary of the strategies underlying analogical thinking (p. 10):

To summarize, analogical reasoning is not just comparing two analogs based on the similarities we perceive. Instead, it is a complex process of retrieving structured knowledge from long-term memory, representing and manipulating role-filler bindings in working memory, generating new inferences, and finding structured intersections between analogs to form new abstract schemata (Holyoak, Reference Holyoak2012). Besides, an aspect that is generally highlighted is that analogy is an active process that can shape our perception, and it is a central component to learning and transfer.

According to Gentner and Smith (Reference Gentner, Smith and Ramachandran2012), analogical processes can augment and extend knowledge in four ways. In inference projection, spontaneous candidate inferences are made from a well-structured representation to one that is not entirely complete. Schema abstraction, instead, is abstraction of shared relational structure across different exemplars. This structure may be stored in memory as an abstraction and used again for later exemplars. In difference detection, the structural alignment process highlights alignable discrepancies between analogs and makes them more salient. Finally, new representations could be made by relying on re-representation: Even when two potential analogs have nonidentical conceptual relations, they may still be analogous if altering one or both analog representations improves the relational match.

In that sense, analogical reasoning leads to learning in terms of categorization, abstraction, and category extension: “The cognizer needs the ability to compare two structures and notice discrepancies as well as similarity or overlap. When source and target (the new item) match in the relevant respects, the target is categorized as an item belonging to the source category” (Langacker, Reference Langacker1999, p. 4). Gentner, Holyoak, Hofstadter, and numerous other scholars have conducted extensive research on the role of analogy in the process of learning, substantiating their investigations with insights from psychology and, more recently, neurology.

In brief, analogy can be defined as “an inductive mechanism based on structured comparisons of mental representations” (Holyoak, Reference Holyoak2012, p. 1). More broadly, analogical thinking plays a crucial role in creative discovery, problem-solving, categorization, learning, and knowledge transfer. A fundamental aspect of research is its interdisciplinary nature, allowing multiple fields to contribute collectively to our comprehension of cognitive processes. Psychological experiments, naturalistic observations, linguistic analyses, and computer simulations offer diverse perspectives on analogy. For the present discussion, the following section provides an overview of how the analogical mechanism has been proposed as an explanation of language production in linguistic theory.

4.2 Analogy in Language Use

Considerable evidence from cognitive psychology underscores the pivotal role of analogical reasoning as a foundational element of human cognition. Consequently, it is not surprising that analogy has been recognized as a core component of linguistic competence from the earliest times (Blevins & Blevins, Reference Blevins, Blevins, Blevins and Blevins2009). In linguistics, the mechanism of analogy has been conceptualized as a principle governing regularities in language, mostly behind morphological regularization (Anttila, Reference Anttila1977; Hock, Reference Hock, Brain and Richard2003, among others). However, this process has gained renewed attention within usage-based and constructionist approaches, which regard analogy as a fundamental mechanism behind language development and productivity (cf. Behrens, Reference Behrens, Hundt, Mollin and Pfenninger2017 for a systematic review).

As already mentioned, analogical reasoning involves a structural mapping process, wherein parallel structures are aligned to draw inferences about the less familiar structures based on knowledge derived from a more familiar counterpart. Due to the intrinsic relational nature of language, it is possible to align and map identical or similar linguistic structures to which speakers have been exposed (Holyoak, Reference Holyoak2012). Consequently, introducing a novel element within a linguistic construction requires a great deal of relational or structural alignment knowledge and substantial similarity to existing elements, thereby departing from a strong rule-governed view of productivity (cf. Table 3). The commonalities of the two structures could also generate an abstraction (or generalization) of these patterns. The following pages explore previous approaches to analogy-based productivity.

According to the constructionist view, language is a large repository of constructions with several levels of abstraction, from specific expressions to general patterns (Dąrowska, Reference Dąbrowska2017). However, novel utterances also occur. The question is then to understand what role our ample stored knowledge has in language productivity (and creativity, if possible). The argument proposed is that, even when we create novel expressions, people exploit the similarity with already encountered, stored sequences: Some new expressions are “more similar” to existing prefabs, and some are more remote to them. The term analogy is then used to identify the mechanism that, given a new sequence, determines the pattern that best serves as a foundation on which a speaker might articulate new linguistic forms (Ambridge, Reference Ambridge2020a; Bybee, Reference Bybee2010; Diessel, Reference Diessel2019). According to this view, the organization and productivity of language are the result of analogies between form and/or meaning in a structured inventory of constructions (Ibbotson, Reference Ibbotson2013), or, in other words:

The notion of analogy as a mechanism driving productivity has been explored in various linguistic domains.

Several examples are found in morphology, where substantial evidence suggests that new formations consistently rely on similarity to existing exemplars (Bybee, Reference Bybee2010). Among others, a great deal of research has focused on an apparent simple morphological phenomenon: English past-tense marking. The general idea of these works is that speakers do internalize rules, but these rules are few and cover only regular processes; the remaining patterns are attributed to analogy (Pinker & Prince, Reference Pinker and Prince1988). A series of studies involving acceptability judgments and production, conducted with both adults (Albright & Hayes, Reference Albright and Hayes2003) and children (Ambridge, Reference Ambridge2010; Blything, Ambridge, & Lieven, Reference Blything, Ambridge and Lieven2018), revealed that both the acceptability and the likelihood of producing “regular” past-tense forms for a novel verb (e.g., wiss, bredged, chooled, daped) are determined by the phonological similarity of the verb to existing stored “regular” past-tense forms (e.g., wissed being similar to missed, hissed, and wished; cf. Ambridge, Reference Ambridge2020a, p. 520). The same pattern holds for “irregular” forms (e.g., flept being similar to slept, wept, and crept), as it was also observed by Bybee and Moder (Reference Bybee and Moder1983).

In word formation, analogy is described as the process of creating a new word, patterned after an existing one, such as the formation of software after hardware (“surface analogy”; cf. Mattiello, Reference Mattiello2017). Of the various word formation processes, compound words represent one of the most studied phenomena, as the productivity of compounding is considerably greater than other word formation processes. Recently, Mattiello (Reference Mattiello2016, Reference Mattiello2017) studied novel analogical compounds in English, where words are formed either through a specific model (e.g., beefcake after cheesecake) or a schema model (e.g., green-collar after white-collar, blue-collar, pink-collar, and other similar compounds).

There is also a large amount of literature in psycholinguistics that has investigated the production, representation, and processing of compounds by means of analogical inferences (cf. Krott, Reference Krott, James and Blevins2009 for an introduction). For instance, Coolen, Van Jaarsveld, and Schreuder (Reference Coolen, Van Jaarsveld and Schreuder1991) suggest that “the interpretability of isolated novel compounds may be determined by the availability of lexicalized compounds that can serve as a model for the interpretation” and the “[r]elations within these lexicalized compounds may be among the first ones that are considered in the interpretation process” (p. 350). Research by Christine Gagné and colleagues (see, for example, Gagné, Reference Gagné2001; Gagné & Shoben, Reference Gagné and Shoben1997) highlights that the frequency of how compound constituents are used in existing compounds with similar interpretations affects how a new compound is interpreted. For example, when presented with the novel compound honey soup (interpreted as “a soup made of honey”), people read it faster if it is preceded by a compound with the same relation, like honey muffin (“a muffin made of honey”) compared to when it is preceded by a compound with a different relation, such as honey insect (“a honey made by insect(s)”). Notably, this phenomenon occurs only when the prime and target compounds share a common modifier (Gagné, Reference Gagné2001).

Behavioral evidence points toward a model of compounding that is far from classic rule-based morphology: “New compound words rarely are formed de novo from two independent words. Rather, they are created through a process of partial analogy in which one element of an existing compound is exchanged” (G. Libben, Reference Libben2014, p. 15). Overall, the compounds seem to invite individuals to search their memories for experiences with exemplars of the referents being identified by a compound and to create a new, ad hoc, category for it on the spot based on the examples found in memory (Chandler, Reference Chandler2017).

At the construction level, some works in CxG have presented evidence about the importance of prior constructions in producing novel combinations; that is, the meaning of a new sequence can derive from an extension of the meaning of a learned construction (Diessel, Reference Diessel2019; Goldberg, Reference Goldberg2019; Hilpert, Reference Hilpert2019). For instance, Boas (Reference Boas2003, pp. 260–284) usees the term “analogical creativity” to argue that the creative use of verbs in novel syntactic contexts could be explained by “item-based analogy,” driven by local similarities between particular verbs. For instance, given the sentence “She sneezed the napkin off the table,” a speaker can associate the resultative meaning stemming from the conventionalized [NP V NP XP] syntactic frame of blow (the source) with sneeze (the target). This position is in line with Goldberg’s view on the productivity of construction, whose focus includes the impact of type frequency and the coherence of a constructional schema and others (Goldberg, Reference Goldberg2019). Let us consider another example. The meaning of the Ditransitive construction is closely connected with “transfer of possession” as in John gave Mary a goat. Metaphorical extensions of this pattern, such as John gave the goat a kiss or even Cry me a river, are understood by analogy to the core meaning of the construction from which they were extended, which in the case of the ditransitive is something like “X causes Y to receive Z” (Goldberg, Reference Goldberg2006). Goldberg (Reference Goldberg2019, pp. 63–64) directly refers to the theory proposed by Gentner and colleagues: “The formal surface regularities of constructions invite learners to seek other types of regularities across exemplars, through a process of structural alignment, which we recall involves relating two (or more) distinct relational structures.”

In addition, Goldberg stated that by aligning the abstract relational structure of, for instance, I love you and You want a cookie, the shared relational structure, “animate entity experiences attitude toward something,” becomes more salient, and also the individual differences stand out (e.g., a pronoun object versus a lexical noun phrase object; (Goldberg, Reference Goldberg2019, p. 64).

To summarize, the usage-based constructionist approach relies on the fact that learners attend to and retain aspects of both the form and interpretation of utterances. This assumption leads to clustering the instances of constructions in the hyper-dimensional space we use to represent language so that more general constructions can emerge. In other words, the process of aligning exemplars relies on both formal properties and the meaning of the exemplars.

A final consideration that must be accounted for regards the risk of approaching language productivity as only and exclusively determined by stored exemplars and analogy. Indeed, the notion that we store direct linguistic experience and use it to understand a novel expression is comparable to exemplar models of language. Exemplar-based approaches provide both a model of how language is represented and how learning and using language takes place.

Combining exemplar models with the mechanism of analogy as the driving process of productivity can lead, in its most extreme version, to a complete repudiation of any form of abstraction, considering concrete (i.e., experienced) exemplars as the only stored elements. According to Ambridge (Reference Ambridge2020a), forms of which one has never had direct experience are produced and understood through on-the-fly analogical processes with respect to multiple stored exemplars and weighed according to their degree of similarity to the new instance, without reference to any kind of abstraction (such as the concepts of [VERB] [NAME], [SUBJECT], etc.). Chandler (Reference Chandler2017) has supported a similar position (p. 81):

However, the ensuing discussion (see Ambridge, Reference Ambridge2020b) showed the profound difficulties with such a view; namely, abstraction is necessary for the psychologically realistic storage of linguistic experiences. The version proposed by Goldberg in the CxG framework is more realistic than a radical exemplar-based view of language. While we can affirm that we memorize more linguistic units of language, human brain architecture is shaped to generalize from single items of experience: “language processing requires that our brains recode and compress incoming information. Thus memory traces of experiences, no matter how vivid, are partially abstracted from our experience” (Goldberg, Reference Goldberg2019, p. 16). In that sense, Ambridge’s radically exemplar model is not coherent with our cognitive architecture.

4.3 Analogy in Computational Models of Language

Several approaches within the domain of artificial intelligence (AI) have been proposed to replicate abstraction and analogy-making in computational systems. These strategies range from earlier symbolic or hybrid approaches, like Gentner et al.’s structure-mapping approach and the “active symbol” approach of Hofstadter and colleagues (Hofstadter, Reference Hofstadter and Hofstadter1985; Hofstadter & Mitchell, Reference Hofstadter, Mitchell, Holyoak and Barnden1994), to recent techniques employing deep neural networks and probabilistic program induction (see M. Mitchell, Reference Mitchell2021 for a complete review). Among others, structure mapping theory has been translated into computational form through the Structure Mapping Engine (SME) (Falkenhainer, Forbus, & Gentner, Reference Falkenhainer, Forbus and Gentner1989; Forbus et al., Reference Forbus, Ferguson, Lovett and Gentner2017). Structure Mapping Engine’s input consists of descriptions of two entities or situations, a base and a target, each consisting of a set of logical propositions. This model primarily focuses on the mapping process of analogy-making; thus, the situations to be mapped have already been represented in a logical form. Structure Mapping Engine provides a domain-independent explanation of analogy-making, concentrating on mapping the structure or syntax of its input representations rather than delving into domain-specific semantics. The challenge arises from the fact that human mental representations of real-world situations (including linguistic knowledge) are typically not as rigidly segmented as required by this architecture.

In the linguistic domain, at least three exemplar-based computational models have been proposed to replicate analogy in the realm of phonological, morphological, and lexical usage: Nosofsky’s Generalized Context Model (Nosofsky, Reference Nosofsky1990), Daelemans’s Tilburg Memory Based Learning Model (Daelemans & Van Den Bosch, Reference Daelemans and Van Den Bosch2010), and Skousen’s Analogical Model (Skousen, Reference Skousen1989). These models share the assumption that we recognize and interpret the significance of present experiences by directly comparing them with memories of past experiences. This approach involves specific memories rather than schematized representations abstracted from those collective experiences. Consequently, each of the three exemplar-based models suggests a continuous accumulation of rich sensory memories over an individual’s life, along with a procedure for comparing the sensory input of a current experience with the stored representations of one or more of those earlier experiences (Chandler, Reference Chandler2017).

Analogical Modeling (AM) (Skousen, Reference Skousen1989, Reference Skousen1992) seems to be fully compatible with our present comprehension of the psychological abilities and functions governing categorization behavior (Chandler, Reference Chandler2017). This model is based on the idea that past linguistic experiences are stored within the mental lexicon. When the need arises to analyze certain linguistic behaviors (such as pronunciation, morphological relationships, words, etc.), the lexicon itself is accessed. The process involves searching for the stored exemplars that closely resemble the one whose behavior is being predicted. Typically, the behavior of highly similar stored entities predicts the behavior of the one in question, although less similar ones also have a small probability of being applicable.

In detail, AM comprises three main components: (i) the dataset, which consists of exemplars accumulated in long-term memory and used as a basis for performing analogical operations on a current target form; (ii) the core of the AM, an algorithm designed to select from the dataset the exemplar(s) serving as the basis for analogically interpreting the target form; and (iii) decision rule(s) used to choose one or more forms in the analogical set, determining the basis for the analogical interpretation of the target form. In the process of classifying a target form, AM takes into account all exemplars that share certain features with the target (supracontext). However, only those that do not add uncertainty to the classification (homogeneity) are chosen for the final list, namely, the analogical set. Ambridge (Reference Ambridge2020a, p. 521) provides an example of how AM works:

While these models have allowed linguists to test the claims and implications of the exemplar approach explicitly against both observationally and experimentally obtained data, none of the three exemplar models has yet been applied to the incremental syntactic interpretation of word strings. Recently, Chandler (Reference Chandler2020), in its commentary to Ambridge (Reference Ambridge2020a), illustrated how to apply AM incrementally to the ambiguous sentence such as They fed her dog biscuits, arguing that a similar computational model could test Ambridge’s hypothesis empirically (p. 571). Within the field of language acquisition, Bod (Reference Bod2009) demonstrated how a computational learning algorithm is able to employ structural analogy in a probabilistic way. This process mimicked children’s language development, going from item-based constructions to abstract constructions, even simulating some errors witnessed in children producing complex questions.

A final consideration regards how the mechanism of analogy can be modeled in Distributional Models (introduced in Section 2.3). Word analogies have been used as a standard intrinsic evaluation task for measuring the quality of word (O. Levy & Goldberg, Reference Levy and Goldberg2014; Linzen, Dupoux, & Goldberg, Reference Linzen, Dupoux and Goldberg2016; Mikolov, Yih, & Zweig, Reference Mikolov, Yih and Zweig2013) and sentence embeddings (Ushio et al., Reference Ushio, Espinosa Anke, Schockaert and Camacho-Collados2021; Wang, Daille, & Hathout, Reference Wang, Daille and Hathout2021; Zhu & de Melo, Reference Zhu and de Melo2020). However, the task is usually defined as a candidate retrieval: Given a pair of words (Tokyo, Paris) and a third one (Paris), the goal is to identify the underlying relation behind the first pair (IS THE CAPITAL OF) and find the correct completion from a list of candidates to solve the analogy (France, in this case). However, a different perspective has been brought from works in visual analogy and deep learning techniques in the last few years. Researchers in computer vision (Ichien et al., Reference Ichien, Liu and Fu2021; Reed et al., Reference Reed, Zhang, Zhang, Lee, Cortes, Lawrence, Lee, Sugiyama and Garnett2015; Sadeghi, Zitnick, & Farhadi, Reference Sadeghi, Zitnick, Farhadi, Cortes, Lawrence, Lee, Sugiyama and Garnett2015; Upchurch, Snavely, & Bala, Reference Upchurch, Snavely and Bala2016) have built deep-learning neural network architectures that organize visual representations in the same way as distributional semantic vector spaces organize linguistic data. These works consisted of recognizing a visual relationship between two images and generating a transformed query image accordingly. Among others, Reed et al. (Reference Reed, Zhang, Zhang, Lee, Cortes, Lawrence, Lee, Sugiyama and Garnett2015) developed a novel deep network that was trained to perform visual analogies, transforming an image in the same way shown by a pair of example images. For instance, given a 3D image of a car in a frontal pose and its left-rotated version, the network should replicate the same rotation for another object, for example, a truck. What is relevant for language is that this model is directly trained on the objective of analogy completion, that is, it generates an appropriate image to make a valid analogy. Recently, Rambelli et al. (Reference Rambelli, Chersoni, Blache and Lenci2022) proposed a neural network simulating the construction of phrasal embedding as an analogical process by taking inspiration from word embeddings and computer vision techniques. The authors proposed an analogical model to create the distributional embeddings of new expressions by applying a variant of Reed et al.’s network and evaluated different architectures in terms of generalization and systematicity.

To conclude, current computational models of analogical inference in language are still rather rudimentary, and we are nowhere near possessing a model that captures not only the statistical abilities of speakers but also their preferences and limitations.

4.4 Summary

The present section introduced the cognitive mechanism of analogy in the debate on mechanisms of language processing. First, it delineated the main characteristics of analogy. While a comprehensive review of analogy in cognitive science might appear excessive, the primary objective is to inform the reader about the true nature of analogy, and the specific characteristics of this cognitive process. Indeed, a thorough description of the mechanisms of analogy could be beneficial for those who aim to incorporate analogy into a linguistic model.

Subsequently, it illustrated the usage-based rationales that support the proposition that analogical processing constitutes the basis of the human ability to create novel utterances. Specifically, the main assumption is that the interpretation of a novel linguistic expression can be derived from stored word sequences, which function as analogical bases. While this perspective has been successfully adopted in recent studies of language acquisition, we agree with Bybee’s thesis that analogy could also be applied in adult production to account for novel utterances. The notion of analogy as a mechanism behind language processing has profound implications for linguistic theory, as it attenuates the role of compositionality. For instance, a transparent sentence like reading a papyrus could be understood in analogy with stored the frequent expression reading a book (Rambelli et al., Reference Rambelli, Chersoni, Blache and Lenci2022) instead of assembling the meaning of the lexemes (which is, however, still possible). Consequently, examples of language productivity could be explained by analogical inferences rather than by sequential compositional operations. The question of which mechanism (combinatorial or analogical) takes place in a given moment is an open question the author is still interested in exploring.

The section also included a review of analogical models to underscore the necessity of implementing a computational model designed to process sentence-level constructions, raising the question of how to incorporate analogical mechanisms into this architectural framework.

5.1 Redefining Compositionality as a Property of Natural Language

A common assumption is that human thought and language are compositional by nature (Martin & Baggio, Reference Martin and Baggio2020). By specifically posing the problem that way, this statement implies that both the language of thoughts and natural language are intrinsically compositional. The relationship between language and thought is vast and particularly controversial, and this Element is not interested in entering this debate. Generally, an influential assumption is that thought is mainly prior to and independent of linguistic communication: It is the system of thought (semantics) that shapes language. The connection between language and thought has been examined by Jerry Fodor, who has claimed that just thought, and not language, is compositional. Addressing the question of which precedes the other – thought or language – he proposed that at least one of them must be compositional, and if only one is compositional, that is the one that has underived semantic content. Fodor suggested that if natural languages lack compositionality, their content then derives from the content of thought (Fodor, Reference Fodor2001, p. 234).

From a different field of study, Christiansen and Chater (Reference Christiansen and Chater2016a) mentioned that “compositionality, function argument structure, quantification, aspect, and modality are properties of the thoughts that language may express” (p. 51). In this perspective, if thoughts are compositional, then the language should be itself compositional. However, it seems more accurate to say that there is a “capacity” for compositional processing and representation in our mind, which is recruited and expressed in language (Baggio, Reference Baggio, Christiansen and Chater2020, p. 5). Analogously, the usage-based perspectives support the hypothesis that cognitive processes shape language. From this stance, the problem can be rephrased as follows: As we can aggregate concepts, we somehow apply this cognitive mechanism to linguistic processing.

This assumption yields a reformulation of what we mean when saying “language is compositional.” For instance, Dowty (Reference Dowty, Barker and Jacobson2007) proposes to apply the term natural language compositionality “to whatever strategies and principles we discover that natural languages actually do employ to derive the meanings of sentences, on the basis of whatever aspects of syntax and whatever additional information (if any) research shows that they do in fact depend on” (p. 6). In other words, it is one of the possible strategies used to explain productivity, but it is not the only one. A different position is represented by Baggio’s works. The author still considers compositionality a backbone of language; however, his idea has less to share with the traditional Fregean compositionality. For instance, Baggio, Van Lambalgen, & Hagoort (Reference Baggio, Van Lambalgen, Hagoort, Hinzen, Werning and Machery2012) claimed that the real issue about compositionality and open-ended productivity in natural language is “the balance between storage and computation.” While the centrality of compositionality is diminished, it is evident that human languages have algorithms for building meanings from their parts (Călinescu, Ramchand, & Baggio, Reference Călinescu, Ramchand and Baggio2023). The open question is so when this computational constraint occurs and modulates language processing. One possibility is that “compositionality can often be rescued by increasing the demand on the storage component of the architecture, whereas it must be abandoned if one puts more realistic constraints on storage” (Baggio, Van Lambalgen, & Hagoort, Reference Baggio, Van Lambalgen, Hagoort, Hinzen, Werning and Machery2012, p. 18).

The research on ERPs during sentence processing has brought evidence that novel sentences evoke stronger N400 components in the ERP waveform than sentences composed of more expected combinations. The effect reveals a cognitive effort to combine the meaning of a word with the current contextual meaning, suggesting that there is a large amount of stored knowledge in semantic memory about event contingencies and concept combinations, the so-called realistic constraints on storage (Baggio & Hagoort, Reference Baggio and Hagoort2011). In other words: “[compositionality] is no longer a principle applying to language or to linguistic theory as a whole, but a computational constraint on one processing phase $\dots$ in the brain’s language system” (Baggio, Reference Baggio2021, p. 15).

In this context, explaining what compositionality is in language should benefit from studies about how systems in the brain realize meaning composition within the bounds of neurophysiological computation. Is the human brain, our computational device, compositional? The challenge is to identify cortical networks and neurophysiological events responsible for composition.

Among others, Hendriks (Reference Hendriks2020) reviewed the literature about the role of syntax in meaning composition, focusing on children’s acquisition of simple transitive sentences such as The car is pushing the boy. The major conclusion is that children’s production of subject–object word order in languages such as English appears to be ahead of their comprehension of the subject–object word order. In other words, syntax plays a lesser role (or perhaps a different role) from what is envisaged by the view of syntax–semantics relations in Formal Semantics and generative syntax. Syntactic structure does not fully determine meaning composition. Instead, syntax is merely one of the sources of information constraining meaning and does not have a special status. Conversely, Mollica et al. (Reference Mollica, Siegelman and Diachek2020) investigated how semantic computation can take place when the syntactic structure is not licensed by the language’s grammar. The authors introduced a novel manipulation aimed at investigating the neural responses to sentences in which word order is disrupted by increasing the number of local word swaps while maintaining local dependency relationships – that is, combinable words remain close to each other.

1. 1. a. She left the museum and walked to her rooms to save money. (Intact)

   2. b. She left the and museum walked to rooms to her save money. (3swaps)

Using fMRI, they observed that word order degradation did not decrease the magnitude of the blood oxygen level-dependent response in the language network, except when combinable words were put so far apart that the composition among nearby words was doubtful. This observation means that even when the syntactic structure is violated, the language regions respond with equal strength as they did to syntactically correct inputs, confirming that some form of composition still occurs. Given these results, the authors can affirm that “semantic composition,” defined as combining the meaning of the words in a sentence without strict syntactic parsing, is the core computation of the language network (Mollica et al., Reference Mollica, Siegelman and Diachek2020, pp. 125–126).

To conclude, we could argue that compositionality, in the Fregean sense, cannot be considered the core property of natural language. It is still valid that compositional mechanisms exist at the cognitive and brain level, allowing the aggregation of the meaning of expressions. However, compositionality does not exclusively rely on syntactic parsing. In this sense, a reformulation of the classic representation proposed in generative tradition is needed.

5.2 Redefining Compositionality as a Processing Principle

Any model of comprehension aims to explain how language is processed in real time. Specifically, the central question concerns how individuals construct the meaning of a sentence by accessing the meanings of its component lexical items and by integrating those meanings into a coherent configuration. Two questions are thus the basis of any model of language comprehension: (i) what items are combined, and (ii) how these items are combined and integrated into a final, structured representation. As discussed in Section 2, the strong version of compositionality posits that interpretation is derived from the structure of utterances, with the syntactic form not contributing to semantic information. However, CxG offers a different perspective. First, the semantic primitives of combination are not lexical items, but constructions, that is, form-meaning pairs varying in schematicity and complexity. Some constructions have schematic slots that can be filled with other constructions, which in turn might have slots that can be filled in. Moreover, some constructions are syntactic patterns associated with a specific meaning (e.g., Ditransitive construction), so the interpretation of a sentence is also dependent on syntactic form. In a first, general stance, compositionality in CxG can be defined as follows: “By recognizing the existence of contentful constructions we can save the compositionality in a weakened form: The meaning of an expression is the result of integrating the meaning of the lexical items into the meanings of constructions” (Goldberg, Reference Goldberg1995, p. 16).

Therefore, constructions combine freely to form actual expressions as long as they do not conflict (Goldberg, Reference Goldberg2003, Reference Goldberg2019). For example, a sentence like Liza sent storage a book is unacceptable because the ditransitive construction requires an animate recipient argument, while the word storage refers to an inanimate argument (Goldberg, Reference Goldberg2003, p. 10). This composition is not related to inserting a lexical item into an argument construction, but it can be extended to any construction of different complexity. Grounded on a more formal representation, SBCG (cf. Section 2.2.3) offers a unification-based symbolic formalism for describing the mechanism by which two signs (constructions) are compared to ensure their features do not conflict. If compatible, these signs are merged to form a new, unified sign that combines the attributes and values of the original signs. This unification process operates under constraints specified within the signs themselves, which thus constitute the language grammar. Even though different CxG approaches define the way constructions are combined differently, one thing is clear: Constructions are the rules of compositionality. Syntactic, hierarchical structure with abstract representation does not play a role in comprehension: The specific properties of constructions (which can include surface form constraints as specific word order) are all that matter. In this sense, CxG frameworks adopt a “what you see is what you get” approach (Goldberg, Reference Goldberg2003, p. 10).

However, another aspect to consider is that people come to the task of interpretation with a vast amount of shared world knowledge and context (Goldberg, Reference Goldberg and Riemer2015). To this end, the semantic composition is constantly enriched (Jackendoff, Reference Jackendoff1997) with background knowledge and contextual constraints: The meaning of a sentence could be computed in a good-enough manner, using expectations instead of building a complete, accurate bottom-up analysis, with the result of leading to shallow interpretation or even misinterpretation (e.g., the renowned Moses illusion, cf. Section 3.2). Models that assume the online processing relies on chunking support this hypothesis: Instead of building a syntactic structure serving as support for the comprehension of a sentence, they hypothesize a mechanism that consists of incrementally building chunks at all levels of linguistic structure as rapidly as possible, using all available information predictively to process current input before new information arrives (Blache, Reference Blache2016; Christiansen & Chater, Reference Christiansen and Chater2016b, among others). This perspective is shared by usage-based constructionist approaches: “virtually all linguistic expressions, when first constructed, are interpreted with reference to a richly specified situational context, and much of this context is retained as they coalesce to form established units” (Langacker, Reference Langacker1987, p. 455). The notion of compositionality should account for different levels of semantic contribution, from constructional meaning to contextual information and world knowledge. However, most CxG formalisms lack a detailed formalization of how these sources of information are activated and how they contribute to the final interpretation.

A different perspective is offered by Baggio (Reference Baggio2021), which tries to include a syntactic form of compositionality. According to his model of language processing, semantic representations may be generated by both a syntax-driven processing stream and an “asyntactic” processing stream, either jointly or independently. Compositionality is viewed as a constraint on computation only in the former stream. This framework, which includes parallel streams for meaning and grammar, embodies these representational and processing capacities, with compositionality serving as a constraint on the syntax-driven stream. When complex natural language expressions have multiple meanings, and at least one of those meanings is solely a function of the meanings of the parts and their syntax, the language system can make predictions about upcoming linguistic inputs based on semantic constraints established by the material already processed. Compositionality is then preserved, but it is considered a specific constraint on computation.

While the centrality of compositionality has been largely minimized, it remains true that human languages possess algorithms for predictably constructing meanings from their parts. However, it is still unclear how the generativity of meaning should be modeled as a computational constraint that influences language processing and its outputs, even though not all complex meanings are equally governed by compositionality (Călinescu, Ramchand, & Baggio, Reference Călinescu, Ramchand and Baggio2023).

5.3 Compositionality, Analogy, and Productivity

Compositionality is often referred to as our ability to compose meanings into endlessly novel configurations. In linguistics, the question of productivity remains a central one: How can a speaker, who has been exposed to a few tons of thousands of sentences, become capable of understanding (and producing) virtually an infinity of utterances?

The previous section introduced the CxG perspective: A speaker interprets a new sentence by relying on the composition of different constructions, which can be defined as “emergent clusters of lossy memory traces that are aligned within our high- (hyper!) dimensional conceptual space on the basis of shared form, function, and contextual dimensions” (Goldberg, Reference Goldberg2019, p. 7). Besides, the successful production (or reception) of an utterance depends on previously encountered linguistic expressions, and it is likely to bring up a slight modification to the linguistic knowledge stored in our long-term memory.

This idea of productivity leads to a shift in the linguistic description as well. The mechanism underlying language production is not an a priori set of rules, but it is a force that dynamically changes previous inputs while generating novel outputs. A productive use of a construction is supported to the extent that the potential coinage falls within a densely covered existing cluster of cases that exemplifies the construction. When no conventional constructions are available to express an intended message in context, speakers must extend their existing constructions in novel ways. In the absence of conventional formulations, speakers rely on (combinations of) representations that are sufficiently effective for communication (Goldberg, Reference Goldberg2019).

Therefore, the usage-based constructionist perspective allows constructional knowledge to be both remarkably specific and flexible (Goldberg, Reference Goldberg2024). According to Goldberg (Reference Goldberg2019), if a cluster of lossy overlapping memory traces that constitute a construction is very specific, the range of contexts in which it is observed will be narrow. In other words, when observed utterances share similar contexts of use, the resulting learned cluster will be correspondingly narrow and specific. However, even highly specific constructions are occasionally extended flexibly, as speakers must use constructions in constantly changing contexts to convey an open-ended range of messages (Goldberg, Reference Goldberg2024).

A tenet of this Element is that a possible mechanism responsible for dealing with extending new constructions is the cognitive process of analogy, the “core of cognition” (Hofstadter, Reference Hofstadter, Holyoak, Gentner and Kokinov2001; cf. Section 4.2). Resuming the previously mentioned assumptions of Bybee (Reference Bybee2010), analogy depends on similarity in form and meaning between constructions, whether these constructions are of a concrete type (as in collocations or fixed structures) or an abstract type: A novel instance is compared to those stored in our long-term memory to infer the new representation. In this perspective, the probability or acceptability of a novel item is gradient and depends on the extent of similarity to prior uses of a construction. In a more radical stance, Ambridge (Reference Ambridge2020a) proposed to disregard completely abstraction: Unwitnessed forms are produced and comprehended “by on the fly analogy” across multiple stored exemplars. Similar to what Ambridge (Reference Ambridge2020a) argued, forms of which one has never had direct experience are produced and understood through “on the fly” analogical processes with respect to multiple stored exemplars (i.e., concrete representations of experiences) and weighed according to their degree of similarity to the new instance; comprehenders generalize via analogy to interpret and generate new linguistic experiences. This mechanism could be applied to entire sentence comprehension: The evolving syntactic structure of the new sentence emerges on the fly as it is compared to the previously interpreted exemplars (Chandler, Reference Chandler2020). In that sense, the mental lexicon could be conceived as a “vast storehouse of triggerable analogies” (Hofstadter, Reference Hofstadter, Holyoak, Gentner and Kokinov2001, p. 504): Every lexical expression, when used in speech (whether received or transmitted), could constitute one side of an analogy being made in real time in the speaker’s/listener’s mind.

However, this assumption does not entirely endorse the radical vision of Ambridge where there is no abstraction: Any kind of analogy is simply untenable without an abstract structure of some sort (Adger, Reference Adger2020). As already introduced, even constructions are somehow abstracted from their specific instances, and some schemata are more syntactic than concrete expressions. The hypothesis that analogical mapping with existing, lexicalized construction can be at the basis of productivity does not exclude a priori the existence of other processes that aggregate meaning. Analogy is one possible method of meaning production within a broader array of cognitive mechanisms that not only contribute to productivity but also generate more creative expressions (e.g., conceptual blending; cf. Hoffmann, 2024). The central proposal is that productivity is explainable as a continuum: Sometimes, a novel expression can be interpreted analogically from partially overlapping stored sequences, and sometimes, it is the result of a bottom-up compositional computation (defined as unification or other formalisms).

In conclusion, while today’s approaches of CxG offer a flexible framework of the construction combinations, more efforts should be made toward a comprehensive description of the mechanisms that undergo three different steps of language (adapted from Kleinschmidt & Jaeger, Reference Kleinschmidt and Jaeger2015):

1. 1. How we “recognize the familiar,” that is, how we deal with previously experienced and stored aspects of language (lexicalized construction, but also the integration with contextual knowledge);

2. 2. How we “generalize to the similar,” that is, how we comprehend a novel situation based on similar previous (linguistic) knowledge to not start from scratch each time a new situation is encountered (productivity); and finally

3. 3. How we “adapt to the novel,” that is, how is it possible to adapt beyond what is expected based on previous experience (creativity).

5.4 Toward a Constructionist Model of Language Processing

Defining compositionality is not just a theoretical matter, it is a pressing need for developing a cognitive (and computational) model of language processing. The observations and reviewed literature aim to establish a common ground for designing a formal representation of constructions integrated into a usage-based computational model of language processing – specifically, of language comprehension. Although this is a relatively recent line of research, some works have proposed different hypotheses about bridging the gap between linguistic and psycholinguistic theory (Huettig, Audring, & Jackendoff, Reference Huettig, Audring and Jackendoff2022; Lindes, Reference Lindes2022; Michel, Reference Michel2023). An example of frameworks that attempt this integration is presented here.

In terms of representation, Rambelli et al. (Reference Rambelli, Chersoni, Blache, Huang and Lenci2019) provided the formal basis for a constructionist model of language processing. Specifically, they introduced a novel semantic representation of CxG, termed Distributional Construction Grammar (DCxG), which integrates constructions with the vector representations used in Distributional Semantics. The primary objectives of this theoretical proposition were twofold: (i) to offer a comprehensive representation of semantic information within the CxG framework, and (ii) to incorporate distributional vectors into the construction representation, thereby accommodating the more usage-based aspects of meaning (Busso, Pannitto, & Lenci, Reference Busso, Pannitto and Lenci2018; Lebani & Lenci, Reference Lebani and Lenci2017; Levshina & Heylen, Reference Levshina, Heylen, Boogaart, Colleman and Rutten2014; Perek, Reference Perek2016, Reference Perek2018). Specifically, each construction is represented as an attribute-value matrix following the Signed-Based CxG formalism (Sag et al., Reference Sag, Boas, Kay, Boas and Sag2012). The sources of meaning are encoded separately in three components, which interact but can still be instantiated separately: constructional meaning, frames, that is, the schematic knowledge describing scenes and situations in terms of their semantic roles, and events, that is, the semantic information concerning particular event instances with their specific participants (McRae & Matsuki, Reference McRae and Matsuki2009). All these three components are associated with a distributional representation (Figure 4).

Figure 4 DCxG formalism from Rambelli et al. (Reference Rambelli, Chersoni, Blache, Huang and Lenci2019). (a) The read lexeme construction (with its distributional vector), (b) the reading frame containing the distributional representation of the semantic roles, and (c) student-read event as the specialization of reading-frame.

Distributional Construction Grammar stands as one of the few works aiming to establish a unified representation of grammar and meaning, grounded on the assumption that language structure and properties emerge from language use. As a linguistic representation, the model develops language structure, properties, and meanings from the distributional statistics observed in text corpora, coherent with the idea that language emerges from language use. On the modeling side, this framework is structured to incorporate linguistic information and world knowledge into the semantic representation of linguistic input, employing an incremental and predictive process. Specifically, this representation has been developed to be the basis of a computational semantic processing model founded on the interaction between the three informational structures with the mechanisms to meaning access: activation, similarity, and unification (Blache et al., Reference Blache, Chersoni, Rambelli and Lenci2023).

While the previous work is mostly related to a representation aspect, Blache (Reference Blache2024) uses a similar representation to propose a neurocognitive architecture of language processing, integrating constructions and the Memory, Unification, and Control model (MUC; cf. Hagoort, Reference Hagoort2013, Reference Hagoort, Hickok and Small2016), a general framework for sentence comprehension that aims at accounting for the balance between storage and computation (Baggio, Van Lambalgen, & Hagoort, Reference Baggio, Van Lambalgen, Hagoort, Hinzen, Werning and Machery2012). Without going into the specific details, this framework relies on two mechanisms: prediction and unification. Prediction calculates the most likely next sign (constructions) based on the context. It is always at work and signs (constructions) at any granularity can be predicted. While linguistic theories rely on a mechanism (derivation, constraint solving, etc.) that linearly aggregates objects of categories at the same level, linguistic objects used as basic components can be of any granularity (in line with CxG) and do not necessarily correspond to a category of the same level, meaning that the integration mechanism is no longer linear. Unification, on the other hand, is an operation consisting in comparing two structures, assessing their compatibility, and building a resulting structure merging both. In the case of lexical access, unification is the controlling mechanism for identifying the matching entry. In the case of situation model updating, it implements nonlinear compositionality by integrating the current sign into a structure. This prediction–unification model provides a framework bringing together unique architecture facilitation mechanisms besides classical incremental processing. This is done due to a single processing cycle based on the integration of complex multilevel structures. In addition to explaining how to integrate facilitation mechanisms, this model also brings a new vision about the two different ways to build meaning: compositionality or direct access. In this approach, these two mechanisms only differ in one point: the granularity of the signs to integrate into the situation model. Building the meaning is always done compositionally but can correspond to a word-by-word incremental mechanism (the classical view of compositional principle) or on the opposite in the integration of entire and large pieces of meaning.

The presented models are far from being a complete representation of language processing, and different works could propose different versions of composition (instead of unification) or the alternation of separated cognitive processes, such as analogy, conceptual blending, and so on (Goldberg & Ferreira, 2022; Hoffmann, 2024; Rambelli et al., 2022). In conclusion, new efforts in CxG should be oriented toward integrating construction formalisms into an architecture that combines several processing mechanisms observed in cognitive and neurolinguistic literature. To correctly implement a model of language comprehension, two aspects should be explicitly defined: (i) which type of information resides in the lexicon and in the long-term memory, and how such information is represented, and (ii) which type of principles guide the constraint-based unification that also produces potentially novel combinations. On top of this model, it should be included how the most creative expressions are interpreted (in contrast with productive sentences).