2.1. The deceleration of innovation

Earlier studies have provided coarse-grained empirical evidence on possible declining trends in innovation. For instance, Huebner (Reference Huebner2005) discovered a declining trend of breakthrough inventions over time-based on the count of noticeable innovations in history. Jones (Reference Jones2009) observed an increase in inventors’ age at their first invention, which suggests that more education and exploration time are required before actualizing originality in their ideas. Luo & Wood (Reference Luo and Wood2017) found a decrease in the number of patents per average inventor, indicating a decline in inventive productivity. Bloom et al. (Reference Bloom, Jones, Van Reenen and Webb2020) reported declining research productivity based on firm- and industry-level census data in the semiconductor, crops and health industries. Park, Leahey & Funk (Reference Park, Leahey and Funk2023) investigated 45 million papers and 3.9 million patents between 1945 and 2010 and how they change citation networks to prior work over time as a way to measure their disruptive impact. Their results show a declining trend of such impact for both papers and patents.

2.2. Measures of innovation

These prior studies commonly investigate patent data but only employ measures that are extrinsic to innovations to detect potential trends in innovations. A technological innovation often appears and gets recognized as a new method, tool, product, system, service or artifact. The newness or originality distinguishes it as an innovation from normal designs or artifacts, but not all elements of an innovation need to be original (Simonton Reference Simonton1999; Sternberg & Lubart Reference Sternberg and Lubart1999; Kaufman and Baer Reference Kaufman, Baer, Sternberg, Grigorenko and Singer2004; He & Luo Reference He and Luo2017). In this study, we aim to look inside innovations and investigate the elemental concepts that constitute innovations and contribute to their originality. More specifically, we focus on measuring the originality of new concepts when they appear for the first time in the total technological concept space to investigate innovation trends over time. Note that the term “originality” is often used interchangeably with “novelty” in literature. Hereafter, we use “originality” for consistency.

Prior studies present various ways to assess originality in innovation. Creative design evaluation is mostly carried out by experts or other types of human subjects, with or without structured guidance or procedures (Sarkar & Chakrabarti Reference Sarkar and Chakrabarti2007; Brown Reference Brown2015; Ahmed & Fuge Reference Ahmed and Fuge2018; Hay et al. Reference Hay, Duffy, Gilbert, Lyall, Campbell, Coyle and Grealy2019; Sosa Reference Sosa2019). However, human experts are subject to their personal knowledge, experiences, opinions and intuitions (Weisberg Reference Weisberg2006; Oman et al. Reference Oman, Tumer, Wood and Seepersad2013). What appears original to one group of experts might not be so to others. Ideally, the originality of a technological concept should be assessed with reference to all previous concepts in history (Boden Reference Boden1996). Experts may have incomplete knowledge of prior art, insufficient cognitive capacity to assess numerous new concepts, ideas or innovations – for example, at the magnitude of millions – and are unlikely to provide statistical significance in their assessments.

To cope with the limitations of human evaluation, data-driven and computational approaches have been developed to evaluate the originality of designs, inventions, creative design ideas and so on (Luo Reference Luo2023a). Many of these approaches are based on patent databases. Patents contain rich design or technological information and have been widely used as proxies for technological innovations in empirical studies. Patent data have been extensively employed as digital design repositories to develop engineering design theories and data-driven design support methods and tools (Jiang et al. Reference Jiang, Sarica, Song, Hu and Luo2022), although patent data sources are subject to a complex institutional system and many non-technical factors, such as examination processes, policy changes and industry differences.

Patent databases host millions of patent documents in all technological domains and thus can support statistically significant, rigorous and systematic data-driven evaluations of inventiveness or originality. Fleming & Sorenson (Reference Fleming and Sorenson2001), Kim et al. (Reference Kim, Cerigo, Jeong and Youn2016) and He & Luo (Reference He and Luo2017) statistically analyze the rareness of historical co-occurrences of co-classes of patents (or the classes of patent references) to measure the originality of patented inventions from a recombination perspective. Uzzi et al. (Reference Uzzi, Mukherjee, Stringer and Jones2013) assess the rareness of the combinations of references of millions of academic papers to provide statistically significant indicators of the originality of published research.

2.3. Natural language processing

More recent studies have leveraged large pre-trained lexical databases and natural language processing (NLP) techniques to automatically assess the originality of newly generated design concepts in natural language descriptions (Siddharth et al. Reference Siddharth, Blessing and Luo2022a). Siddharth, Madhusudanan & Chakrabarti (Reference Siddharth, Madhusudanan and Chakrabarti2020) propose a method to evaluate the novelty of a design solution, based on its distance to a reference product database. This distance is calculated as text similarity using the SAPPhIRE model. Camburn et al. (Reference Camburn, He, Raviselvam, Luo and Wood2020) evaluate the novelty of a large quantity of crowdsourced design ideas according to the semantic distance among the terms in the design idea description texts, with semantic distance derived from Freebase. Han et al. (Reference Han, Forbes, Shi, Hao and Schaefer2020) employ ConceptNet to assess the novelty of new design ideas based on the semantic distance between elemental concepts. Gerken & Moehrle (Reference Gerken and Moehrle2012) used Subject-Action-Object triplets to measure the semantic similarity between patents to indicate the novelty of patents concerning others. Olson et al. (Reference Olson, Nahas, Chmoulevitch, Cropper and Webb2021) similarly used a measure of semantic distance among words that a person generates as an indicator of thought space divergence and creativity.

To calculate the semantic distance (or similarity) between words or phrases, a comprehensive knowledge base is necessary. WordNet and ConceptNet have been the most used knowledge bases for this purpose (Linsey, Markman & Wood Reference Linsey, Markman and Wood2012; Georgiev & Georgiev Reference Georgiev and Georgiev2018; Kan & Gero Reference Kan and Gero2018; Goucher-Lambert & Cagan Reference Goucher-Lambert and Cagan2019; Han et al. Reference Han, Sarica, Shi and Luo2022). TechNet is a relatively newer knowledge base, trained on engineering design data, such as patent texts, and is more suitable than ConceptNet, WordNet and other common-sense knowledge bases for assessing semantic distance or similarity between technological concepts (Sarica et al. Reference Sarica, Luo and Wood2020, Reference Sarica, Han and Luo2023).

On the other hand, the originality of terms used in a patent or an academic paper can serve as an indicator of the originality of the described design or research finding. New terms, compared to prior terms in the literature or technological knowledge base, represent original concepts that may challenge the status quo and shape the future directions of the corresponding field (Kuhn Reference Kuhn1970; Wray Reference Wray2011). For instance, Park et al. (Reference Park, Leahey and Funk2023) discovered a trend of declining new word-pair occurrences in patent titles, indicating a decrease in the diversity of word usage and a decline in combinatorial novelty. Originality is a matter of degree.

In this study, we employ TechNet, a large semantic network of technical concepts, as the semantic knowledge base for assessing the degree of originality of technological concepts created over time. The originality assessment centers on the semantic distance (the opposite of semantic similarity) between new terms in new patents and prior terms in earlier patents, using several metrics based on graph theory and information theory. Consequently, our analysis focuses on concepts in patentable technological inventions from the past four decades. In the following sections, we introduce TechNet, the patent data source and our metrics in detail.

3.1. Technology semantic network (TechNet)

The technology semantic network utilized for this research, termed TechNet, has been pre-trained and published in our prior work (Sarica et al. Reference Sarica, Luo and Wood2020). It is made publicly available on the web (https://www.tech-net.org) for external researchers, allowing access to broader research and applications. TechNet is a semantic network comprising of 4,038,924 technology concepts (words and phrases up to 4 grams) and their pairwise semantic distances. To ensure comprehensive coverage of technology concepts across all technology and engineering domains, the source data for constructing TechNet included all 5,771,030 utility patents granted between 1976 and October 2017 from the USPTO database.

Due to incomplete data coverage for the year 2017, we chose not to include data from that year. Furthermore, the digital patent database only became available starting in 1976. Since concepts created before 1976 could be reused in innovation and appear in patents after 1976, the “new” concepts in the initial years of the database may not be genuinely new, but only new to the database. Our analysis reveals that concepts appearing in patents between 1976 and the end of 1980 constitute over 90% of the total concepts used throughout the entire period from 1976 to 2016, indicating comprehensive coverage of baseline patent knowledge. Consequently, we established an initial semantic network, including concepts that appeared in patents between 1976 and the end of 1980 as the baseline. Our trend analysis commences in 1981.

In constructing TechNet, we began by extracting words and phrases from the raw texts of patent titles and abstracts. These extracts represent meaningful technological concepts (e.g., functions, components, structures, materials, configurations, working principles) and were processed using natural language processing techniques for phrasing, denoising and lemmatizing, among others. The original text database contains 26,756,162 sentences and approximately 699 million words. Utilizing this data, several word embedding models were trained on the preprocessed sentences to derive the embedding vectors of individual concepts, thereby forming a unified embedding vector space representing the total technological concept space. A technological semantic network was then constructed by connecting these technological concepts based on the cosine similarity of their embedding vectors, that is, semantic similarity or an inverse indicator of semantic distance. By using the total database to train the embedding space of concepts, we anticipate that the embedding vector similarity will reflect the most intrinsic technical relations between the technical concepts represented by the terms.

Sarica et al. (Reference Sarica, Luo and Wood2020) performed a benchmark comparison with other large semantic networks and knowledge databases (e.g., WordNet and ConceptNet, most of which were trained on common-sense databases) and found that the word2vec embedding model (Mikolov et al. Reference Mikolov, Sutskever, Chen, Corrado and Dean2013) yielded the best-performing technology semantic network for concept retrieval and inference tasks within the specific context of technology and engineering. Consequently, the technology semantic network (TechNet) used in this study is based on word2vec. Footnote ¹

Figure 2 shows an example subgraph consisting of 30 concepts sampled from the technology semantic network cumulative to the year of 1990. With an interest in new concept creation and the evolution of concept space over time, we created the longitudinal technology concept network cumulative to each year from 1980 to 2016 and identified the concepts that appeared for the first time and were “original” in each respective year. The associations of subsets of concepts in the yearly networks are based on their pairwise semantic similarity (i.e., embedding vector similarity) and are derived from the total embedding space trained on the complete patent database from 1976 to 2016. Therefore, although the concepts included in yearly networks change, their pairwise semantic similarity is universal. In the next section, we introduce the graph-theoretic and information-theoretic metrics in our analysis based on TechNet.

Figure 2. An example subgraph of 30 concepts sampled from the total technology concept network cumulative to 1990. (A) The adjacency matrix representation of the subgraph where the value of each cell is the semantic similarity of the corresponding tuple. (B) A filtered network representation of the subgraph. In the total concept network cumulative to 1990, the share of the new concepts in cumulative total concepts is 5.4%. Preserving this ratio, the sample subgraph has 2 new concepts and 28 prior concepts. The concepts “artificial neural network” and “unsupervised learning” appeared for the first time in 1990, whereas the other 28 concepts had occurred in previous years.

3.2. Graph and information theoretic metrics

3.2.1. The network of concepts and originality of new concepts

For a concept network $ G=\left(V,E\right) $ , let two concepts be $ {v}_i $ and $ {v}_j\in V $ . The semantic similarity between them is denoted by $ {w}_{ij} $ . Then, the mean semantic similarity of concepts in the network (representing a concept space) is calculated as:

(1) $$ {w}_G=\frac{2}{N\left(N-1\right)}\sum \limits_{i,j,i\ne j}{w}_{ij}, $$

where N is the number of concepts in the network.

$ {w}_G $ is an inverse indicator of the divergence of the concept space. Recently, Olson et al. (Reference Olson, Nahas, Chmoulevitch, Cropper and Webb2021) similarly used a measure of the average semantic distance among words that a person generates as the indicator of thought space divergence and creativity. This measure will further allow us to detect whether the cumulative technological concept space has been diverging or converging over time, as new concepts continually enter the space each year.

We further assess the mean semantic similarity $ {w}_N $ between the new and prior concepts as:

(2) $$ {w}_N=\frac{1}{\left|U\right|\mid V\mid}\sum \limits_{\forall i\in U,\forall j\in V}{w}_{ij}, $$

where U is the set of concepts that appeared prior to the corresponding year and V is the set of new concepts that appeared in the corresponding year. The calculation considers only the edges between the concepts in sets $ U $ and $ V $ .

$ {w}_N $ is an inverse indicator of the originality of the new concepts that appeared for the first time in a year. It follows the spirit of a few recent studies that similarly used measures of semantic distance between words or terms as novelty indicators of design ideas (Goucher-Lambert & Cagan Reference Goucher-Lambert and Cagan2019; Camburn et al. Reference Camburn, He, Raviselvam, Luo and Wood2020). This metric will further allow us to detect the longitudinal change in the originality of the new concepts appearing for the first time each year.

Due to the large size of the total TechNet, for each year, we randomly sample 100 subgraphs of the total concept network cumulative to each year for calculating the graph theoretic metrics. Each subgraph contains 1,000 (or 500, 2000, 5000 in the robustness tests) randomly sampled concepts from the total network cumulative to each year. In each random subgraph, the share of new concepts is preserved to be the same as the share in the total network cumulative to that year.

3.2.2. Concept information content

We also measure the additional new information content that is brought by a new concept to the cumulative space of technological concepts. This can imply the amount of learning required to remove uncertainty over the meaning of the new concept. Assuming that all prior concepts have been known by the collective intelligence of human innovators, the information content of a new concept can be approximated as the sum of the information content of the most similar prior concept to the new concept and the additional information content that the new concept brings in. This can be expressed as:

(3) $$ \mathrm{IC}\left({x}_{\mathrm{new}}\right)=\mathrm{IC}(p)+\Delta \mathrm{IC}\left({x}_{\mathrm{new}}|p\right), $$

where $ \mathrm{IC}\left({x}_{\mathrm{new}}\right) $ is the information content of the newly introduced concept $ {x}_{\mathrm{new}} $ , $ \mathrm{IC}(p) $ is the information content of the prior concept p most similar to $ {x}_{\mathrm{new}} $ , and $ \Delta \mathrm{IC}\left({x}_{\mathrm{new}}|p\right) $ is the additional information content that $ {x}_{\mathrm{new}} $ brings to collective concept space given the most similar prior concept p.

According to Shannon’s information entropy, the information content of an event x is,

(4) $$ \mathrm{IC}(x)=-\log P(x). $$

It states that if an event x has a lower probability of occurrence, that is, if P(x) is lower, its information content is higher. Thus, an unexpected event carries more information than a highly expected event.

By inserting equation (4) into equation (3), we obtain $ \Delta \mathrm{IC}\left({x}_{\mathrm{new}}|p\right) $ as:

(5) $$ \Delta \mathrm{IC}\left({x}_{\mathrm{new}}|p\right)=\log \frac{P(p)}{P\left({x}_{\mathrm{new}}\right)}. $$

Since it is assumed that prior concepts are collectively known, $ P(p) $ can be approximated as 1 (i.e., the probability that someone is an expert on prior concept p is 1). On the other hand, $ P\left({x}_{\mathrm{new}}\right) $ can be approximated as the cosine similarity between the new concept and its most similar prior concept. In other words, the probability of inferring the meaning of the new concept is approximated by its maximum similarity to prior concepts.

Equation (5) with log base 2 is used to calculate each new concept’s expected additional information content when it appeared for the first time. On this basis, we measure the mean additional information content that a sample of 1,000 (or 500, 2,000 and 5,000 in the robustness tests) randomly sampled new concepts bring to the technology semantic network each year. Then we detect the changes in the mean additional information content of new concepts each year.

Our data and codes are available at https://github.com/SerhadS/techspace-evolution.

4.1. Findings

We now report the longitudinal changes in the macro- and micro-structures of the technological concept network over the past four decades, focusing on the originality and information content addition of new concepts that appeared for the first time in the technological concept space cumulative to each year.

First, we observe that the number of accumulated concepts within the total technological concept space has exhibited linear growth, signifying a relatively steady annual creation of new concepts. The annual growth rate of new concepts has consistently decreased over the past four decades (Figure 3).Footnote ² This rate is calculated by dividing the number of new concepts in each year by the cumulative number of concepts in the total space up to that year. Had the growth rate been constant or increasing, the cumulative curve would have taken an exponential shape. Moreover, the ratio of new concepts to all unique concepts, assessed in rolling 5-year windows, has also seen a decline (refer to Supplementary Material). Concurrently, there has been a reduction in the average number of new concepts per patent, decreasing from 2.15 to 0.54, while the total number of concepts per patent has remained relatively stable. In summary, the expansion of the total technological concept space is linear, rather than exponential.

Figure 3. The total number of concepts and the proportion of new concepts to the total number of concepts in the network, accumulated up to a given year.

Second, as shown in Figure 4, from 1981 to 2016, the mean semantic similarity of all concepts increased by 23%. This suggests that the concepts in the technological concept space are converging over time. We conducted Kolmogorov–Smirnov tests and the results show significant increases across years and periods of 5 years (see Supplementary Material). Furthermore, the mean semantic similarity between new and prior concepts increased by 31%.Footnote ³ This suggests that concepts created each year (i.e., appearing for the first time in the network cumulative to that year) are becoming more similar to prior concepts, implying that the originality of new concepts is diminishing over time. To test the robustness of this pattern, we also experimented with samples containing 500, 2,000 and 5,000 concepts to calculate mean semantic similarity, and the results are reported in Figure 5. The trend remains consistent.

Figure 4. The mean semantic similarity of all concepts and the mean semantic similarity between new and prior concepts in the network accumulated up to a given year. Due to the size of the technology concept network, for computational efficiency, we sampled 100 subgraphs, each comprising 1,000 randomly selected concepts, from the total network accumulated up to each year, and calculated the means and standard deviations of the mean semantic similarity for the 100 subgraphs.

Figure 5. Robustness tests for mean semantic similarity measurement. The mean (node) and standard deviation (error bar) of semantic similarities of the concepts in 100 randomly sampled subgraphs, each consisting of (A) 500 concepts, (B) 2,000 concepts and (C) 5,000 concepts each year. The differences between sub-plots suggest higher variance for smaller subgraph sizes and lower variance for larger subgraphs, as expected.

Third, Figure 6 shows a continuous 21% decrease in the mean additional information content that an average new concept contributes to the prior total concept space from 1981 to 2016. We conducted Kolmogorov–Smirnov tests, and the results demonstrate that the decreases across years and periods of 5 years are significant (see Supplementary Material). To test the robustness of this trend, we experimented with samples containing 500, 2,000 and 5,000 concepts to calculate mean additional information content, and the results are displayed in Figure 7. This consistent trend suggests that the newly created concepts are contributing a diminishing amount of new information to the existing knowledge base. For example, when the term “deep learning” first appeared in TechNet, it was highly associated with many previously existing concepts, such as “neural network,” “machine learning” and “regression,” thus adding little new information to the prior concept space. The same applies to “blockchain,” “Web 3,” “metaverse,” and many other emerging terms, which can be considered as “rehashed concepts” with low originality.

Figure 6. The mean additional information content contributed by 1,000 randomly selected new concepts to the technology concept network. The means and standard deviations are denoted by the nodes and error bars, respectively.

Figure 7. Robustness tests for mean additional information content measurement. Longitudinal change in mean (node) and standard deviation (error bar) additional information content brought by new concepts in samples of (A) 500 concepts, (B) 2,000 concepts and (C) 5,000 concepts in each year. Although the sub-plots are similar, smaller samples exhibit slight fluctuations, which diminish in larger ones.

Similar or related trends have been documented in several prior studies, albeit with more coarse-grained empirical evidence at the level of discrete breakthrough inventions (Huebner Reference Huebner2005), patented inventions (Luo & Wood Reference Luo and Wood2017), or regarding the behaviors and productivity of individuals (Jones Reference Jones2009), firms and industries (Bloom et al. Reference Bloom, Jones, Van Reenen and Webb2020). Our analysis examines the elemental concepts that constitute inventions and their relations that form the total technology space, thereby providing both finer-grained and more macro-empirical evidence on the possible trends of innovation. Notably, our work is enabled by the latest natural language processing technologies and, specifically, the large pre-trained technology semantic network, TechNet, which has only recently become publicly available.

4.2. Interpretations

The linear expansion of the total technological concept space, coupled with the shrinking originality of new concepts over time, implies diminishing returns on the creation of new concepts. This trend may stem from the dominant negative feedback illustrated in Figure 1, which outweighs the positive feedback.

The cumulative expansion of the total technological concept space as a result of continual innovation implies more knowledge prerequisites for future innovators (Jones Reference Jones2009; Callaghan Reference Callaghan2021) and more prior concepts to benchmark against to derive originality in future innovation. To invent new technologies and create original concepts against an expanding space of prior art, future innovators must learn, master, utilize and synthesize an ever-increasing number of prior concepts and knowledge than ever before. Many new concepts are “rehashed concepts” rather than truly original ones and might confuse the learning of young students and future innovators. Additionally, the increasing complexities in new technologies, design processes and organizations, as evidenced and analyzed in many prior studies (de Weck, Roos & Magee Reference de Weck, Roos and Magee2011; Luo & Wood Reference Luo and Wood2017), may also introduce further challenges to future innovation.

On the other hand, the accumulation and expansion of the total technology space may offer more knowledge ingredients for potential reuse, recombination and synthesis into new concepts. This is the positive feedback depicted in Figure 1. However, such creative recombination potentials are conditioned on the cognitive capabilities of humans to fully utilize, learn and synthesize the growing space of prior concepts. The linear expansion of the technological concept space and diminishing originality of new concepts characterize the era to date when innovation primarily relied on biologically limited human intelligence. Moving forward, AI may help cope with the growing knowledge burden and “complexity” challenges to human intelligence for innovation.