Sentiment Analysis and Sentiment Score

Sentiment analysis is a recently popularised technique in natural language processing capable of deciphering the underlying attitudes, emotions, or opinions expressed within author-generated texts. Essentially, sentiment analysis quantifies subjective language elements such as opinions, emotions, or attitudes related to a topic, person, or entity (Reference Korenek and ŠimkoKorenek & Šimko, 2014; Reference RoldosRoldos, 2020). Pioneering studies (Reference Martins, Almeida, Henriques and NovaisMartins et al., 2021; Reference Narayanan, Gaston and DozierNarayanan et al., 2018; Reference Panicheva, Cardiff and RossoPanicheva et al., 2010) have found correlations between authorship analysis and the authors’ sentiment.

Reference Panicheva, Cardiff and RossoPanicheva et al. (2010) was among the few researchers to discover that each author individually and uniquely expresses their subjective evaluation and appraisal, thereby linking authorship attribution with sentiment polarity classification. Reference SchneiderSchneider (2015) explored the efficacy of sentiment analysis in authorship attribution, finding that this new stylometric feature underperformed traditional stylometric features, implying a correlation between authorship attribution and sentiment analysis (Reference Martins, Almeida, Henriques and NovaisMartins et al., 2021; Reference Narayanan, Gaston and DozierNarayanan et al., 2018; Reference Panicheva, Cardiff and RossoPanicheva et al., 2010). Over the past decade, sentiment analysis has been used to profile and identify authorship despite most related studies not focusing on forensic authorship.

Reference Ezaldeen, Misra, Bisoy, Alatrash and PriyadarshiniEzaldeen et al. (2022) generated an e-learning recommendation system for individual learners using sentiment analysis of online reviews to extract individual preferences. Reference Martins, Almeida, Henriques and NovaisMartins et al. (2021) demonstrated how to identify authors’ emotional profiles from social media texts through sentiment analysis and use these profiles to improve authorship attribution. According to Reference Narayanan, Gaston and DozierNarayanan et al. (2018, p. 931), ‘sentiment analysis is a valuable asset to authorship attribution’.

These studies pioneered authorship analysis utilising sentiment analysis from the perspective of NLP. However, according to Reference GrantGrant (2007), these computational studies were primarily designed to enhance authorship algorithms or classification models rather than exploring and explaining texts and authorship patterns in relation to demographic factors, which is an essential aspect of forensic linguistic studies. In contrast, forensic linguistics emphasises the exploration and explanation in authorship analysis, which aligns with this Element’s main focus: authorship, sentiment, and their relationship.

Keyness Analysis and Keyword Features

The keyness approach, generating keywords, is a very promising corpus technique for exploring discourses of various disciplines (Reference Gabrielatos, Taylor and MarchiGabrielatos, 2018; Reference GriesGries, 2021). Data-driven, keyness-based keywords feature prominently in a text for particular reasons. They are identified due to their statistically significant frequency compared to reference texts (Reference Bondi, Bondi and ScottBondi, 2010; Reference Gabrielatos, Taylor and MarchiGabrielatos, 2018; Reference GriesGries, 2021), offering insights into a text’s ‘aboutness’ (Reference Bondi and ScottBondi & Scott, 2010) and ‘style’ (Reference Bondi and ScottBondi & Scott, 2010; Reference Scott and TribbleScott & Tribble, 2006; Reference Taylor and MarchiTaylor & Marchi, 2018), which are thought to be indicative of an author’s demographic characteristics.

Aboutness denotes a text’s main concepts, subject matter, and attitudes (Reference Gabrielatos, Taylor and MarchiGabrielatos, 2018; Reference Scott and TribbleScott & Tribble, 2006; Reference ZhangZhang, 2021), such as the name of the main characters in a play. Style, on the other hand, is characterised by unique textual qualities and an author’s ‘position and identity’ (Reference Bondi, Bondi and ScottBondi, 2010, p. 7) in a text or corpus, such as the use of pronouns and exclamations (Reference Scott and TribbleScott & Tribble, 2006). For example, personal pronouns have been identified as gender features in a study of Chinese language use on weblogs (Reference ZhouZhou, 2007). Additionally, intensifiers, which are part of graduations (Reference Martin and WhiteMartin & White, 2005), should also be considered part of the ‘style’ of a text as they are gender-related features and could potentially be the clue to distinguishing the gender of a Chinese weblog writer (Reference ZhouZhou, 2007).

Keyness analysis, by generating keywords, provides a robust technique to explore both the similarities (Reference KilgarriffKilgarriff, 1997; Taylor, 2013) and the differences (Reference KilgarriffKilgarriff, 1997; Reference ScottScott, 1998) between texts or corpora because ‘any difference in the linguistic character of two corpora will leave its trace in differences between their word frequency lists’ (Reference KilgarriffKilgarriff, 1997, p. 233).

Furthermore, based on preliminary research titled ‘From Keywords to Authorship Profiling: A Keyness Approach’ (Reference ZhangZhang, 2019) and its findings, the keyness approach demonstrates its effectiveness in identifying clues to various profiling characteristics of a text’s author. Statistically, as mentioned earlier, the keyness-based keywords or key phrases are words or phrases with a frequency which is statistically significant (Reference Bondi, Bondi and ScottBondi, 2010; Reference Gabrielatos, Taylor and MarchiGabrielatos, 2018; Reference GriesGries, 2021). Linguistically, these words or phrases make the text distinctive (Reference Gabrielatos, Taylor and MarchiGabrielatos, 2018). Therefore, it is hypothesised that the keyness approach could provide clues to an author’s identity despite not being specifically designed for authorship profiling.

Furthermore, the keyness approach is noteworthy for two more reasons. Firstly, a pilot exploration has found that this approach is not sensitive to the length of the observed texts, which allows for the exploration of short texts for potential authorship profiling. Even a very brief text could generate a keyword list with the aid of the keyness approach. Secondly, the approach is cross-linguistic, making it possible to analyse the authorship of anonymous texts written in any language, a feature especially useful in investigating transnational crimes.

Authorship Features and Idiolect

An author’s idiolect is composed of a habitual and unique combination (Reference McMenaminMcMenamin, 2002) and co-selection of various authorship features. The term ‘co-selection’ refers to the simultaneous choice or use of multiple linguistic features, much like how a person’s shopping habits reflect their specific needs and individual preferences. For instance, the habitual shopping list of a cat owner would significantly differ from that of a parent with a baby. Just as individuals can be differentiated by the unique combinations of items on their shopping lists, which are tailored to their particular circumstances and lifestyle, writers exhibit a ‘linguistic shopping list’. This list encompasses their preferred vocabulary, syntactic patterns, and other linguistic styles used consistently and distinctively (Reference GrantGrant, 2013) within their writing. These specific features that make up an individual’s ‘linguistic shopping list’ are not random assortments but are often used together in a patterned way. These patterns, unique and distinct to each person, comprise their linguistic fingerprints, or idiolects, and can be pivotal in distinguishing one individual’s language use from another’s. This is because, while many people might share some features, the exact pattern of co-selected features is highly individual.

In the context of this Element, whether the patterned combination of the aforementioned authorship features can constitute distinct and unique aspects of authors’ idiolects is assessed. Each idiolect is considered a distinct pattern of linguistic features. Consequently, subsequent sections use the term ‘pattern’ in this specific sense, denoting the individualised linguistic characteristics that comprise a person’s idiolect.

Authorship Features and Mindset

An author’s unique set of linguistic features constitutes their idiolect, reflecting individual choices in language use. This idiolect is not just a set of random linguistic features; it is closely tied to the author’s mindset, mirroring cognitive patterns, and perspectives. Therefore, there exists a profound interconnection between our linguistic preferences and our mental frameworks.

As noted earlier, a mindset is defined as a cognitive filter through which we look at the world (Reference Earley, Murnieks, Mosakowski, Javidan, Steers and HittEarley et al., 2007; Reference Gupta and GovindarajanGupta & Govindarajan, 2002; Reference Sistek-ChandlerSistek-Chandler, 2019). Thus, our mindset, encompassing our beliefs, values, attitudes, and perspectives, influences our decision-making processes, habitual ways of thinking, and communication styles (Reference Pourdehnad, Warren, Wright and MairanoPourdehnad et al., 2006; Reference Sistek-ChandlerSistek-Chandler, 2019). This, in turn, affects the words we select and how we express ourselves. Consequently, the writings are essentially filtered through the author’s mindset.

Synchronically, these writings combine various linguistic features. Some of these features remain consistent across an author’s body of writings and distinctive enough to differentiate one author from another (Reference GrantGrant, 2013). Such features in combination form what could be referred to as the author’s writing patterns or habits, which are important for characterising and predicting authorship. It is believed that such writing habits or patterns develop as a result of an action-oriented mindset which is characterised by a focus on taking action and getting things done quickly (Reference Verplanken and OrbellVerplanken & Orbell, 2019). This contrasts with a deliberative mindset that focuses on careful consideration and analysis prior to taking action (Reference Verplanken and OrbellVerplanken & Orbell, 2019). While these two mindsets may seem contradictory, they generally co-exist in each individual and can actually complement each other in certain circumstances.

Diachronically, our mindsets are relatively stable, given that ‘our mindsets are a product of our histories’ (Reference Gupta and GovindarajanGupta & Govindarajan, 2002, p. 116). They remain relatively stable and consistent because they are embedded within people’s attitude and belief system. This differs from a stance which is more situational and can change from topic to topic. Therefore, under the influence of our mindsets, our writing habits or patterns can be sticky and hard to change because ‘even if one chooses to act differently, a habit may easily take over’ (Reference Verplanken and OrbellVerplanken & Orbell, 2019, p. 76). However, the relative stability of a mindset does not imply stagnation. Our mindsets also ‘evolve through an iterative process’ (Reference Gupta and GovindarajanGupta & Govindarajan, 2002, p. 116). Consequently, our linguistic features should also be relatively stable but capable of gradual evolution.

This Element adopts mindset as its central perspective due to its role as a specific way of thinking, characterised by a psychological predisposition to certain thought patterns and behaviours. This unique cognitive framework is hypothesised to serve as the psychological bedrock underlying an individual’s implicit thought processes and explicit linguistic patterns or idiolect.

Idiolect and Mindset

A person’s idiolect is a linguistic manifestation of their mindset, which is postulated to underlie and drive these unique linguistic patterns. Since mindset refers to the established set of attitudes held by someone, it is a predisposition to think or feel in a certain way. The way individuals express themselves through their idiolect can reflect their mindset. For instance, an optimistic person may use certain phrases, expressions, or types of syntax that convey positivity, which would be evident in their idiolect. Conversely, a pessimistic or critical mindset might influence a person to use more negative language or critical phrasings. Mindset influences perception and thought patterns, which in turn affect communication and language use. Someone with a growth mindset, who believes they can improve with effort, might use language that is more open, hopeful, or focused on progress and learning. In contrast, someone with a fixed mindset might use more absolute or limiting language.

Idiolects can also provide social cues about a person’s mindset. For instance, the consistent use of certain phrases or styles of speaking can signal a mindset even if it’s not explicitly stated. In addition, the relationship is dynamic, meaning changes in a person’s mindset over time might reflect changes in their idiolect as well.

In linguistic studies, analysing idiolects can provide insights into a person’s mindset, though it’s a complex endeavor requiring consideration of various factors beyond just the words used. In psychology and sociolinguistics, understanding this relationship might involve looking at how language use both influences and is influenced by cognitive processes, social identity, group memberships, and personal aspirations.

In summary, while idiolects are a manifestation of individual linguistic choices and styles, they are influenced by and reflective of a person’s mindset, though this relationship is multifaceted and influenced by several external and internal factors. This interplay can be a rich area of study, particularly in fields interested in the intersections of language, psychology, and social identity. The exploration of the interplay between authors’ patterned idiolects and mindset is a key aspiration of this Element.

Mindset, Gender, and Age

The impact of mindset on both gender and age has been explored as well. Several sociocultural studies (Reference Degol, Wang, Zhang and AllertonDegol et al., 2018; Reference Lee, Lee, Song and BongLee et al., 2021; Reference Sistek-ChandlerSistek-Chandler, 2019) have highlighted a trend where, in certain contexts, females are often less motivated than males in academic or career pursuits, partly due to a prevailing gender-stereotyped mindset. These studies suggest that challenging and transforming such a stereotyped mindset could be instrumental in fostering an environment that encourages and enhances females’ achievements. Some biological studies have found correlations between sex hormones, neural structures, and differences in emotional expression between males and females (Reference Berenbaum and BeltzBerenbaum & Beltz, 2016; Reference Einstein, Downar and KennedyEinstein et al., 2013; Reference PennebakerPennebaker, 2011a; Reference PennebakerPennebaker, 2011b). An experiment involving transgender men who had undergone successful hormone treatment showed changes in their ways of emotion processing (Reference Kiyar, Kubre and ColletKiyar et al., 2022).

Besides, some research suggests that women experience and report more negative emotions because they are capable of perceiving and responding more intensely to unpleasant events than men do (Reference Berenbaum and BeltzBerenbaum & Beltz, 2016; Reference Bianchin and AngrilliBianchin & Angrilli, 2012; Reference Fiorentini, Latu, Schmidt Mast and KaiserFiorentini, 2013; Reference Kring and GordonKring & Gordon, 1998; Reference Latu, Mast and KaiserLatu et al., 2013; Reference Ruberg and SteenberghRuberg & Steenbergh, 2011), making them more susceptible and vulnerable to depression.

Lastly, age-related differences in language use may also be influenced by hormones (Reference Shakouri, Mellati and SheikhyShakouri et al., 2015). However, this aspect has received considerably less attention than gender differences.

Longitudinal Study of Sentiment Scores in Four Authors

Reference GrantGrant (2013) and Reference French and HarrisonFrench and Harrison (2007), emphasised the importance and essence of within-author consistency and between-author distinctiveness of observable features in authorship or speaker comparison. Consequently, as sentiment features are a relatively recent addition to authorship features, the profiling analysis commenced with a longitudinal testing of the consistency of sentiment scores in four authors’ texts. This testing aimed to scrutinise their scores closely. The four authors were selected for two primary reasons. Firstly, their offered texts cover a significant period, providing valuable insight into historical events or changes. This is necessary and essential for testing the within-author consistency of a feature across varied timeframes. Secondly, the four authors were both available and willing to discuss some personal experiences, aiding the scrutiny of significant changes in their sentiment scores.

Specifically, the selected four authors comprise two younger females, one older female, and one older male. The two younger females’ texts belong to type I data, encompassing Diaries, Blogs, and WeChat Moments. The texts from the older female author and the old male author form part of type II data, primarily Online Chats and instant messaging.

The two younger authors’ texts were initially segmented based on their different education stages. These segments were further divided into smaller units, providing sufficient details for scrutinising the consistency of sentiment scores across different stages. The two older authors’ texts were initially segmented by year and broken down into smaller units. The texts written by the same author at different stages or in different years were treated as independent groups. This process aimed to test the stability of sentiment scores across various timeframes and years, rather than testing sentiment scores among independent individuals.

1) Sentiment score consistency in the texts of Author 1: a younger female Overall, the ANOVA test showed a significant difference among the sentiment scores at the p < .05 level across the three different stages of education [F(2, 90) = 4.07, p = 0.02].

However, the sentiment scores remained relatively stable and consistent during the high school and college stages. Specifically, the post hoc comparisons using the Least Significant Difference (LSD) test revealed that, firstly, there was no significant difference between the mean sentiment scores of the texts written in high school (M = 41.68, SD = 165.86) and those written in college (M = 37.06, SD = 128.30). In other words, the sentiment scores remained relatively stable and consistent in the first two educational stages from 2014 to 25 June 2019, namely, high school and college. Secondly, the mean sentiment scores varied significantly from college to the postgraduate stage (M = 138.53, SD = 97.24) from 8 November 2019 to 10 September 2020. The results are displayed in Figure 2.

Figure 2 Stability of sentiment scores of Author 1

Further scrutiny of this tumultuous postgraduate stage was undertaken by examining the stability of the sentiment scores within this particular stage. Therefore, the scores of this stage were further divided into three groups for comparison. The ANOVA test demonstrated that there was no significant difference among the scores at the p < .05 level among the three groups [F(2, 16) = 0.44, p = 0.65]. Specifically, the post hoc comparisons using the LSD test revealed no significant difference between the sentiment scores of any two groups (M = 114.91, SD = 88.14; M = 133.20, SD = 133.72; M = 168.37, SD = 94.74). Therefore, it was reasonable to conclude that the sentiment scores within this upheaval postgraduate stage were consistent.

Until now, both stability and fluctuation of sentiment scores have been observed among the different stages. Whether this could be termed ‘relative consistency’ (Reference GrantGrant, 2013) is still uncertain. A clearer picture may emerge if compared to an established and relatively stable feature. Hence, average sentence length was chosen as the reference feature as it is well recognised as a ‘relevant authorship attribute’ (Reference LehtonenLehtonen, 2015, p. 817; Reference Rangel, Rosso and ChugurRangel et al., 2014; Reference Rangel, Celli, Rosso, Potthast, Stein and DaelemansRangel et al., 2015; Reference Zangerle, Mayerl, Specht, Potthast and SteinZangerle et al., 2020)

In particular, the same segments of Author 1’s texts that were used to calculate sentiment scores were also used to determine the average sentence lengths and to conduct the ANOVA test for stability. Overall, the ANOVA test revealed no significant difference among the average sentence lengths at the p < .05 level across the three different educational stages of Author 1 [F(2, 90) = 2.86, p = 0.06].

However, the average sentence lengths fluctuated significantly between high school and college. Specifically, the post hoc comparisons using the LSD test revealed a significant difference between the mean average sentence length of texts written in high school (M = 32.60, SD = 18.09) and those in college (M = 25.98, SD = 9.53) from 2014 to 25 June 2019. However, the mean average sentence length did not vary significantly when transitioning from college to the postgraduate stage (M = 25.00, SD = 10.71). Thus, the average sentence lengths were relatively stable and consistent in the final two educational stages from 8 November 2019 to 10 September 2020. Therefore, alongside relative stability, fluctuations existed in the average sentence lengths of Author 1 throughout the three stages. The results are displayed in Figure 3.

Figure 3 Stability of sentence length of Author 1

Given the relatively consistent performance of the average sentence length, which comprises both stable and fluctuant stages, it appears that the ‘relative consistency’ (Reference GrantGrant, 2013, p. 480) of the sentiment score in Author 1’s texts could be roughly established.

A Closer Examination of the Different Stages of Author 1

Author 1 demonstrated much lower but stable sentiment scores (ranging from 37 to 42) during high school and college (2014–2019), and much higher sentiment scores (139) during her postgraduate period (2019–2022). After a thorough interview and detailed exploration of her longitudinal texts, it was found that she was highly sentimental during her high school years, evidenced by numerous sentimental words, critical ideas, and doubts in her texts during this period. She expressed feelings of confusion and interest in profound topics and frequently discussed her fantasies. Additionally, she expressed dissatisfaction with both her high school and the university she subsequently attended, particularly after her peers entered more prestigious institutions. This dissatisfaction spurred her determination to pursue a master’s degree at a more prestigious university, a goal she eventually achieved. This accomplishment likely explains why her sentiment scores surged (139), and she became optimistic during her postgraduate years. During those years, her writings were filled with excitement, and she was enthusiastic about attending various campus lectures and acquiring new ideas.

It is important to note that a more rigorous ‘relative consistency’ could only be established with more cases involved. Thus, the longitudinal texts of three more authors were further tested and analysed.

2) Sentiment score consistency in the texts of Author 2: a younger female Overall, the ANOVA test revealed that there was a significant difference among the sentiment scores at the p <.05 level across the three different stages of education [F(2, 142) = 7.22, p = 0.001].

However, the sentiment scores remained relatively stable and consistent during the college and postgraduate stages. Specifically, the post hoc comparisons using the LSD test indicated that, firstly, there was a significant difference between the mean sentiment scores for the texts written during the pre-college stage (encompassing both middle and high school periods) (M = 162.26, SD = 144.57), those written during college (M = 90.00, SD = 117.66), and the postgraduate stage (M = 27.03, SD = 246.75) spanning from 2012 to 2015. Secondly, there was no significant difference between the mean sentiment scores of the texts written in college (M = 90.00, SD = 117.66) and those in the postgraduate stage (M = 27.03, SD = 246.75). This implies that the sentiment scores were relatively stable and consistent during the second and third educational stages from 2016 to 2022. The results are displayed in Figure 4.

Figure 4 Stability of sentiment scores of Author 2

In a similar vein, in order to assess the stability of the sentiment scores within the texts of the single upheaval pre-college stage, the scores were further divided into three groups to conduct the comparison. The ANOVA test showed that there was no significant difference among the scores at the p < .05 level across the three groups [F(2, 53) = 1.057, p = 0.36]. Specifically, the post hoc comparisons using the LSD test indicated that there was no significant difference between the sentiment scores of any two groups (M = 167.84, SD = 131.01; M = 195.17, SD = 180.85; M = 127.61, SD = 116.44). Therefore, it was reasonable to conclude that the sentiment scores within the upheaval precollege stage were consistent.

The performance of the average sentence length in Author 2’s texts was also investigated. Overall, the ANOVA test showed that there was a significant difference among the average sentence lengths at the p < .05 level across the three different stages of Author 2’s education [F(2, 141) = 2.86, p < 0.001].

Specifically, the post hoc comparisons using the LSD test showed that: firstly, there was no significant difference between the mean average sentence length of texts written in the pre-college stage (M = 73.53, SD = 40.84) and those in college (M = 80.35, SD = 49.69). This indicates that the average sentence lengths were relatively stable and consistent in the first two educational stages from 2012 to 2015, namely, middle school, high school, and college. Secondly, there was a significant variation in the mean average sentence lengths when transitioning from college to the postgraduate stage (M = 23.23, SD = 13.17) from 2016 to 2022. The results are displayed in Figure 5.

Figure 5 Stability of sentence length of Author 2

A Closer Examination of the Different Stages of Author 2

In contrast to Author 1, Author 2 displayed much higher starting scores (162) in high school (2012–2016). However, the scores steadily decreased (from 90 to 27) during the stages of college and postgraduate studies (2016–2022). Based on the further interview and detailed exploration of her longitudinal texts, it was found that she was very sentimental when she was in high school.

Author 2 was one of the top high school students and she had great expectations for her future. The university she attended was not her first choice but her backup option. Nevertheless, she clarified that this was not the main reason for her decrease in positive emotions. The sudden suicide of a close relative had a profound and devastating impact on her emotional wellbeing. She described the event as the most shocking and distressing of her life. She struggled to move on from the incident, but was unsuccessful.

To make things worse, she contracted tuberculosis (TB) during university and only recovered after a year-long break post-graduation. TB has been found to be closely linked with stress and negative emotions, and could further reinforce these feelings (Reference Nascimento and BianchiNascimento & Bianchi, 2021; Reference Febi, Manu, Mohapatra, Praharaj and GuddattuFebi et al., 2021). Therefore, it is understandable that her sentiment scores continued to drop.

The following is the demonstration of the sentiment score consistency for the third case: an older female.

3) Sentiment score consistency in the texts of Author 3: an older female Overall, there was a significant difference among the sentiment scores at the p < .05 level across the eight different years of Author 3’s texts from November 2015 to October 2022 [F(7, 369) = 2.65, p = 0.01].

However, the sentiment scores of Author 3 remained stable except for two specific years which deviated from the norm. Specifically, the ANOVA post hoc comparisons using the LSD test indicated that: the years 2015 (M = –30.63, SD = 159.61) and 2017 (M = 32.67, SD = 67.89) were the two deviations from the main body of the mean sentiment scores of the texts written in the remaining six years. Upon removal of these two years, the ANOVA demonstrated that there was no significant difference among the sentiment scores at the p < .05 level across the remaining six different years [F(5, 300) = 1.96, p = 0.09]. The results are displayed in Figure 6.

Figure 6 Stability of sentiment scores of Author 3

Therefore, it is reasonable to infer that the sentiment scores were relatively consistent and stable across most of the years. The fluctuations occurred only during times of significant emotional turmoil. As Author 3, I can confidently identify the causes of these emotional upheavals in 2015 and 2017 due to their profound impact.

A Closer Examination of the Years 2015 and 2017 for Author 3

Overall, the year 2015 was very challenging, while 2017 brought promise for Author 3. Specifically, at the very beginning of 2015, I became a full-time university teacher after earning my PhD in forensic linguistics. Unfortunately, frustration ensued when I realised my research interest was not endorsed nor supported as it was not deemed a legitimate discipline. I was even encouraged to change my research focus, leading to my confusion and despair. In addition, I lost one of my cats abruptly in November that year, a trauma that had a lasting impact on my life and strained family relationships. Reflections of such emotional strain were evident in my texts, marking a record low in my emotional history.

However, by the end of 2016, I had transitioned to a different university where I could continue my research. This period marked a record high in my emotional history.

Unfortunately, it was impossible to calculate average sentence lengths from my texts as they consisted of online instant chats. In these chats, sentences were often sent individually without punctuation, closely mirroring spoken dialogue. This phenomenon, where exchanges are made sentence by sentence rather than paragraph by paragraph, is common in online instant chats, rendering punctuation unnecessary.

4) Sentiment score consistency in the texts of Author 4: an older male For Author 4, overall, there was a significant difference among the sentiment scores at the p < .05 level across the seven different years from January 2016 to October 2022 [F(6, 223) = 2.71, p = 0.02] (Figure 7).

Figure 7 Stability of sentiment scores of Author 4

However, the sentiment scores for Author 4 remained stable except for one specific year where a deviation from the norm was observed. Specifically, the ANOVA post hoc comparisons using the LSD test indicated that the sentiment score for the year 2018 (M = 16.03, SD = 101.81) was the only outlier which detached from the norm of the mean sentiment scores of the texts written in the other six years. When this year was removed, the ANOVA demonstrated no significant difference among the sentiment scores at the p < .05 level across the remaining six years [F(5, 194) = 1.62, p = 0.16]. The results are displayed in Figure 7.

Therefore, it is reasonable to state that the sentiment scores for Author 4 were relatively consistent and stable in most years, with significant fluctuations arising only amidst intense emotional disruption and turbulence. Based on a subsequent interview with Author 4, it was revealed that a business crisis was the primary cause of the emotional turmoil in 2018.

Author 4’s texts were also online instant chats, which precluded the calculation of average sentence lengths.

Summary

In summary, the in-depth analysis of the texts from the four cases supports the argument that the sentiment score is a relatively consistent and stable language feature. The sentiment scores of the four authors showed similar patterns and trends to a well-established consistent feature, namely average sentence length. However, both features demonstrated different patterns across authors, with some exhibiting more consistency and others more variation.

Discussion

Based on the results from the four authors, it is reasonable to believe that the sentiment scores are relatively consistent within authors. However, they are also vulnerable and sensitive to significant emotional changes and disruptions over time. Additionally, the emotional reflections in sentiment scores vary individually. Some authors (Author 1) showed dramatic changes, while others displayed more modest changes (Authors 2, 3, and 4).

Most importantly, the relative consistency of the sentiment scores may be attributed to our mindsets, which are relatively stable as ‘our mindsets are a product of our histories and evolve through an iterative process’ (Reference Gupta and GovindarajanGupta & Govindarajan, 2002, p. 116). Our past histories cannot be overturned overnight, thus, our current mindsets should be relatively stable. Furthermore, our mindset is defined as a cognitive filter through which we view the world (Reference Earley, Murnieks, Mosakowski, Javidan, Steers and HittEarley et al., 2007; Reference Gupta and GovindarajanGupta & Govindarajan, 2002; Reference Sistek-ChandlerSistek-Chandler, 2019). The filtered outputs, including our emotional and sentiment expressions, should also be relatively stable if this filter is relatively stable.

However, the relative stability of the mindset does not imply stagnation. Our mindsets ‘evolve through an iterative process’ (Reference Gupta and GovindarajanGupta & Govindarajan, 2002, p. 116). Such an iterative evolution process would introduce new changes to the relatively fixed mindsets. Specifically, according to Reference Gupta and GovindarajanGupta and Govindarajan (2002), when new experiences and information arise, our mindsets must decide if the new information is consistent with our current mindsets or inconsistent with them. In the former situation, our current mindsets would be reinforced.

In the latter situation, we would experience resistance within ourselves (Reference LopezLopez, 2020). The final outcome would be either ignoring the new information or changing our current mindsets. Therefore, our sentiment scores vary over time primarily due to the introduction of inconsistent new information and the subsequent changes in mindsets (Reference Gupta and GovindarajanGupta & Govindarajan, 2002). Such a change could result in either a positive or a negative emotional shift, which would accordingly cause a rise or fall in sentiment scores. In addition, as each individual has unique life experiences and encounters unique inconsistent new information, each of us has a unique mindset, which results in unique performance and variations in sentiment scores.

In summary, the stability or within-author consistency of sentiment scores was established in the tests presented earlier. Some studies have already used sentiment analysis to predict public mood and conduct opinion mining based on the polarity of emotions (positive or negative) (Albahli, 2022; Reference Birjali, Kasri and Beni-HssaneBirjali et al., 2021; Reference Neogi, Garg, Mishra and DwivediNeogi et al., 2021; Reference Ruz, Henríquez and MascareñoRuz et al., 2022). Reference Martins, Almeida, Henriques and NovaisMartins et al. (2021) even identified authors through polarity of emotions (positive or negative) based on the presence or absence of emotions such as anger, joy, and surprise, along with consideration of authors’ categories (politicians and non-politicians). The study treated the combination of these sentiment features as part of an author’s writing style even though their main purpose was primarily commercial rather than forensic. However, for the present study, it is premature to assert that sentiment score or sentiment expression is a relevant authorship style until the distinctiveness of the sentiment scores between authors is tested. Therefore, the following sections will focus on testing the distinctiveness of the sentiment scores between authors.

1) Distinctiveness of sentiment scores between genders In this part of the experiment, two separate tests were conducted between female and male authors in private and public texts, respectively. The dichotomy was implemented due to the likelihood of individuals behaving differently when they write or speak privately (e.g., in a diary or small friend group) as opposed to when they do so publicly.

Gender Difference in Private Texts

The section began with a test of the distinctiveness of the sentiment score in private texts, including type I and type II texts for female and male authors.

Specifically, the tested texts comprised 24 texts from female authors ranging in age from about 15 to 72 years old and 16 texts from male authors ranging in age from about 17 to 72 years old. Each author’s body of work contained at least one type of text (type I or/and type II). For the 24 female-authored texts, the count of Chinese characters ranged from 4,421 to 13,306, spanning a timeframe from one to 12 years. For the 16 male-authored texts, the Chinese characters ranged from 4,615 to 21,391, spanning a timeframe from one to 13 years.

Overall, there was a noticeable difference in the representation of gender in private texts. Specifically, the standardised sentiment scores revealed that the 16 male texts (M = 94.02, SD = 54.03), compared to the 24 female texts (M = 37.08, SD = 73.53), demonstrated significantly higher and more positive sentiment scores, t(38) = –2.65, p = .012. The results of the sentiment scores for the females and males in the private texts are presented in Figure 8.

Figure 8 Gender difference in private texts

Subsequent to this, there was a test for differences in gender representation in public texts.

Gender Difference in Public Texts

The second test of gender differences focused on assessing the distinctiveness of the sentiment scores in public (type III) texts from both male and female authors. These texts consisted of public articles published on WeChat and available to all subscribers.

Specifically, the tested 15 texts came from female authors aged from 22 to 44 years old and 17 texts from male authors aged from 24 to 52 years old. The 15 female-authored texts contained between 4,318 and 4,361,292 Chinese characters and spanned from four to eight years. The 17 male-authored texts contained between 6,749 and 3,371,270 Chinese characters, spanning from two months to eight years.

Overall, based on the standardised sentiment scores, there was no significant difference between genders in the sentiment scores in public texts, t(30) = .606, p = .549, although the 17 male texts (M = 102.16, SD = 52.50) demonstrated slightly lower sentiment scores than the 15 female texts (M = 114.27, SD = 60. 71). The results of these sentiment scores for the female and male authors in the public texts are shown in Figure 9.

Figure 9 Gender difference in public texts

Based on the preceding results, it was observed that the sentiment scores in the female-authored texts varied significantly from 37.08 in private texts to 114.42 in public texts. However, the sentiment scores in the male-authored texts showed only a minor fluctuation from 94.02 in private texts to 101.92 in public texts. These variations warrant further investigation into sentiment scores between different text genres within a controlled gender group. As such, the following are the tests of sentiment score distinctiveness with respect to different text genres.

2) Distinctiveness of sentiment scores between different text genres In this section, two different tests were conducted to compare sentiment scores between different text types, specifically private and public texts, within controlled gender groups.

Genre Difference in Female Authors

The analysis involved 24 private texts and 16 public texts authored by females. Overall, based on the standardised sentiment scores, the 24 private texts (M = 37.08, SD = 73.53) were compared with the 16 public texts (M = 114.42, SD = 57.76). The results showed significantly lower and more negative sentiment scores in the private texts written by females, t(38) = –3.537, p = .001. The results of the sentiment scores for private and public texts authored by females are demonstrated in Figure 10.

Figure 10 Genre difference in female authors

Following this, the difference between the private and public texts authored by males was explored.

Genre Difference in Male Authors

The data set comprised 16 private texts and 16 public texts from male authors. Overall, based on the standardised sentiment scores, the 16 private texts (M = 94.02, SD = 54.03) were compared with the 16 public texts (M = 101.92, SD = 55.65). The results showed no significant difference in sentiment scores across genres of texts authored by males, t(30) = –.408, p = .686, despite slightly lower sentiment scores in the private texts than the public texts. The results of the sentiment scores for the private and public texts authored by males are demonstrated in Figure 11.

Figure 11 Genre difference in male authors

To summarise, the results indicated that the sentiment scores demonstrated distinctiveness between private and public texts only for female authors.

3) Distinctiveness of sentiment scores between different age groups Similarly, two tests were conducted to compare the sentiment scores between younger and older author groups, specifically those born around 2000 and 1980, in private texts and public texts, respectively.

Age Difference in Private Texts

This section began with testing the distinctiveness of the sentiment scores between different age groups in private type I and type II texts.

The data set included 42 texts from the younger group and 30 texts from the older group. Each author’s texts contained at least one type of private text (type I or/and type II). Overall, based on the standardised sentiment scores, there was no significant difference in sentiment scores between different age groups, t(70) = –1.842, p = .070, despite that the younger group (M = 55.72, SD = 90.51) showing markedly lower sentiment scores than the older group (M = 92.10, SD = 69.90). The results of the sentiment scores for the younger and older authors in private texts are shown in Figure 12.

Figure 12 Age difference in private texts

Following this was the test of sentiment scores between younger and older authors in public texts.

Age Difference in Public Texts

The second test focused on the distinctiveness of the sentiment scores across different age groups in public type III texts.

Specifically, the analysis included 14 texts from younger authors and 18 texts from older authors. The results demonstrated that there was no significant difference in sentiment scores between different age groups, t(30) = 0.608, p = .548, despite the younger group (M = 114.73, SD = 63.76) displaying slightly lower sentiment scores than the older group (M = 102.48, SD = 50.20). The results of the sentiment scores for the younger and older authors in public texts are shown in Figure 13.

Figure 13 Age difference in public texts

In summary, based on the preceding results, the sentiment scores of public texts were notably higher than those of private texts for both younger and older groups. A separate exploration of sentiment scores between different text genres was conducted for both younger and older authors to provide a clearer picture,

Genre Difference in Younger Group

This test involved 42 private and 14 public texts written by younger authors. Based on the standardised sentiment scores, the 42 private texts (M = 114.73, SD = 63.76) compared with the 14 public texts (M = 55.72, SD = 90.51) demonstrated significantly lower and more negative sentiment scores, t(54) = – 2.25, p = .028. The results of the sentiment scores for the private and public texts written by younger authors are shown in Figure 14.

Figure 14 Genre difference in younger group

Genre difference in older group

Similarly, this test involved 30 private texts and 18 public texts written by older authors. According to the standardised sentiment scores, there was no significant difference in sentiment scores between private texts and public texts in the older group, t(46) = –0.597, p = .554. The 30 private texts (M = 92.10, SD = 69.90) displayed slightly lower sentiment scores than the 18 public texts (M = 102.48, SD = 50.20). The results of the sentiment scores for the private and public texts written by older authors are displayed in Figure 15.

Figure 15 Genre difference in older group

In summary, based on the preceding results, the sentiment scores showed distinctiveness between private and public texts only for younger authors.

4) Distinctiveness of sentiment scores between different education groups As detailed in the research design, the educational groups could not be fully established due to practical constraints. Thus, in the private texts, only one comparative test was conducted between university-educated and non-university-educated authors.

This test included 57 ‘educated’ texts (two extreme outliers were deleted) and 10 ‘uneducated’ texts. Overall, according to the standardised sentiment scores, there was no significant difference in the sentiment scores between the ‘educated’ and ‘uneducated’ texts, t(65) = 0.566, p = .573.

The 57 ‘educated’ private texts (M = 73.38, SD = 81.30) showed slightly higher sentiment scores than the 10 ‘uneducated’ private texts (M = 58.30, SD = 49.29). The results of the sentiment scores of the ‘educated’ and ‘uneducated’ private texts are displayed in Figure 16.

Figure 16 Education difference in private texts

Summary

In summary, the preceding results reveal significant distinctiveness of the sentiment scores between female and male authors in private texts, and between private and public texts authored by females.

As for the younger and older age groups, even though the distinctiveness of the sentiment score was not statistically significant, the pattern and the interaction between age groups and text genres/types were striking.

Interestingly, a clear trend was observed in both private and public writings. The younger authors (SD = 90.51 in private texts and SD = 63.76 in public texts) and the female authors (SD = 73.53 in private texts and SD = 60.71 in public texts) tend to have higher standard deviations in their sentiment scores than the older (SD = 69.90 in private texts and SD = 50.20 in public texts) and the male (SD = 54.03 in private texts and SD = 52.50 in public texts) ones, respectively. This result may hint at mood swings or emotional instability in younger and female authors, which will be discussed further later in the study.

However, the sentiment scores were not significantly distinctive between educated and uneducated authors. Moreover, the preceding results highlighted a complex interplay between the distinctiveness of the sentiment scores, gender, age, and text type, suggesting further investigation is needed for a deeper understanding.

5) Testing the distinctiveness of sentiment score in interaction Based on the preceding tests, this section focuses on exploring the interactions among the distinctiveness of the sentiment scores, gender, age, and text type. More data were collected in this stage for this exploration to ensure representativeness and lack of bias.

Specifically, a factorial three-way ANOVA was conducted to compare the main effects of genre, gender, and age (independent variables) as well as their interaction effects on the sentiment scores (dependent variable). This three-way ANOVA analysis had three levels of genre (Diary Blog Moments, Online Chats, WeChat Article), two levels of gender (female, male), and two levels of age (younger, older). Specifically, in this test, the genre types were investigated in three more-specific categories than the previous ones: the private and public texts. As mentioned earlier, Diary, Blog, Moments, and Online Chats are considered private texts, and WeChat articles are public texts.

The results showed that there was a significant main effect of gender, F(1, 98) = 4.87, p < .05, and a significant interaction of gender and genre on the distinctiveness of the sentiment scores, F(2, 98) = 4.87, p < .05.

In particular, the main effect of gender [F(1, 98) = 4.87, p = 0.03] yielded an effect size of 0.047, indicating that gender accounted for 4.7% of the variance in the sentiment scores. Secondly, the interaction of gender and genre was significant [F(2, 98) = 3.47, p = 0.035], indicating a combined effect for gender and genre on the distinctiveness of the sentiment scores. This interaction yielded an effect size of 0.066, indicating that the combined effect of gender and genre explained 6.6% of the variance in the sentiment scores. According to Reference LakensLakens (2013), η² = 0.01 indicates a small effect, η² = 0.06 a medium effect, and η² = 0.14 a large effect. Therefore, the effect size of 4.7% is nearly a medium effect, and the effect size of 6.6% is a medium effect.

These results suggest that gender and genre contribute significantly to the distinctiveness of the sentiment scores both individually and combined. In order to better understand these results, the corresponding effects of gender and genre in different age groups are illustrated in Figures 17, 18, 19, and 20.

Figure 17 Genre difference in younger females and males

Figure 18 Genre difference in older females and males

Figure 19 Gender difference in younger authors’ three genres

Figure 20 Gender difference in older authors’ three genres

In Figure 17, the younger male authors exhibit higher sentiment scores than the younger female authors in both Online Chats and Diary Blog Moments texts, which are private texts. However, when it comes to WeChat articles, namely, the public texts, sentiment scores for male authors are slightly lower, although the difference (134 vs. 90) is not statistically significant. Interestingly, the younger female authors display the highest sentiment scores in these public texts than the younger male ones.

In Figure 18, the older male authors consistently demonstrate higher sentiment scores than their older female counterparts in all three text types. However, the sentiment score difference between the older females and older males (92 vs. 109) in the WeChat articles (i.e., public texts) is not statistically significant, in contrast to the significant gender difference observed in the private texts.

Besides, similar to the case of younger female authors in Figure 17, the older female authors also displayed the highest sentiment scores in the public texts.

Taken together, Figures 17 and 18 reveal two conspicuous patterns in the distinctiveness of the sentiment scores regarding gender and genre: first, male authors generally exhibit higher sentiment scores than female authors. Second, female authors tend to have the highest sentiment scores in public texts compared with private texts.

In addition, Figures 19 and 20 illustrate the distinctiveness of the sentiment scores between different texts, providing a different perspective on the interplay between gender and genre.

In Figure 19, the younger male and younger female authors exhibit similar sentiment scores in both public WeChat articles (type III texts) and private Diary Blog Moments texts (type I texts). However, significant differences emerge in Online Chat texts: the younger female authors demonstrate notably the lowest sentiment scores across all three text types, whereas the younger male authors display markedly the highest sentiment scores across all three text types.

Figure 20 shows that the older male authors consistently demonstrate higher sentiment scores than the older female authors across all three text types, particularly in the private texts, namely Online Chat texts and Diary Blog Moments texts.

In summary, these four figures present two conspicuous patterns concerning the distinctiveness of the sentiment scores in relation to gender and text type: firstly, female authors display the highest sentiment scores in public texts among the three text types. Secondly, in most instances, male authors exhibit higher sentiment scores than female authors.

Discussion

In the interaction test, both gender and the interaction of gender and genre emerged as significant factors for the distinctiveness of the sentiment scores.

Regarding gender, similar to the pattern in the separate test, the female authors demonstrated statistically significantly lower sentiment scores than the male authors. This suggests that male authors are more optimistic than female authors, a finding consistent with the previous studies (Reference Berenbaum and BeltzBerenbaum & Beltz, 2016; Reference Bianchin and AngrilliBianchin & Angrilli, 2012; Reference Fiorentini, Latu, Schmidt Mast and KaiserFiorentini, 2013; Reference Kring and GordonKring & Gordon, 1998; Reference Latu, Mast and KaiserLatu et al., 2013; Reference Ruberg and SteenberghRuberg & Steenbergh, 2011). Females are thought to experience and report more negative emotions due to their “stronger reactions than men in response to unpleasant events” (Reference Fiorentini, Latu, Schmidt Mast and KaiserFiorentini, 2013, p. 24; Reference Kring and GordonKring & Gordon, 1998).

As for the interaction of gender and genre, the results revealed that the female authors displayed significantly lower sentiment scores in private texts than in public ones, implying female authors’ greater pessimism in private texts. However, in public texts, female authors demonstrated no significant difference in sentiment scores compared with their male counterparts, suggesting a similar level of optimism between genders in these texts.

In order to explore the possible reason for this inconsistency, I went back to the public texts, namely, WeChat articles. Upon reviewing the texts, most female authors wrote on neutral daily life topics such as reading, learning skills, fashion, and health. Conversely, male-authored texts tended to focus on hotly debated topics like COVID-19 restrictions, the Ukraine war, and social injustice. Compared with male-authored topics, female-authored topics present fewer opportunities to elicit intense negative emotions. Whether this preference for different types of topics is conscious or unconscious remains unknown.

Furthermore, interestingly and unexpectedly, the results showed that the male authors, especially younger male authors, displayed the significantly highest sentiment scores in Online Chat texts, which suggests that they are more positive in the interactive communication than monologue or soliloquy. This finding aligns with previous studies suggesting men are more dependent on social interaction and its support (Caetano et al., 2013). Specifically, Caetano et al. (2013) found that low social network involvement is closely associated with poor self-rated health in older men and older men’s mental health is associated with emotional support from close relations. They spend less time alone and have more people to count on to get social support (Caetano et al., 2013, p. 12). This may suggest that men are more likely to be dependent on social interaction and the support from which it arises. Therefore, it is much more likely that they are supposed to be more supportive and thus more positive towards others in the social interaction in order to get the same support from others. In addition, societal expectations might encourage men to disguise unpleasant emotions and appear more optimistic to garner more support from others (Reference Fiorentini, Latu, Schmidt Mast and KaiserFiorentini, 2013). Specifically, he believed that males learn to disguise and cover their emotions more than women because they are socialised differently. This suggests that in interactive social communications, they tend to disguise their true selves and mask their unpleasant emotions and try to be more optimistic towards others in order to get more likes and support.

Regarding age, it is worth mentioning that even though statistical results did not indicate a significant difference in sentiment scores between the younger and older authors, there is a clear trend in both private and public writings: the younger authors tend to have higher standard deviations (SD = 90.51 in private texts and SD = 63.76 in public texts) in their sentiment scores than the older authors (SD = 69.90 in private texts and SD = 50.20 in public texts). This suggests greater variation or diversity within the sentiment scores of younger authors, a trend that aligns with the notion that people become more emotionally stable as they age (Reference PennebakerPennebaker, 2011a). This is mainly because older people have had more time to develop and refine their emotional regulation skills (Reference PennebakerPennebaker, 2011a). Over time, they might have learned to better understand and manage their emotions, allowing them to cope more effectively with stressful or challenging situations.

There are also other factors that might contribute to increased emotional stability in older age, such as a greater sense of perspective, a more positive outlook on life (Reference PennebakerPennebaker, 2011a), and increased social support from family and friends (Reference PennebakerPennebaker, 2011a). A more positive outlook in older people could be supported by the results in the context of private texts only. The possible reason is that people tend to disguise less, be more authentic and show their true selves in private context, which is similar to the above-mentioned situation of gender. Additionally, changes in brain chemistry and structure that occur as we age might also play a role in promoting emotional stability.

Notably, the same tendency was observed between the female and male authors. In other words, in both private and public writings, female authors tend to have higher standard deviations (SD = 73.53 in private texts and SD = 60.71 in public texts) in their sentiment scores than male authors (SD = 54.03 in private texts and SD = 52.50 in public texts). This parallel between female and young authors will be further discussed in Section 3.3, in which possible style matching between younger and female authors will be elaborated upon in light of a similar tendency in the use of personal pronouns and graduations.

1) Distinctiveness of keywords in predicting gender The results demonstrate that the logistic regression model for predicting gender proved significant, χ²(3, N = 114) = 27.87, p < .001 for Omnibus tests, and p = .510 for the Hosmer–Lemeshow test. This result indicates that the predictor variables (i.e., personal pronouns, exclamations, and graduations) collectively explained a significant amount of variance in the outcome variable (i.e., gender of text authors). More precisely, the model explains 29.3% (Nagelkerke R Square) of the variance in gender and correctly classified 70.2% of the cases.

In more detail, the results in Figure 21 reveal that firstly, male authors (target) were ten times more likely to avoid using personal pronouns than to use them [OR = 9.828, 95%CI (2.719, 35.522)]. In other words, for male authors, the ratio of not using to using personal pronouns was 9.828. Secondly, male authors were nearly three times more likely to avoid using exclamations than to use them [OR = 2.837, 95%CI (1.065, 7.560)]. In other words, for the male authors, the ratio of not using exclamations to using them was 2.837. Thirdly, male authors were five times more likely to avoid using graduations than to use them [OR = 5.09, 95%CI (1.856, 13.959)]. In other words, for male authors, the ratio of not using graduations to using them was 5.09.

Figure 21 Distinctiveness of keywords in gender

2) Distinctiveness of keywords in predicting age The logistic regression model for predicting age was also significant, χ²(3, N = 114) = 15.68, p = .001 for Omnibus tests, and p = .533 for the Hosmer–Lemeshow test, indicating that the predictor variables (i.e., personal pronouns, exclamations, and graduations) collectively explained a significant amount of variance in the outcome variable (i.e., the age of text authors). Precisely, the model explained 17.2% (Nagelkerke R Square) of the variance in gender and correctly predicted 64.0% of the cases.

A detailed analysis of Figure 22 reveals, firstly, that older authors (target) were nearly five times more likely to avoid using personal pronouns than to use them [OR = 4.673, 95%CI (1.484, 14.715)]. In other words, for older authors, the ratio of not using personal pronouns to using them was 4.673. Secondly, older authors were three times more likely to avoid using graduations than to use them [OR = 3.142, 95%CI (1.297, 7.609)]. In other words, for older authors, the ratio of not using graduations to using them was 3.142. However, there was no significant association between the use of exclamations and the author’s gender (p = .681).

Figure 22 Distinctiveness of keywords in age

From these results, it can be inferred that personal pronouns, exclamations, and graduations are all distinctive in predicting an author’s gender, but only personal pronouns and graduations were distinctive in predicting the author’s age.

However, the identified targets in the two regression tests were only the male authors and the older authors (the larger coded group is identified as the ‘target’). Thus, the specific gender differences between females and males, and age differences between younger and older authors in terms of the distinctive keyword features remain unclear. To address these detailed gender and age differences, two more logistic regression analyses were conducted with the outcome variables and predictor variables interchanged, as the original predictor variables are also binary. This approach facilitated a more thorough investigation of the correlations.

3) The detailed correlation between gender, age, and personal pronoun The logistic regression model demonstrating the gender and age differences in terms of predicting the use of personal pronouns proved significant, with χ²(2, N = 114) = 18.616, p < .001 for Omnibus tests, and p = .816 for the Hosmer–Lemeshow test. The results indicate that the predictor variables (i.e., gender and age) collectively explained a significant amount of variance in the outcome variable (i.e., the use of personal pronouns). The model accounted for 25.4% (Nagelkerke R Square) of the variance in personal pronoun usage and accurately classified 83.3% of the cases.

Figure 23 reveals that firstly, personal pronouns were six times more likely to be used by the female authors than the male authors [OR = 6.387, 95%CI (1.909, 21.372)]. In other words, the female authors were six times more likely to use personal pronouns than the male authors. Secondly, personal pronouns were three times more likely to be used by the younger authors than the older authors [OR = 3.288, 95%CI (1.035, 10.452)]. In other words, younger authors were three times more likely to use personal pronouns than older ones.

Figure 23 Correlation between gender, age, and personal pronoun

4) The detailed correlation between gender, age, and graduation The results from the logistic regression model highlighted significant gender and age differences in predicting the use of graduations, with χ²(2, N = 114) = 13.439, p = .001 for Omnibus tests, and p = .081 for the Hosmer–Lemeshow test. This suggests that the predictor variables (i.e., gender and age) collectively explained a significant portion of the variance in the outcome variable (i.e., the use of graduations). Specifically, the model explained 15.3% (Nagelkerke R Square) of the variance in personal pronoun usage and correctly classified 71.9% of the cases.

As shown in Figure 24, firstly, graduations were nearly three times as likely to be used by the female authors compared with the male authors [OR = 2.907, 95%CI (1.188, 7.115)], suggesting that female authors were three times more likely to use graduations than their male counterparts. Secondly, graduations were almost three times as likely to be used by the younger authors compared with the older authors [OR = 2.59, 95%CI (1.098, 6.111)], indicating that younger authors were three times more likely to use graduations than the older authors. However, there was no significant association between the author’s gender (p = .122) and age (p = .195) differences in relation to the use of exclamations alone.

Figure 24 Correlation between gender, age, and graduation

5) Discussion for gender difference Overall, personal pronouns, exclamations, and graduations, either individually or collectively, effectively predict an author’s gender. More specifically, it is projected that female authors tend to use more personal pronouns, exclamations, and graduations than female authors.

The findings align with previous studies on gender differences in the use of pronouns (Reference PennebakerPennebaker, 2011a; Reference Säily, Nevalainen and SiirtolaSäily et al., 2011; Reference TannenTannen,1991; Reference Tausczik and PennebakerTausczik & Pennebaker, 2010; Reference ZhouZhou, 2007), showing that females are more likely to use more intensifiers (one type of graduations) and personal pronouns than males. They also support the notion that men’s communication style tends to be more informational, while women’s is more interactive and involved, which inevitably leads to more frequent use of personal pronouns (Reference Biber, Conrad and ReppenBiber et al., 1998; Reference PennebakerPennebaker, 2011a; Reference Rayson, Leech and HodgesRayson et al., 1997; Reference Säily, Nevalainen and SiirtolaSäily et al., 2011).

More specifically, females tend to use more social words and pronouns (Reference PennebakerPennebaker, 2011a; Reference Tausczik and PennebakerTausczik & Pennebaker, 2010), and view matters from the perspective of human relationships, demonstrating understanding and empathy by sharing personal experiences (Reference ZhouZhou, 2007). Personal pronouns, such as ‘我(I)’, ‘我们(we)’, and ‘你(you)’, are frequently used when sharing personal experiences and emotions (Reference PennebakerPennebaker, 2011a; Reference Tausczik and PennebakerTausczik & Pennebaker, 2010) because they refer to the speakers or writers themselves and the listeners or readers. Indeed, the use of personal pronouns is often essential for conveying the personal nature of shared experiences, distinguishing them from hypothetical or generalised situations.

In the context of human relationships, personal pronouns enable us to establish connections, express empathy, and build trust with others. This explains why they are frequently used in discussions related to human relationships. Especially the first-person pronoun, such as ‘we’, is frequently used for rapport building (Reference TannenTannen, 1991) in relationships.

Furthermore, showing understanding and sympathy may involve not only personal pronouns, but also exclamations and graduations as they’re closely related to expressing emotions and feelings (Reference ZhouZhou, 2007). Exclamations are words or phrases that express strong emotions or feelings, and are typically punctuated with an exclamation mark (!) to highlight the intensity of the conveyed emotion. Examples of exclamations include ‘哦(o)’, ‘呀(ya)’, ‘啊(a)’, and ‘哈(ha)’. Consistent with the more emotional tendency of females (Reference PennebakerPennebaker, 2011a; Reference Pennebaker and StonePennebaker et al., 2003; Reference Latu, Mast and KaiserLatu et al., 2013; Reference Ruberg and SteenberghRuberg & Steenbergh, 2011), it is natural for female authors to use more exclamations in their writing and communication.

As for graduation, it is an important aspect of appraisal because it allows speakers to convey subtle differences in their social attitudes and beliefs, and to convey their stance towards the subject matter in a more nuanced and sophisticated way (Reference Martin and WhiteMartin & White, 2005). It is a key element in creating evaluative meanings that are contextually appropriate and reflect the speaker’s individual viewpoint (Reference Martin and WhiteMartin & White, 2005). Graduations are closely associated with women’s social interests (Reference PennebakerPennebaker, 2011a; Reference Tausczik and PennebakerTausczik & Pennebaker, 2010) and their more involved and interactive tendencies (Reference Biber, Conrad and ReppenBiber et al., 1998; Reference Rayson, Leech and HodgesRayson et al., 1997; Reference Säily, Nevalainen and SiirtolaSäily et al., 2011; Reference ZhouZhou, 2007). Specifically, a greater use of graduations might suggest the use of intensive adverbs to indicate the degree or intensity of a particular evaluative category (Reference Newman, Groom, Handelman and PennebakerNewman et al., 2008).

Notably, force, one of the two types of graduation (which includes both intensification and quantification), can increase or decrease the ‘volume’ of the involved attitude. Thus, force has an impact on social alignment and solidarity. When the attitude is upscaled (made more forceful), it tends to portray the speakers or writers as strongly involved themselves in the expressed value position. This, in turn, may align the reader or audience more closely with that value position.

In summary, graduations can shape the reader’s perception of the speaker or writer’s attitude and alignment with a particular value position. The more forceful the language, the more strongly the speaker or writer is seen as committed to that position, and the more likely the audience or reader is to align with it. Consistent with the greater social interests (Reference PennebakerPennebaker, 2011a; Reference Tausczik and PennebakerTausczik & Pennebaker, 2010) and more involved or interactive tendency (Reference Biber, Conrad and ReppenBiber et al., 1998; Reference Rayson, Leech and HodgesRayson et al., 1997; Reference Säily, Nevalainen and SiirtolaSäily et al., 2011) of women, it is not surprising that they are more likely to use graduations than men.

6) Discussion for age difference Compared to the predictive power for gender, personal pronouns, exclamations, and graduations are less strongly associated with an author’s age. Age is a more complex and intriguing social factor than gender because gender is usually stable, while age changes throughout our lives (Reference PennebakerPennebaker, 2011a). In spite of this, based on the results, the age of text authors could still be correctly predicted by some keyword features, especially personal pronouns and graduations.

The results indicate that younger authors (born around and after the year 2000) are more likely to use personal pronouns and graduations than older authors (born around and before the year 1980). The results are consistent with prior findings that younger individuals, such as teenagers, tend to use personal pronouns, such as ‘I’ and ‘you’, more frequently (Reference PennebakerPennebaker, 2011a). Conversely, the older the people become, the more likely they are to use bigger words, prepositions, and articles (Reference PennebakerPennebaker, 2011a). Similarly, the results also support the findings that people use fewer self-references as they age (Reference Pennebaker and StonePennebaker & Stone, 2003), and thus their use of ‘first-personal singular decreased with time’ (Reference Tausczik and PennebakerTausczik & Pennebaker, 2010, p. 36) as people become less self-focused with increasing age (Reference Pennebaker and StonePennebaker & Stone, 2003).

As for graduations or intensifiers, as mentioned previously, in most cases, they are used to convey strong and intensified emotions and feelings. The findings are consistent with the concept that ‘intensifying words are generally used and created by young people’ (Reference BordetBordet, 2015, p. 6), and that younger people tend to use more intensifiers than their older counterparts.

There are a few possible explanations for this age-related disparity. One possibility is that younger people, akin to females, may use language in a more emotional or expressive manner (Reference PennebakerPennebaker, 2011a), and intensifiers serve to articulate these emotions effectively. Additionally, and similar to females, younger people may be more inclined to use intensifiers as a marker of their affiliation with a specific social group or subculture (Reference BordetBordet, 2015). In my analysis, typical observed examples of such intensifier usage include ‘超(extremely)’ and ‘狠(very)’. Notably, the use of ‘狠(very)’ is actually a creative misuse and reinterpretation of ‘很 (very)’, a linguistic style often adopted by young people as a fashionable use of intensifiers.

However, it is essential to acknowledge that individual differences and contextual factors can also play a role in the use of graduations or intensifiers. For instance, some older people may use intensifiers frequently, while some younger people may use them sparingly. Overall, the relationship between the use of graduations and age is complex and multifaceted, and more research is needed to fully understand it.

Check for Basic Assumptions of Discriminant Analysis

Discriminant analysis is a statistical method employed to classify cases or texts into groups guided by a set of predictor variables. The assumptions for discriminant analysis encompass feature reliability, outliers, the number of predictor variables in each attribution model, multivariate normality within groups, and homogeneity of variance-covariance matrices (Reference Tabachnick and FidellTabachnick and Fidell, 2007). However, it is important to note that violating certain assumptions does not necessarily invalidate the discriminant analysis results. On the one hand, discriminant analysis tends to be robust against violations of the normality and the homogeneity of variance-covariance matrices when sample sizes are equal (Reference Tabachnick and FidellTabachnick & Fidell, 2007). In this authorship attribution experiment, each author’s texts were segmented into 15 smaller texts, ensuring an equal sample size for each author.

On the other hand, discriminant analysis is highly sensitive to outliers, and the size of the smallest group must exceed the number of predictor variables. Therefore, basic assumptions, which include feature reliability, outlier, and number of predictor variables were checked and reported. This aligns with the experiment’s primary objective of testing the feasibility of using the distinctive profiling features in attribution tasks through comparison with well-established features.

1) Feature reliability All the lexico-syntactic features were generated using WordSmith Tools 8.0.

The sentiment scores and keyword features were generated using Python, specifically through pysenti 0.1.7, which was introduced earlier. Exclamations, graduations, and personal pronouns were identified and calculated. Exclamation words (including exclamation marks) were derived from a detailed examination of the texts, keyword lists, and the most frequently used Chinese exclamation words (Reference ZhangZhang, 2022). In total, 11 exclamation words were analysed, which include ‘哦(o)’, ‘噢(ao)’, ‘呀(ya)’, ‘啊(a)’, ‘哈(ha)’, ‘哇(wa)’,’吖(ya)’, ‘唉(ai)’, ‘哼(heng)’, ‘啊喂(awe)’, and ‘呵呵(hehe)’. Graduations were derived from an intricate analysis of the texts, keywords list, and English appraisal system (Reference Martin and WhiteMartin & White, 2005). In total, 11 graduations were analysed, which include ‘很(very)’, ‘非常(very)’, ‘还(still)’, ‘有点(a little)’, ‘真(really)’, ‘完全(totally)’, ‘贼(extremely)’, ‘正(exactly)’, ‘超(extremely)’, ‘的确(indeed)’, and ‘狠(very)’. Personal pronouns were derived from the keyword lists. In total, four personal pronouns were analysed, which include ‘我(I/me)’, ‘你(you)’, ‘他(he/him)’, and ‘她(she/her)’.

2) Outlier Each group was subjected to data screening to ensure the absence of extreme outliers.

3) Number of predictor variables In the attribution tests, each author’s texts were segmented into 15 smaller pieces for discriminant analysis. Depending on the two mentioned feature sets, among all the following attribution tests, the maximum feature set included 14 predictor variables/features (when both feature sets were used simultaneously for attribution) and the minimum feature set contained seven predictor variables/features (when each feature set was used separately and independently). Thus, the number of features utilised in each test was less than the sample size of each author texts group (15 texts for each author), which means that the assumption regarding the number of predictor variables is met.

Indicators of the Discriminant Effect of Feature Set

The discriminant effect of a feature set signifies the extent to which the features in the set are able to distinguish between different groups in a classification task. The following are several indicators gauging the discriminant effect of a feature set:

1) Wilks’s lambda and eigenvalue Both Wilks’s lambda and eigenvalue serve as indicators of the significance of the discriminant function of a feature set (Reference Cigic and BugarskiCigic & Bugarski, 2010; Reference ZhangZhang, 2016). Wilks’s lambda, specifically, represents the proportion of the total variance in the discriminant scores that is not accounted for by differences among groups. Therefore, the value of Wilks’ lambda, ranging from 0 to 1, should ideally be as small as possible; smaller values suggest a stronger discriminatory power of the independent or predictor variables. In other words, a lower Wilks’ lambda value indicates that the groups are more distinct and are better separated by the predictor variables or features.

Moreover, to evaluate the significance of the differences between groups in discriminant analysis, the observed value of Wilks’ lambda is expected to be smaller than the critical value, namely 0.05.

Conversely, the eigenvalue indicates the discriminant function’s effectiveness in differentiating groups. Larger eigenvalues denote superior discriminant function. The ‘Kaiser criterion’ is a common method to evaluate eigenvalues, suggesting retention of eigenvalues greater than 1 (Reference MichalosMichalos, 2014).

2) Original accuracy rate and cross-validated accuracy rate Original accuracy refers to the performance of a model on the same dataset it was trained on, while cross-validated accuracy refers to the performance of a model on unseen or new data. Specifically, in the authorship attribution, both types of accuracy rates assess the effectiveness of the feature sets in distinguishing between groups of author texts. Particularly, the cross-validated accuracy rate provides a more truthful reflection of the discriminant function’s power than the original rate (Reference Burns and BurnsBurns & Burns, 2008).

3) Significance evaluation of the accuracy rate The accuracy rate is statistically significant if it surpasses the proportion achieved by chance (Reference ZhangZhang, 2004). The expected criterion is (100%/G)*1.2 or 1.25, with G representing the number of authors (Reference ZhangZhang, 2004, p. 264). For instance, in an 18-author discriminant analysis, if both the original and cross-validated accuracy rate exceed the expected criteria of 41.7% (100%/18*1.25), it would be reasonable to state that the observed accuracy rate is significantly higher than what would be expected by chance.

Data

The authorship attribution tests were conducted on the Diaries, Blogs, and Moments, namely type I texts. These data were selected for two reasons. Firstly, it represents one of the most populous groups in terms of author diversity, thereby increasing the complexity and challenge of the authorship attribution task and enhancing the reliability of the test results. Secondly, this type of data comprises several sub-genres of texts, making the attribution task more complicated and challenging, thereby potentially boosting the reliability and validity of the results. Thirdly, the punctuations in this data type are used normally and conventionally, enabling the calculation of the ‘sentence length’ feature. By contrast, the punctuations in Online Chats (type II texts) are often omitted, especially periods, which makes the calculation of ‘sentence length’ extremely difficult, if not impossible. Therefore, the authorship attribution tests were only conducted on the private type I texts which include Diaries, Blogs, and Moments.

Based on the data and the two feature sets, the authorship attribution tests were carried out from within-group tests to between-group tests based on different gender groups, age groups, and their combined groups. More specifically, nine tests were conducted: younger females group (18 authors), older females group (5 authors), younger males group (6 authors), older males group (6 authors), all females group (23 authors), all males group (12 authors), all younger authors group (24 authors), all older authors group (11 authors), and all author combined group which includes all 35 authors.

Each subsequent attribution test comprises three tables, demonstrating the results of the lexico-syntactic feature set, the sentiment and keyword feature set, and their combined feature set, respectively. More specifically, the results of each feature set include the number of authors, number of predictors, discriminant functions, eigenvalue, Wilks’ lambda, significance of the functions, original accuracy rate, and cross-validated accuracy rate.

Sentiment Scores and Keywords in the Authorship Attribution of Younger Females

In the attribution of the 18 younger females, the sentiment scores and keyword feature set surpassed the performance of the lexico-syntactic feature set. Furthermore, their combined effectiveness also exceeded their separate capabilities.

According to the attribution results of the lexico-syntactic feature set for the younger females (see Table 3), all of the seven eigenvalues indicate weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda show that five of the seven functions are significant (p < .05). Overall, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates were markedly higher than the proportional by-chance accuracy rate of 6.9% (100%/18*1.25).

In the attribution results of the sentiment scores and keyword feature set for the younger females (see Table 4), one of the seven eigenvalues indicates a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda show that five of the seven functions were significant (p < .05). Consequently, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates were considerably larger than the proportional by-chance accuracy rate, which stands at 6.9% (100%/18*1.25).

The results from the combined feature set (see Table 5) indicate that 2 out of the 14 eigenvalues showed strong discriminant functions (eigenvalue > 1). Wilks’s lambda reveals that 7 out of the 14 functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates were much larger than the proportional by-chance accuracy rate, which is 6.9% (100%/18*1.25).

Based on these results, it is clear that in the attribution of the 18 younger female authors, the sentiment and keyword feature set outperformed the lexico-syntactic feature set. In addition, the combined feature set surpassed the performance of the individual feature set, which implied that both feature sets contributed positively to the authorship attribution of the 18 younger female authors. Interestingly, the combined strength of the two feature sets seemed to exceed the sum of the strengths of each individual group in terms of the number of larger eigenvalues.

Sentiment Scores and Keywords in the Authorship Attribution of Older Females

In the case of attributing the five older females, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. The combined strength of these two feature sets also surpassed their individual performances.

According to the attribution results of the lexico-syntactic feature set for the older females (Table 6), one of the four eigenvalues suggested a strong discriminant function (eigenvalue > 1). Wilks’s lambda results revealed that three out of the four functions were significant (p < .05). Subsequently, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates were much larger than the proportional by-chance accuracy rate of 25% (100%/5*1.25).

According to the attribution results of the sentiment and keyword feature set for the older females (Table 7), all of the four eigenvalues suggested weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda showed that two of the four functions were significant (p < .05). Both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates were considerably larger than the proportional by-chance accuracy rate, which is 25% (100%/5*1.25).

The results from the combined feature set (see Table 8) showed that two of the four eigenvalues suggested strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that three of the four functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, exceeding the proportional by-chance accuracy rate of 25% (100%/5*1.25) by a substantial margin.

Based on these results, it is clear that in attributing the five older female authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. In addition, the combined feature set outperformed the individual feature sets, which suggested that both feature sets positively contributed to the attribution of the five older female authors. Furthermore, the combined strength of the two feature sets seemed to surpass the sum of the strengths of each individual group in terms of the number of larger eigenvalues.

Sentiment Scores and Keywords in the Authorship Attribution of Younger Males

In attributing the authorship of the six younger males, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. Furthermore, the combined strength of these two feature sets surpassed their individual performances.

According to the attribution results of the lexico-syntactic feature set for the younger males (see Table 9), two of the five eigenvalues indicated very strong discriminant functions (eigenvalue > 1). The results of Wilks’s lambda revealed that three of the five functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, far surpassing the proportional by-chance accuracy rate of 20.8% (100%/6*1.25).

According to the attribution results of the sentiment and keyword feature set for the younger males (see Table 10), one of the five eigenvalues indicated a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda showed that four of the five functions were significant (p < .05). Accordingly, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly exceeding the proportional by-chance accuracy rate, which is 20.8% (100%/6*1.25).

The results of the combined feature set (Table 11) show that three of the five eigenvalues indicated very strong discriminant functions (eigenvalue > 1). Wilks’s lambda results showed that all of the five functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, greatly exceeding the proportional by-chance accuracy rate of 20.8% (100%/6*1.25).

Based on these results, it is clear that in the attribution of the six younger male authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. In addition, the combined feature set outperformed the individual feature sets, which suggested that both feature sets positively contributed to the attribution of the six younger male authors.

Sentiment Scores and Keywords in the Authorship Attribution of Older Males

For the attribution of the six older males, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. Furthermore, the combined strength of these features also outshone their individual performances.

According to the attribution results of the lexico-syntactic feature set for the older males (see Table 12), two of the five eigenvalues indicated strong discriminant functions (eigenvalue > 1). Additionally, the results of Wilks’s lambda indicated that four of the five functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, exceeding the proportional by-chance accuracy rate of 20.8% (100%/6*1.25).

According to the attribution results of sentiment and keyword feature set for the older males (see Table 13), one of the five eigenvalues suggested a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda indicated that three of the five functions were significant (p < .05). Accordingly, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, exceeding markedly the proportional by-chance accuracy rate of 20.8% (100%/6*1.25).

The results of the combined feature set (see Table 14) show that three of the five eigenvalues indicated very strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that all of the five functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, considerably exceeding the proportional by-chance accuracy rate, which is 20.8% (100%/6*1.25).

Based on the preceding results, it is obvious that in the attribution of the six older male authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. In addition, the combined feature set outperformed the individual feature sets, which suggested that both feature sets were making their positive contributions to the attribution of the six older male authors.

Sentiment Scores and Keywords in the Authorship Attribution of Females

In attributing the authorship of the 23 female authors, the sentiment and keyword feature set outperformed the lexico-syntactic feature set. Furthermore, the combined strength of these two feature sets surpassed their individual strengths.

According to the attribution results of the lexico-syntactic feature set for the females (see Table 15), all of the seven eigenvalues suggested weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda indicated that five of the seven functions were significant (p < .05). Overall, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, exceeding the proportional by-chance accuracy rate, which is 5.4% (100%/23*1.25).

According to the attribution results of sentiment and keyword feature set for the females (Table 16), all of the seven eigenvalues suggested weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda showed that five of the seven functions were significant (p < .05). Overall, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, exceeding the proportional by-chance accuracy rate of 5.4% (100%/23*1.25).

The results of the combined feature set (see Table 17) show that 2 of the 14 eigenvalues indicated strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that 7 of the 14 functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly exceeding the proportional by-chance accuracy rate, which is 5.4% (100%/23*1.25).

Based on these results, it is obvious that in the attribution of the 23 female authors, the sentiment and keyword feature set performed better than the lexico-syntactic feature set. In addition, the combined feature set outperformed the individual feature sets, which suggested that both feature sets were making positive contributions to the attribution of the 23 female authors. In addition, the combined strength of the two feature sets seemed to surpass the sum of the strengths of each individual group in terms of the number of larger eigenvalues.

Sentiment Scores and Keywords in the Authorship Attribution of Males

In attributing the authorship of the 12 male authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. Furthermore, the combined strength of these two feature sets outperformed their individual strengths.

According to the attribution results of the lexico-syntactic feature set for the males (see Table 18), two of the seven eigenvalues indicated strong discriminant functions (eigenvalue > 1). The results of Wilks’s lambda indicated that five of the seven functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly surpassing the proportional by-chance accuracy rate of 10.4% (100%/12*1.25).

According to the attribution results of sentiment and keyword feature set for the males (Table 19), one of the seven eigenvalues suggested a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda showed that five of the seven functions were significant (p < .05). Accordingly, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly exceeding the proportional by-chance accuracy rate of 10.4% (100%/12*1.25).

The results of the combined feature set (see Table 20) show that four of the 11 eigenvalues suggested very strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that eight of the 11 functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, considerably exceeding the proportional by-chance accuracy rate, which stands at 10.4% (100%/12*1.25).

Based on the preceding results, it is clear that in the attribution of the 12 male authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. In addition, the combined feature set outperformed the individual feature sets, which suggested that both feature sets were making their positive contributions to the attribution of the 12 male authors. In addition, the combined strength of the two feature sets seemed to surpass the sum of the strengths of each individual group in terms of the number of larger eigenvalues.

Sentiment Scores and Keywords in the Authorship Attribution of Younger Authors

In the task of attributing the 24 younger authors, the sentiment and keyword feature set outperformed the lexico-syntactic feature set. Further, the combined strength of these two feature sets also surpassed their individual performances.

According to the attribution results of the lexico-syntactic feature set for the younger authors (see Table 21), one of the seven eigenvalues suggested a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda revealed that four of the seven functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates significantly exceeded the proportional by-chance accuracy rate of 5.2% (100%/24*1.25).

According to the attribution results of sentiment and keyword feature set for the younger authors (see Table 22), all of the seven eigenvalues indicated weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda indicated that five of the seven functions were significant (p < .05). Overall, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates significantly exceeded the proportional by-chance accuracy rate of 5.2% (100%/24*1.25).

The results of the combined feature set (see Table 23) showed that 2 of the 14 eigenvalues indicated strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that nine of the 14 functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly exceeding the proportional by-chance accuracy rate of 5.2% (100%/24*1.25).

Based on these results, it is clear that in attributing the 24 younger authors, the sentiment and keyword feature set performed better than the lexico-syntactic feature set. In addition, the combined feature set outperformed the individual feature sets, suggesting that both feature sets positively contributed to the attribution of the 24 younger authors. Furthermore, the combined strength of the two feature sets seemed to surpass the sum of the strengths of each individual group in terms of the number of larger eigenvalues.

Sentiment Scores and Keywords in the Authorship Attribution of Older Authors

In the task of attributing the 11 older authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. Furthermore, the combined strength of these two feature sets also surpassed their individual performances.

According to the attribution results of the lexico-syntactic feature set for the older authors (see Table 24), one of the seven eigenvalues suggested a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda revealed that four of the seven functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates significantly exceeded the proportional by-chance accuracy rate of 11.4% (100%/11*1.25).

According to the attribution results of sentiment and keyword feature set for the older authors (see Table 25), all of the seven eigenvalues suggested weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda showed that four of the seven functions were significant (p < .05). Overall, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates significantly exceed the proportional by-chance accuracy rate of 11.4% (100%/11*1.25).

The results of the combined feature set (see Table 26) showed that two of the ten eigenvalues indicated strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that six of the ten functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly exceeding the proportional by-chance accuracy rate of 11.4% (100%/11*1.25).

Based on these results, it is clear that in attributing the 11 older authors, the lexico-syntactic feature set outperformed the sentiment and keyword feature set. Moreover, the combined feature set outperformed the individual feature sets, suggesting that both feature sets were making their positive contributions to the attribution of the 11 older authors. In addition, the combined strength of the two feature sets seemed to surpass the sum of the strengths of each individual group in terms of the number of larger eigenvalues.

Sentiment Scores and Keywords in the Authorship Attribution of All Authors

In the attribution of all of the 35 authors, the sentiment and keyword feature set outperformed the lexico-syntactic feature set. Furthermore, the combined strength of these two feature sets also surpassed their individual performances.

According to the attribution results of the lexico-syntactic feature set for all of the authors (see Table 27), one of the seven eigenvalues suggested a strong discriminant function (eigenvalue > 1). The results of Wilks’s lambda revealed that five of the seven functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes. These rates significantly exceeded the proportional by-chance accuracy rate of 3.6% (100%/35*1.25).

According to the attribution results of sentiment and keyword feature set for all of the authors (see Table 28), all of the seven eigenvalues suggested weak discriminant functions (eigenvalue < 1). The results of Wilks’s lambda indicated that all of the seven functions were significant (p < .05). Overall, both the original and the cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, exceeding the proportional by-chance accuracy rate of 3.6% (100%/35*1.25).

The results of the combined feature set (see Table 29) showed that 2 of the 14 eigenvalues suggested strong discriminant functions (eigenvalue > 1). Wilks’s lambda results indicated that 10 of the 14 functions were significant (p < .05). Accordingly, both the original and cross-validated accuracy rates provided statistically significant evidence of the model’s ability to predict outcomes, significantly exceeding the proportional by-chance accuracy rate, which is 3.6% (100%/35*1.25).

Based on these results, it is obvious that in the attribution of the 35 authors, the sentiment and keyword feature set outperformed the lexico-syntactic feature set. Moreover, the combined feature set outperformed the individual feature set, suggesting that both feature sets positively contributed to the attribution of the 35 authors. In addition, the combined strength of the two feature sets seemed to surpass the sum of the strengths of each individual group in terms of the number of larger eigenvalues.