Introduction
Adverts targeted and tailored to psychological traits have been shown to be effective in non-political domains (Matz et al., Reference Matz, Kosinski, Nave and Stillwell2017). Given the alleged roles of Cambridge Analytica and AggregateIQ in recent political campaigns, the question of whether psychologically tailored and targeted arguments are effective in political contexts has important social implicationsFootnote 1.
Research has demonstrated that personal and moral values reliably correlate with political decisions in the US and UK (Gerber et al., Reference Gerber, Huber, Doherty, Dowling and Ha2010; Graham et al., Reference Graham, Haidt and Nosek2009; Sibley et al., Reference Sibley, Osborne and Duckitt2012) and so targeting and tailoring arguments in line with these values could be effective. In addition to the research showing the effectiveness of personality-based tailoring in advertising (Hirsh et al., Reference Hirsh, Kang and Bodenhausen2012; Matz et al., Reference Matz, Kosinski, Nave and Stillwell2017), there is some evidence that Moral Foundations Theory (MFT) can be effective in tailoring political arguments (Feinberg & Willer, Reference Feinberg and Willer2013; Reference Feinberg and Willer2015; Voelkel & Feinberg, Reference Voelkel and Feinberg2018), although the combined role of tailoring and targeting based on individual differences in Moral Foundations has not been tested. Moreover, while some longitudinal research has suggested an influence of personality on political decisions over time (Ekstrom & Federico, Reference Ekstrom and Federico2019) the exact role personality plays within political decision-making is still under debate (Hatemi et al., Reference Hatemi, Medland, Klemmensen, Oskarrson, Littvay, Dawes, Verhulst, McDermott, Nørgaard, Klofstad, Christensen, Johannesson, Magnusson, Eaves and Martin2014).
Overall, the efficaciousness of psychologically tailored arguments and psychological targeting remains largely untested within the political domain while the causal influence of psychological constructs like Personality and Moral Foundations in political decision-making requires further investigation.
Objective
To investigate psychological-based argument targeting and tailoring, and the potential causal influence of personality and moral foundations in political decision-making, this study sought to test four hypothesesFootnote 2 in terms of the targeting and tailoring of arguments for Leaving the European Union:
-
1. In-group loyalty tailored arguments will be more effective for Remain-supporting voters who are high on the Moral Foundation of Ingroup Loyalty.
-
2. Fairness tailored arguments will be more effective for Remain-supporting voters who are high on the Moral Foundation of Fairness.
-
3. Openness tailored arguments will be more effective for Remain-supporting voters who are high on the personality trait Openness.
-
4. Conscientiousness tailored arguments will be more effective for Remain-supporting voters who are high on the personality trait Conscientiousness.
Loyalty and Fairness (study 1) were chosen because they had previously been effective in moral framing research in the US, with Loyalty particularly appealing to conservatives, and Fairness to liberals (Voelkel & Feinberg, Reference Voelkel and Feinberg2018). Openness and Conscientiousness (study 2) were chosen because, of the big 5 personality traits, they have shown the most robust links to liberal and conservative political attitudes respectively (Gerber et al., Reference Gerber, Huber, Doherty, Dowling and Ha2010).
Methods
Study 1 – Moral Foundations
398 Remain supporting participantsFootnote 3 undertook an abbreviatedFootnote 4 Moral Foundations Questionnaire (MFQ) and received MFT-based arguments tailored to the foundations of Loyalty or Fairness.
The research design was mixed; there was one independent variable, message tailoring with two levels – tailored or control. To ensure participants were assigned to the condition reflecting their most (or least) dominant trait, participants first completed a Moral Foundations questionnaire measuring their Loyalty and Fairness. Two standardised z-scores – one for each trait – were created for each participantFootnote 5. Participants were then assigned to read either the Loyalty or Fairness arguments based on their most extreme (highest or lowest) z-score, to maximise the power of the design. Those in the Loyalty condition received arguments tailored to high Loyalty, and a non-tailored control, whilst those in the Fairness condition received arguments tailored to high Fairness and a non-tailored control.
After the Moral Foundation Questionnaire participants read four counter-attitudinal arguments: three MFT tailored arguments and one non-tailored control. The control argument was included to test the causal influence of argument tailoring on decision-making; participants high vs. low on Loyalty and Fairness are not expected to rate a non-tailored control any differently.
After each argument participants were asked three questions about the argument, two measuring their attitude to the argument (argument credibility and argument compellingness) and one measuring behavioural intentions (intention to vote for a party making this argument) on a scale from 0 to 10Footnote 6.
Prior to the analysis, and for analysis purposes, participants were assigned to a high or low Loyalty (/Fairness) group based on the average Loyalty (/Fairness) score in the sample. Those with scores less than or equal to the average were considered low.
A similar study with Leave supporting participants was discontinued due to a small sample size (N = 83).
Study 2 – Personality
A second study following the exact same procedure was conducted to assess the effect of personality-based argument tailoring. 395 Remain supporting participants undertook an abbreviated Big Five Inventory questionnaire (B-5) and received personality-based arguments tailored to the traits of Conscientiousness and Openness.
Again, a study investigating Leave supporting participants was discontinued due to a small sample size (N = 65).
Analysis Strategy
Given the mixed design, the primary statistical interest was the interaction effect between the high and low groups across the tailored and control arguments. To analyse these effects, we conducted two-way mixed ANOVAFootnote 7 models. These models assessed the between impact of moral foundation (/personality) group (high vs. low), the within impact of argument type (tailored vs. control) and the interaction effect on the argument scores. Given paired comparisons were performed separately for the three rating measures (credible, compellingness, vote intention), for simplicity these were combined for the ANOVA models.
The secondary interest was a between-subjects test of whether participants high on a given moral foundation (/personality) trait rated the arguments differently from those low on the same trait. To assess these paired comparisons independent t-tests were conducted across each of the three rating measuresFootnote 8. Given participants rated the arguments on three separate dependent variables – Credibility, Compellingness and Vote – the statistical threshold for the paired comparisons was controlled using the Holm-Bonferroni correction. All p-values reported for the independent t-tests are two-sided.
Whilst the distributions were not normal, the sample sizes were considered sufficiently large to conduct both two-way mixed ANOVAs and independent t-tests.
Results
Study 1 - Moral Foundations
Loyalty condition
Two-way mixed ANOVA results showed a significant main effect of group on argument scores F(1, 1,172) = 302.96, p < .001, 𝜂 p2 = .21. There was also a significant main effect of argument type on argument scores F(1, 1,172) = 14.97, p < .001, 𝜂p2 = .013. However, there was no significant interaction effect for Loyalty, F(1, 1,172) = .0001, p = .99, 𝜂p2 < 0.001.
Walker Fig 1. – Mean argument rating of high and low Loyalty groups across the tailored and non-tailored arguments. (Error bars denote standard errors).
To investigate the between-subjects effects across the arguments independent t-tests were conducted.
Independent t-tests indicated that individuals high on Loyalty significantly differed from those low on Loyalty across the tailored arguments. Those high on Loyalty found the arguments, more credible, t(200.57) = 6.35, p < .001, d = .89, more compelling, t(177.84) = 8.63, p < .001, d = 1.5 and showed a greater intention to vote, t(169.6) = 8.63, p < .001, d = 1.58 than those low on Loyalty.
This pattern of significant differences between those high and low on Loyalty was also evident across the non-tailored control argument. Those high on Loyalty found the argument more credible, t(199.29) = 4.83, p < .001, d = .68, more compelling, t(169.27) = 6.54, p < .001, d = 1.2 and showed a greater intention to vote, t(155.78) = 6.49, p < .001, d = 1.33 than those low on Loyalty.
Fairness condition
Two-way mixed ANOVA results showed a significant main effect of group on argument scores, F(1, 1,058) = 32.28, p < .001, 𝜂p2 = .03. There was also a significant main effect of argument type on argument scores F(1, 1,058) = 64.98, p < .001, 𝜂p2 = .06. There was no significant interaction effect for Fairness, F(1, 1,058) = 2.86, p = .09, 𝜂p2 = 0.003.
Walker Fig 2. – Mean argument rating of high and low Fairness groups across the tailored and non-tailored arguments. (Error bars denote standard errors).
Independent t-tests indicated that individuals high on Fairness significantly differed from those low on Fairness across the tailored arguments. Those low on Fairness found the arguments, more credible, t(110.17) = −2.16, p = .03, d = −.34, more compelling, t(101.92) = −3.33, p < .01, d = −.54 and showed a greater intention to vote, t(99.04) = −3.42, p < .001, d = −.56 than those high on Fairness.
This pattern of significant differences between those high and low on Fairness was also evident across the non-tailored control argument. Those low on Fairness found the argument more credible, t(98.71) = −2.81, p < .01, d = −.46, more compelling, t(90.82) = −3.34, p < .01, d = −.49 and showed a greater intention to vote, t(82.01) = −3.3, p < .01, d = −.47 than those high on Fairness.
Study 2 - Personality
Conscientiousness condition
Two-way mixed ANOVA results showed a significant main effect of group on argument scores, F(1, 1,166) = 16.18, p < .001, 𝜂p2 = .014. There was also a significant main effect of argument type on argument scores, F(1, 1,166) = 12.78, p < .001, 𝜂p2 = .011. There was a marginally non-significant interaction effect for Conscientiousness, F(1, 1,166) = 3.41, p = .065, 𝜂p2 = 0.003.
Walker Fig 3. – Mean argument rating of high and low Conscientiousness groups across the tailored and non-tailored arguments. (Error bars denote standard errors).
Independent t-tests indicated that individuals high on Conscientiousness significantly differed from those low on Conscientiousness across the tailored arguments. Those high on Conscientiousness found the arguments, more credible, t(145.93) = 2.00, p = .048, d = .3, more compelling, t(135.12) = 2.33, p = .021, d = .35 and showed a greater intention to vote, t(131.05) = 2.56, p = .01, d = .39 than those low on Conscientiousness.
There were no significant differences across the non-tailored control argument: credibility, t(137.49) = .81, p = .42, d = .12, compellingness, t(134.97) = .50, p = .62, d = .07 and vote intention t(128.39) = 1.05, p = .30, d = .16.
Openness condition
Two-way mixed ANOVA results showed no significant main effect of group on argument scores, F(1, 1,136) = 3.15, p = .076, 𝜂p2 = .003. There was a significant main effect of argument type on argument scores, F(1, 1,136) = 19.95, p < .001, 𝜂p2 = .02. There was no significant interaction effect for Openness, F(1, 1,136) = 2.45, p = .12, 𝜂p2 = 0.002.
Walker Fig 4. – Mean argument rating of high and low Openness groups across the tailored and non-tailored arguments. (Error bars denote standard errors).
Independent t-tests indicated there was no significant differences across the tailored arguments: credibility, t(165.56) = −.62, p = .53, d = −.09, compellingness, t(176.7) = −.50, p = .61, d = −.07 and vote intention, t(181.73) = −1.18, p = .24, d = −.16.
On the non-tailored control argument there were no significant difference in terms of credibility, t(160.7) = −.62, p = .54, d = −.09 or compellingness, t(138.31) = −2.08, p = .04, d = −.32. There was a significant difference in terms of vote intention with those low on Openness displaying a greater intention to vote, t(140.85) = −3.02, p < .01, d = −.46 than those high on Openness.
Discussions
Across all four traits there was either no or weak evidence for the interaction between targeting and tailoring. There were however clear effects of targeting, such that participants high or low on a trait rated arguments differently in a manner consistent with that traits association with more liberal or conservative politics. The strength and consistency of these differences differed substantially across traits.
The strongest and most consistent difference was between individuals high and low on Loyalty. This difference – showing a receptivity towards the arguments for those high in Loyalty – was a large effect for the arguments tailored towards Loyalty (d = 0.89–1.58) but also large for the non-tailored argument (d = 0.68–1.33), and there was no interaction. The lack of an interaction between targeting and tailoring for Loyalty suggests that whilst targeting might be effective, these results do not enable us to be certain that receptivity to the moral value of Loyalty is critical; this difference could be mediated by other psychological or demographic differences between these groups. It should be noted however that, whilst the non-tailored argument contained less references to Loyalty, it did contain some (for example to ‘British students’), which may have been sufficient to appeal to participants high on Loyalty.
Conscientiousness was the only trait where participants in the high compared to low groups displayed a difference between the tailored and the non-tailored arguments across the pairwise comparisons. Those high compared to those low on Conscientiousness showed a consistent receptivity towards the tailored arguments whilst showing no differences across the non-tailored control argument. This combination of differences is at least consistent with the idea that personality-based argument tailoring can influence political-based decision-making. However, the absence of a significant interaction effect cautions the veracity of this interpretation. Detecting interaction effects often requires more power however, so the pattern of pairwise comparisons suggests this result is worth testing with a larger sample.
Participants high on Fairness consistently rated the Leave arguments as less persuasive than those low on Fairness across the tailored and non-tailored arguments. Participants high on Openness (compared to low) only differed in their evaluation of one outcome (vote intention) for the non-tailored argument – showing less receptivity towards it. For both Fairness and Openness there was no interaction. The lack of hypothesised effects across the Fairness and Openness conditions suggests that argument tailoring is particularly unlikely to prove effective in the absence of thoughtful targeting. In this instance the lack of an effect for Fairness and Openness could reflect the incongruence between the public discourse surrounding the Leave position and these psychological constructs.
Finally, it should be noted, all of these arguments were tested using arguments for Leaving the EU presented to Remain supporting participants. It would clearly be informative to extend this test to Leave supporting participants with arguments in favour of Remaining. Previous literature has highlighted associations between Loyalty and Conscientiousness and conservativism, and Openness and Fairness with liberalism, so the pattern of targeting effects found here should theoretically be reversed when making arguments for Remaining in the EU.
Conclusions
The results reveal that participants high or low on different traits respond differently to arguments for Leaving the EU. Some of these differences are large, with Remain supporters high on Loyalty consistently finding arguments much more persuasive than Remain supporters low on Loyalty. Some of these differences are more marginal, with participants high on Openness finding arguments for Leaving less persuasive than participants low on Openness only in one instance. Across all of the traits however there is no convincing evidence that the combination of targeting and tailoring is effective. The only potential exception is the case of Conscientiousness, in which participants high on this trait (compared to low) found the tailored arguments more persuasive but showed no difference on the non-tailored argument. Even in this case however the interaction between personality trait and message tailoring was not significant. Thus the results show substantive evidence for psychological targeting, but weak or no evidence for psychological targeting and tailoring.
Author Contributions
CW, SON and LdW conceived and designed the study. CW and SON conducted data gathering and performed statistical analyses. CW, SON and LdW wrote the article.
Funding Information
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Data Availability Statements
All the research data and materials required to replicate this study are available at https://osf.io/fd29m/.
Ethics
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Conflicts of Interest
Conflicts of Interest: Christopher Walker, Stephen O’Neill and Lee de-Wit declare none.
Supplementary Materials
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/exp.2020.43.
Open access
Comments
Comments to the Author: Please forgive my curtness. I only have 250 words.
The study topic is interesting and I think the paper can be revised to make an effective contribution. Some additional details are needed, and some reframing may be appropriate.
-Why dichotomize the moral foundation scores and personality? Can continuous scores be used?
-MFQ scores and personality are observed, not manipulated. The primary IV should be message tailoring. Mixed ANOVAs should take center stage, even if post-hoc; the experimental component of the design is really within-, not between-subjects. The Wilcoxon rank-sum tests are correlational and indicate only that participants who valued loyalty above all else responded more favorably to pro-Brexit arguments (still an interesting finding).
-All variables should be more clearly defined in the main paper or appendix if space allows. It’s unclear how predictors were “abbreviated” and how the DVs were assessed. Of the outcomes, “vote intention” sounds like it could be a non-directional turnout variable, but I assume that’s not the case. Can we also get alphas for the predictors?
-Because MFQ scores and personality are observed, not manipulated, Wilcoxon ranked sum analyses are subject to omitted variable bias. Can authors supplement with analyses that control for ideology or related variables? For example, participants who valued loyalty may have responded favorably to all the pro-Leave arguments not because they valued loyalty per se, but because they were more conservative (or less liberal) than participants who valued loyalty least. Results could imply merely that moderates are more persuadable.