Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-25T14:20:19.128Z Has data issue: false hasContentIssue false

How best to improve upon return-to-player information in gambling? A comparison of two approaches in an Australian sample

Subject: Psychology and Psychiatry

Published online by Cambridge University Press:  27 September 2022

Philip W. S. Newall*
Affiliation:
Experimental Gambling Research Laboratory, School of Health, Medical and Applied Sciences, CQUniversity, Melbourne, Victoria, Australia School of Psychological Science, University of Bristol, Bristol, United Kingdom
Lukasz Walasek
Affiliation:
Department of Psychology, University of Warwick, Coventry, United Kingdom
Elliot A. Ludvig
Affiliation:
Department of Psychology, University of Warwick, Coventry, United Kingdom
*
*Corresponding author. Email: [email protected]

Abstract

“Return-to-player” information is used in several jurisdictions to display the long-run cost of gambling, but previous evidence suggests that these messages are frequently misunderstood by gamblers. Two ways of improving the communication of return-to-player information have been suggested: switching to an equivalent “house-edge” format, or via the use of a “volatility warning,” clarifying that the information applies only in the statistical long run. In this study, Australian participants (N = 603) were presented with either a standard return-to-player message, the same message supplemented with a volatility warning, or a house-edge message. The return-to-player plus volatility warning message was understood correctly more frequently than the return-to-player message, but the house-edge message was understood best of all. Participants perceived the lowest chance of winning in the return-to-player plus volatility warning condition. These findings contribute data on the relative merits of two proposed approaches in the design of improved gambling information.

Type
Research Article
Information
Result type: Novel result, Supplementary result
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Introduction

The provision of information to gamblers is one way to potentially improve gamblers’ knowledge, perceptions, and behavior (Armstrong et al., Reference Armstrong, Donaldson, Langham, Rockloff and Browne2018; Cloutier et al., Reference Cloutier, Ladouceur and Sévigny2006; Ginley et al., Reference Ginley, Whelan, Pfund, Peter and Meyers2017; Newall, Weiss-Cohen, et al., Reference Newall, Weiss-Cohen, Singmann, Walasek and Ludvig2022; Palmer du Preez et al., Reference Palmer du Preez, Landon, Bellringer, Garrett and Abbott2016; Rockloff et al., Reference Rockloff, Newall, Browne, Russell, Visintin, Hing and Thorne2022). In many jurisdictions, “return-to-player” messages are used to display the long-run cost of gambling, as in Australia, which displays: “Theoretical return to player of this game = 90%” (Beresford & Blaszczynski, Reference Beresford and Blaszczynski2019). This message means that for every $100 bet, $90 is on average paid out in prizes. Research across Australia (Beresford & Blaszczynski, Reference Beresford and Blaszczynski2019; Monaghan et al., Reference Monaghan, Blaszczynski and Nower2009), Canada (Harrigan et al., Reference Harrigan, Brown and Barton2017), and the UK (Collins et al., Reference Collins, Green, d’Ardenne, Wardle and Williams2014) reveals that many gamblers struggle to properly understand such return-to-player messages. This finding appears borne out in practice, as the applicant in a recent Australian court case argued that return-to-player messages are misleading, as they imply that gamblers will actually receive these returns (Federal Court of Australia, 2018).

Some previous research suggests the advantages of representing return-to-player information in another way (Newall et al., Reference Newall, Walasek and Ludvig2020a). For example, the “house-edge” format focuses instead on the proportion of money bet retained by the gambling game, for example, “This game keeps 10% of all money bet on average.” A house edge of 10% and a return-to-player of 90% are exactly equivalent (Parke et al., Reference Parke, Parke and Blaszczynski2016), and yet, gamblers perceive a lower chance of winning with a house-edge ranging between 5 and 15% than the equivalent return-to-player of 95–85%, using the UK wording of “this game has an average percentage payout of X%” (Collins et al., Reference Collins, Green, d’Ardenne, Wardle and Williams2014). Furthermore, 66.5% of gamblers selected the correct definition for a house-edge message in a multiple-choice question, compared to 45.6% of gamblers given a return-to-player message. These results suggest risk communication benefits of house-edge over return-to-player warning labels in electronic machine gambling, a costly form of gambling (Harrigan & Dixon, Reference Harrigan and Dixon2009; Woolley et al., Reference Woolley, Livingstone, Harrigan and Rintoul2013).

This finding, however, is limited to UK gamblers and the specific phrasing of UK return-to-player messages. In the Australian court case referred to earlier, the judge ruled that return-to-player information is not misleading, but that it is confusing (Federal Court of Australia, 2018). That court case raised the possibility of a longer corrective message highlighting that the return-to-player is only a statistical long-run average return. This longer potential message is called a “return-to-player plus volatility warning” message here, and was implemented in the experiment with the following text added to the return-to-player message, using wording that closely follows the judge’s recommendation:

It takes millions of games for a gaming machine to tend towards its ‘return to players’ setting. An individual gaming machine will not return a minimum value of prizes in any given period of play.

How this longer message might affect the results described earlier is unknown as is whether those findings would translate to the Australian wording for return-to-player information amongst a group of Australian participants. Although this definition of “volatility” differs somewhat from the definition in gambling research at the level of individual games (Turner, Reference Turner2011), this word will be used here in a more general sense to indicate that an individual machine can readily diverge from the long-term average.

Objective

Therefore, the present research was designed to compare house-edge, return-to-player, and longer return-to-player plus volatility warning messages amongst a sample of Australian residents. It was hypothesized that house-edge messages would continue to have two risk communication advantages compared to the other two return-to-player based messages:

Hypothesis 1. House-edge messages will be correctly understood more often than return-to-player and return-to-player plus volatility warning messages.

Hypothesis 2. House-edge messages will yield a lower perceived chance of winning than return-to-player and return-to-player plus volatility warning messages.

Method

The study was preregistered. The preregistration document, materials, data, and full analysis output can be accessed from https://osf.io/36erj/. Data collection occurred from March 11–12, 2020. These findings were previously made available online as a preprint (Newall et al., Reference Newall, Walasek and Ludvig2020b).

Participants

Australian resident adults (N = 603), with an average age of 32.2 years (SD = 11.5), 48.0% female (0.8% preferred not to answer), were recruited from the crowdsourcing platform Prolific Academic, which allowed for the recruitment of an anonymous sample. Participants were paid $1 AUD each and took an average of 2.6 min on the task ($22/per-hour pro-rata). Responses to the first question on the Consumption Screen for Problem Gambling (CSPG; Rockloff, Reference Rockloff2012) indicated that 46.3% of the sample had gambled in the past year. Of those who had gambled, 73.5% did so monthly or less often, and 65.6% did so on average for less than 30 min at a time.

Design and materials

Much of gambling on electronic gambling machines in Australia occurs in what is colloquially called “pokies clubs.” Participants in all conditions were given some descriptive text to cue this context:

“Imagine that you are gambling in a pokies club. You have played on many of this club’s pokies before.

You know that gambling games are designed so that most gamblers lose money over time. Only a percentage of all the money bet gets paid back out as winnings.

You are about to start playing a new pokie game when you read the following information on the game’s help screen.”

Below are wordings of the three warning labels, manipulated between participants, meaning that participants were randomly assigned to see one of these three different pieces of information:

“This game keeps 10% of all money bet on average” (house-edge condition).

“Theoretical return to player of this game = 90%” (return-to-player condition; this wording was given to us as the current phrasing of this information in the Australian state of Victoria).

“Theoretical return to players of this game = 90%.

It takes millions of games for a gaming machine to tend toward its “return to players” setting. An individual gaming machine will not return a minimum value of prizes in any given period of play” (return-to-player plus volatility warning condition; this wording was given to us as the proposed updated phrasing of this information in the Australian state of Victoria).

Measures and procedure

The measure of objective warning label understanding was a 4-item, multiple-choice question, where the correct answer was: “For every $100 bet on this game about $90 is paid out in prizes” (see Table 1 for all options). The measure of subjective chances of winning involved asking participants “How does the above information affect your perceived chances of winning?”, to which they responded via a 7-point Likert scale (“My chances of winning are… very high/high/somewhat high/neither high nor low/somewhat low/low/very low… chance of coming out ahead”). On this scale, very high was coded as a 7, and very low as a 1. Both measures have been used in previous research (Newall et al., Reference Newall, Walasek and Ludvig2020a).

Table 1. Responses to the measure of objective warning label understanding

Participants in each condition saw the scenario text on two occasions. On one occasion, they provided their response to the measures of objective warning label understanding, and on the other occasion, they provided their response to the measure of subjective chances of winning. The order in which these two responses were presented to participants was counterbalanced. Participants then completed the Consumption Screen for Problem Gambling (Rockloff, Reference Rockloff2012), a measure of gambling involvement that correlates strongly with disordered gambling status. Demographics were collected directly by the crowdsourcing platform.

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki. The Humanities and Social Science Research Ethics Committee of the University of Warwick approved the study. All subjects were informed about the study, and all provided informed consent.

Results

A breakdown of responses to the measure of objective warning label understanding is shown in Table 1. Overall, 79.9% of participants identified the correct response in the house-edge condition, which was shown via logistic regression to be significantly more than the 52.9% in the return-to-player condition (OR = 0.28, z = −5.59, p < .001), and the 63.5% in the return-to-player plus volatility warning condition (OR = 0.44, z = −3.59, p < .001). These results are consistent with Hypothesis 1. Additional exploratory logistic regression analysis showed that significantly more participants correctly understood the longer return-to-player plus volatility warning message than the return-to-player message (OR = 1.55, z = 2.15, p = .032). We then ran an exploratory analysis to see whether these results depended on whether participants had gambled in the past 12 months. Their response to the first CSPG question was used to group participants by whether they had or had not gambled in the previous 12 months. An exploratory model was run, adding a main effect for gambling status and interaction terms between gambling status and the two binary variables for participants’ condition. In this analysis, neither of the interaction terms was significant, and the significance of the main effects from the main analysis was unchanged. Overall, this suggested that the main analysis was robust across both gamblers and nongamblers.

On the 7-point scale for the subjective chances of winning measure, participants indicated an average of 3.75 (SD = 1.64) in the house-edge condition, 3.99 (SD = 2.03) in the return-to-player condition, and 2.74 (SD = 1.71) in the return-to-player plus volatility warning condition. The difference between the house-edge and return-to-player condition was in the hypothesized direction, but the preregistered ordinal logistic regression model indicated that this difference was not statistically significant (OR = 1.23, z = 1.20, p = .232). The average return-to-player plus volatility warning perceived chance of winning was significantly lower than in the house-edge condition, however (OR = 0.36, z = −5.81, p < .001). Hypothesis 2 was therefore not supported. An exploratory interaction model was run, which showed that the significant difference between the house-edge and the return-to-player plus volatility warning condition did not differ between gamblers and nongamblers (OR = 1.03, z = 0.09, p = .926), and the significance of the main effects from the main analysis was unchanged.

Discussion

Hypothesis 1 was supported, as the house-edge message was understood more often than either the return-to-player message or the longer return-to-player plus volatility warning message. An Australian court case suggested that longer return-to-player plus volatility warning messages are less confusing than return-to-player messages (Federal Court of Australia, 2018). That result was observed here, as 63.5% of participants in the return-to-player plus volatility warning condition provided the correct response to the measure of objective warning label understanding, which was higher than the 52.9% who provided the correct response in the return-to-player condition. However, the results also suggested that house-edge messages would be even less confusing for Australian gamblers, as indicated by the 79.9% of participants providing the correct response in that condition. For hypothesis 2, even though subjective chances of winning were lower in the house-edge than return-to-player condition, this difference was not statistically significant. Subjective chances of winning were much lower in the longer return-to-player plus volatility warning condition, however.

This effect of the volatility warning on gamblers’ subjective chances of winning was unexpected but suggests that the addition of this explanatory information may be an effective method of influencing gamblers’ perceived chances of winning and perhaps even behavior. This finding motivated two follow-on pieces of research, which used a 2 × 2 factorial experimental design to test combinations of information format (return-to-player × house-edge) and volatility warning presence (absent × present). These later research projects found that the combination of house-edge information and a volatility warning was the best combination, in terms of gamblers’ accurate understanding and lowering of the perceived chances of winning (Newall et al., Reference Newall, Walasek and Ludvig2020c), and also in terms of their likelihood to cease gambling (Newall, Byrne, et al., Reference Newall, Byrne, Russell and Rockloff2022). Overall, these later works helped support the case for combining a volatility warning with house-edge information, but this was unexplored here.

The present research was limited by being based on a crowdsourced sample, a relatively new data collection method for gambling research (Schluter et al., Reference Schluter, Kim and Hodgins2018). This meant that participants were paid to take part in the experiment, which may have limited the external validity of the results found. Although there were no differences in message effectiveness across gamblers and nongamblers, 46.3% of the sample that had gambled in the last year were fairly low-frequency gamblers. The results may therefore not generalize to high-frequency gamblers. Other potential methods of improving gambling warning labels, such as the use of instructional graphics (Walker et al., Reference Walker, Stange, Dixon, Koehler and Fugelsang2019), were not explored. The study relied on self-report measures to a simplified vignette, rather than using more valid behavioral measures in a realistic gambling task. Furthermore, other ways of communicating house-edge information have been proposed but not tested in similar tasks (Livingstone et al., Reference Livingstone, Rintoul, de Lacy-Vawdon, Borland, Dietze, Jenkinson, Livingston, Room, Smith, Stoove, Winter and Hill2019).

Conclusion

The results suggest that a longer return-to-player plus volatility warning messages can have a large impact on gamblers’ subjective chances of winning, and a smaller effect in improving gamblers’ objective understanding, but that house-edge messages continue to be understood best by gamblers.

Acknowledgments

Thanks to Lindsay Shaw at the Victorian Responsible Gambling Foundation for assisting us with details about the regulation of electronic gambling machines in the Australian state of Victoria.

Data Availability Statement

The data are available from https://osf.io/36erj/.

Authorship Contributions

P.W.S.N., L.W., and E.A.L. conceived and designed the study. P.W.S.N. conducted data gathering and performed the statistical analysis. P.W.S.N., L.W., and E.A.L. wrote the article.

Funding Statement

This research was funded by a Research Development Fund awarded to L.W. from the University of Warwick.

Conflict of Interest

P.W.S.N. is a member of the Advisory Board for Safer Gambling—an advisory group of the Gambling Commission in Great Britain, and in 2020 was a special advisor to the House of Lords Select Committee Enquiry on the Social and Economic Impact of the Gambling Industry. In the last 5 years, P.W.S.N. has contributed to research projects funded by the Academic Forum for the Study of Gambling, Clean Up Gambling, GambleAware, Gambling Research Australia, NSW Responsible Gambling Fund, and the Victorian Responsible Gambling Foundation. P.W.S.N. has received travel and accommodation funding from the Spanish Federation of Rehabilitated Gamblers and received open access fee grant income from Gambling Research Exchange Ontario. L.W. has received open access fee grant income from Gambling Research Exchange Ontario. E.A.L. was co-investigator on a grant funded by the Alberta Gambling Research Institute that ended in February 2019 and has received open access fee grant income from Gambling Research Exchange Ontario. He is also the Research Co-Chair for the Academic Forum for the Study of Gambling.

References

Armstrong, T., Donaldson, P., Langham, E., Rockloff, M., & Browne, M. (2018). Exploring the effectiveness of an intelligent messages framework for developing warning messages to reduce gambling intensity. Journal of Gambling Issues, 2018, 6784.Google Scholar
Beresford, K., & Blaszczynski, A. (2019). Return-to-player percentage in gaming machines: Impact of informative materials on player understanding. Journal of Gambling Studies, 36, 5167. http://doi.org/10.1007/s10899-019-09854-zCrossRefGoogle Scholar
Cloutier, M., Ladouceur, R., & Sévigny, S. (2006). Responsible gambling tools: Pop-up messages and pauses on video lottery terminals. The Journal of Psychology, 140, 434438.CrossRefGoogle ScholarPubMed
Collins, D., Green, S., d’Ardenne, J., Wardle, H., & Williams, S. (2014). Understanding of return to player messages: Findings from user testing. NatCen Social Research.Google Scholar
Federal Court of Australia. (2018). Guy v Crown Melbourne Limited (no. 2) [2018] FCA 36. https://www.judgments.fedcourt.gov.au/judgments/Judgments/fca/single/2018/2018fca0036Google Scholar
Ginley, M. K., Whelan, J. P., Pfund, R. A., Peter, S. C., & Meyers, A. W. (2017). Warning messages for electronic gambling machines: Evidence for regulatory policies. Addiction Research & Theory, 25, 495504. http://doi.org/10.1080/16066359.2017.1321740CrossRefGoogle Scholar
Harrigan, K., Brown, D., & Barton, K. R. (2017). Classification of slot machines in Ontario: Providing relevant information to players. Gambling Research Exchange Ontario.Google Scholar
Harrigan, K., & Dixon, M. (2009). PAR sheets, probabilities, and slot machine play: Implications for problem and non-problem gambling. Journal of Gambling Issues, 23, 81110.CrossRefGoogle Scholar
Livingstone, C., Rintoul, A., de Lacy-Vawdon, C., Borland, R., Dietze, P., Jenkinson, R., Livingston, M., Room, R., Smith, B., Stoove, M., Winter, R., & Hill, P. (2019). Identifying effective policy interventions to prevent gambling-related harm. Victorian Responsible Gambling Foundation.Google Scholar
Monaghan, S., Blaszczynski, A., & Nower, L. (2009). Do warning signs on electronic gaming machines influence irrational cognitions? Psychological Reports, 105, 173187.CrossRefGoogle ScholarPubMed
Newall, P. W. S., Byrne, C. A., Russell, A. M. T., & Rockloff, M. J. (2022). House-edge information and a volatility warning lead to reduced gambling expenditure: Potential improvements to return-to-player percentages. Addictive Behaviors, 130, 107308. http://doi.org/10.1016/j.addbeh.2022.107308CrossRefGoogle Scholar
Newall, P. W. S., Walasek, L., & Ludvig, E. A. (2020a). Equivalent gambling warning labels are perceived differently. Addiction, 111, 17621767. http://doi.org/10.1111/add.14954CrossRefGoogle Scholar
Newall, P. W. S., Walasek, L., & Ludvig, E. A. (2020b). Gambling warning labels: Differing effects of message length and message format. https://psyarxiv.com/69qgrCrossRefGoogle Scholar
Newall, P. W. S., Walasek, L., & Ludvig, E. A. (2020c). Risk communication improvements for gambling: House-edge information and volatility statements. Psychology of Addictive Behaviors, 36, 358363. http://doi.org/10.1037/adb0000695CrossRefGoogle Scholar
Newall, P. W. S., Weiss-Cohen, L., Singmann, H., Walasek, L., & Ludvig, E. A. (2022). Impact of the “when the fun stops, stop” safer gambling message on online gambling behaviour: A randomised, online experimental study. Lancet Public Health, 7, E437E443. http://doi.org/10.1016/S2468-2667(21)00279-6CrossRefGoogle ScholarPubMed
Palmer du Preez, K., Landon, J., Bellringer, M., Garrett, N., & Abbott, M. (2016). The effects of pop-up harm minimisation messages on electronic gaming machine gambling behaviour in new zealand. Journal of Gambling Studies, 32, 11151126.CrossRefGoogle ScholarPubMed
Parke, J., Parke, A., & Blaszczynski, A. (2016). Key issues in product-based harm minimisation: Examining theory, evidence and policy issues relevant in Great Britain. Responsible Gambling Trust.Google Scholar
Rockloff, M. (2012). Validation of the consumption screen for problem gambling (CSPG). Journal of Gambling Studies, 28, 207216.CrossRefGoogle Scholar
Rockloff, M., Newall, P., Browne, M., Russell, A. M. T., Visintin, T., Hing, N., & Thorne, H. (2022). Behavioural trial for consistent gambling messaging under the national consumer protection framework. Gambling Research Australia.Google Scholar
Schluter, M. G., Kim, H. S., & Hodgins, D. C. (2018). Obtaining quality data using behavioral measures of impulsivity in gambling research with amazon’s mechanical turk. Journal of Behavioral Addictions, 7, 11221131.CrossRefGoogle ScholarPubMed
Turner, N. E. (2011). Volatility, house edge and prize structure of gambling games. Journal of Gambling Studies, 27, 607623.CrossRefGoogle ScholarPubMed
Walker, A. C., Stange, M., Dixon, M. J., Koehler, D. J., & Fugelsang, J. A. (2019). Graphical depiction of statistical information improves gambling-related judgments. Journal of Gambling Studies, 35, 945968. http://doi.org/10.1007/s10899-019-09860-1CrossRefGoogle ScholarPubMed
Woolley, R., Livingstone, C., Harrigan, K., & Rintoul, A. (2013). House edge: Hold percentage and the cost of EGM gambling. International Gambling Studies, 13, 388402.CrossRefGoogle Scholar
Figure 0

Table 1. Responses to the measure of objective warning label understanding

Reviewing editor:  Matthew Jenkins Waikato District Health Board, Consultation Liaison/Addictions medicine, 193 London St, Hamilton, New Zealand, 3240
Minor revisions requested.

Review 1: How best to improve upon return-to-player information in gambling? A comparison of two approaches in an Australian sample

Conflict of interest statement

Reviewer declares none.

Comments

Comments to the Author: Nice short paper experimentally examining the efficacy of two different methods of teaching players about the house edge. Need to say how the participants were assigned to groups.

Please provide the question for the “subjective chances” of winning question.

Their definition of volatility isn’t quite right. Volatility is the bet-to-bet variance of a game. The industry standardized it to 10000 plays (see Turner, 2011). What they are talking about is the long-term outcome of a game. Games can vary greatly in volatility from very volatiles (slots with large jackpots) to very low volatility games (baccarat), but in all cases (regardless of volatility) the house edge emerges over the long-term outcome (thousands of games).

Given the results that house edge is better for understanding expected losses, and a statement about the long term (volatility) improves understanding of their chances of winning, perhaps a combination of a statement about the house edge as well as a statement about the long-term outcome would be an optimal message. They might want to add that to the conclusions.

Presentation

Overall score 4.3 out of 5
Is the article written in clear and proper English? (30%)
5 out of 5
Is the data presented in the most useful manner? (40%)
4 out of 5
Does the paper cite relevant and related articles appropriately? (30%)
4 out of 5

Context

Overall score 4.2 out of 5
Does the title suitably represent the article? (25%)
4 out of 5
Does the abstract correctly embody the content of the article? (25%)
4 out of 5
Does the introduction give appropriate context? (25%)
5 out of 5
Is the objective of the experiment clearly defined? (25%)
4 out of 5

Analysis

Overall score 4 out of 5
Does the discussion adequately interpret the results presented? (40%)
4 out of 5
Is the conclusion consistent with the results and discussion? (40%)
4 out of 5
Are the limitations of the experiment as well as the contributions of the experiment clearly outlined? (20%)
4 out of 5

Review 2: How best to improve upon return-to-player information in gambling? A comparison of two approaches in an Australian sample

Conflict of interest statement

Reviewer declares none.

Comments

Comments to the Author: Minor grammatical errors require adjusting (‘they’ in abstract, position of commas and full stops). ‘Australian consumers’ in objective - potentially misleading as meaning consumers of gambling whereas only 46.3% had gambled in the last year. Addition to limitations was that the participants were paid, potentially skewing the sample population. Please clarify the procedure with enough detail (i.e. each of the three warning labels were provided to all participants in an online survey with the subjective and objective measures below. The information collated was anonymous. The warning labels were presented above both subjective and objective measures for easy reference). Some ambiguity here “participants were provided with the scenario on two occasions.”

Presentation

Overall score 4 out of 5
Is the article written in clear and proper English? (30%)
4 out of 5
Is the data presented in the most useful manner? (40%)
4 out of 5
Does the paper cite relevant and related articles appropriately? (30%)
4 out of 5

Context

Overall score 5 out of 5
Does the title suitably represent the article? (25%)
5 out of 5
Does the abstract correctly embody the content of the article? (25%)
5 out of 5
Does the introduction give appropriate context? (25%)
5 out of 5
Is the objective of the experiment clearly defined? (25%)
5 out of 5

Analysis

Overall score 5 out of 5
Does the discussion adequately interpret the results presented? (40%)
5 out of 5
Is the conclusion consistent with the results and discussion? (40%)
5 out of 5
Are the limitations of the experiment as well as the contributions of the experiment clearly outlined? (20%)
5 out of 5