Many judgment analysis studies employ multiple regression procedures to estimate the importance of cues. Some studies test the significance of regression coefficients in order to decide whether or not specific cues are attended to by the judge or decision maker. This practice is dubious because it ignores type II error. The purposes of this note are (1) to draw attention to this issue, specifically as it appears in studies of self-insight, (2) to illustrate the problem with examples from the judgment literature, and (3) to provide a simple method for calculating post-hoc power in regression analyses in order to facilitate the reporting of type II errors when regression models are used.

This case study outlines the accuracy of a rehabilitation counsellor in judgments of the quality of life of actual cases comprising accident victims. Up to seven cues of information relating to health, energy, daily activities, self, personal relations, money and living conditions were provided to the counsellor. He/she had to decide whether the clients considered their own quality of life to be poor or good. The counsellor was consistent in his/her judgments (test–retest reliability = .79). The counsellor was accurate in 60% of cases, but this is only marginally better than an automatic decision that every accident victim is dissatisfied with his/her quality of life. The judgment accuracy was suboptimal. It was influenced by a reliance (93%) on satisfaction with personal relationships, whereas basic issues such as money for everyday living were more important in predicting quality of life. It did not appear that increasing the amount of information led to an increase in accuracy. The correlation between accuracy and the number of cues was very low (-.160).