In this research we have introduced a new test (H-test) for analyzing scale invariance in between group designs, and considering uncertainty in individual responses, in order to study the adequacy of disparate rating and visual scales for measuring abstract concepts. The H-test is easy to compute and, as a nonparametric test, does not require any a priori distribution of the data nor conditions on the variances of the distributions to be tested. We apply this test to measure perceived service quality of consumers of a sports services. Results show that, without considering uncertainty, the 1-7 scale is invariant, in line with the related works regarding this topic. However, de 1-5 scale and the 1-7 scale are invariant when adding uncertainty to the analysis. Therefore, adding uncertainty importantly change the conclusions regarding invariance analysis. Both types of visual scales are not invariant in the uncertainty scenario. Implications for the use of rating scales are discussed.
