Research in bilingualism often involves quantifying constructs of interest by the use of rating scales: for example, to measure language proficiency, dominance, or sentence acceptability. However, ratings are a type of ordinal data, which violates the assumptions of the statistical methods that are commonly used to analyse them. As a result, the validity of ratings is compromised and the ensuing statistical inferences can be seriously distorted. In this article, we describe the problem in detail and demonstrate its pervasiveness in bilingualism research. We then provide examples of how bilingualism researchers can employ an appropriate solution using Bayesian ordinal models. These models respect the inherent discreteness of ratings, easily accommodate non-normality, and allow modelling unequal psychological distances between response categories. As a result, they can provide more valid, accurate, and informative inferences about graded constructs such as language proficiency. Data and code are publicly available in an OSF repository at https://osf.io/grs8x.

English outweighs other languages as a source for linguistic borrowing in present-day Norwegian. Most of the research on this topic has considered direct lexical loans, yet observations indicate that English is increasingly burrowing its way below the lexical surface of Norwegian – evident in the rising number of calques. This study investigates how a selection of recently observed multiword calques from English are evaluated by native Norwegian speakers. This is examined through acceptability judgements among young bilingual Norwegians (n = 83, mean age 18), and among two control groups comprising 10 bilingual adults (mean age 55) and 16 minimally bilingual seniors (mean age 75), respectively. A translation test was included to compare the judgements against production data. Based on our overall findings, we suggest that increased calquing is a sign of heightened proficiency in and exposure to the English language.