The results of song contests offer a unique opportunity to analyze possible distortions arising from various biases in performance evaluations using observational data. In this study we investigate the influence of contestants’ order of appearance on their ranking. We found that, in the New Wave Song Contest, expert judgments were significantly influenced by the contestant’s running number, an exogenous factor that, being assigned randomly, clearly did not influence the output quality. We also found weaker statistical evidence of such an ordering effect in Eurovision Song Contest finals of 2009–2012.
