Article contents
Dead rats, dopamine, performance metrics, and peacock tails: Proxy failure is an inherent risk in goal-oriented systems
Published online by Cambridge University Press: 26 June 2023
Abstract
When a measure becomes a target, it ceases to be a good measure. For example, when standardized test scores in education become targets, teachers may start “teaching to the test,” leading to breakdown of the relationship between the measure – test performance – and the underlying goal – quality education. Similar phenomena have been named and described across a broad range of contexts, such as economics, academia, machine learning, and ecology. Yet it remains unclear whether these phenomena bear only superficial similarities, or if they derive from some fundamental unifying mechanism. Here, we propose such a unifying mechanism, which we label proxy failure. We first review illustrative examples and their labels, such as the “cobra effect,” “Goodhart's law,” and “Campbell's law.” Second, we identify central prerequisites and constraints of proxy failure, noting that it is often only a partial failure or divergence. We argue that whenever incentivization or selection is based on an imperfect proxy measure of the underlying goal, a pressure arises that tends to make the proxy a worse approximation of the goal. Third, we develop this perspective for three concrete contexts, namely neuroscience, economics, and ecology, highlighting similarities and differences. Fourth, we outline consequences of proxy failure, suggesting it is key to understanding the structure and evolution of goal-oriented systems. Our account draws on a broad range of disciplines, but we can only scratch the surface within each. We thus hope the present account elicits a collaborative enterprise, entailing both critical discussion as well as extensions in contexts we have missed.
Keywords
- Type
- Target Article
- Information
- Copyright
- Copyright © The Author(s), 2023. Published by Cambridge University Press
References
- 4
- Cited by
Target article
Dead rats, dopamine, performance metrics, and peacock tails: Proxy failure is an inherent risk in goal-oriented systems
Related commentaries (20)
An updated perspective on teleonomy
Animal welfare science, performance metrics, and proxy failure
Behavioral proxies compete by the time courses of their rewards, including endogenous rewards
Changing the incentive structure of social media may reduce online proxy failure and proliferation of negativity
Dynamic diversity is the answer to proxy failure
Genies, lawyers, and smart-asses: Extending proxy failures to intentional misunderstandings
It's the biology, stupid! Proxy failures in economic decision making
Navigating proxy failures in education: Learning from human and animal play
On abstract goals’ perverse effects on proxies: The dynamics of unattainability
Proxies, heuristics, and goal alignment
Proxy failure and poor measurement practices in psychological science
Proxy failure as a feature of adaptive control systems
Proxy failure in academia: More than just another example
Proxy failure in social policies as one of the main causes of persistent sexism and racism
Proxy failures in practice: Examples from the sociology of science
Reductionism and proxy failure: From neuroscience to target-based drug discovery
Regulator and agent sophistication as an explanation-generating engine for proxy failure dynamics
Subjective and objective corruption of intuition and rational choice
The cost of success or failure for proxy signals in ecological problems
The determinants of proxy treadmilling in evolutionary models of reliable signals
Author response
Teleonomy, legibility, and diversity: Do we need more “proxynomics”?