Hostname: page-component-78c5997874-v9fdk Total loading time: 0 Render date: 2024-11-09T06:45:35.841Z Has data issue: false hasContentIssue false
  • English
  • Français

Is Frequentist Testing Vulnerable to the Base-Rate Fallacy?

Published online by Cambridge University Press:  01 January 2022

Aris Spanos
Get access Rights & Permissions [Opens in a new window]

Extract

This article calls into question the charge that frequentist testing is susceptible to the base-rate fallacy. It is argued that the apparent similarity between examples like the Harvard Medical School test and frequentist testing is highly misleading. A closer scrutiny reveals that such examples have none of the basic features of a proper frequentist test, such as legitimate data, hypotheses, test statistics, and sampling distributions. Indeed, the relevant error probabilities are replaced with the false positive/negative rates that constitute deductive calculations based on known probabilities among events. As a result, the ampliative dimension of frequentist induction—learning from data about the underlying data-generating mechanism—is missing.

Type
Research Article
Information
Philosophy of Science , Volume 77 , Issue 4 , October 2010 , pp. 565 - 583
Copyright
Copyright © The Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

‡

I would like to thank Stathis Psillos for encouraging me to focus on this problem and two anonymous referees whose constructive comments and suggestions helped to improve the article.

References

Achinstein, Peter. 2001. The Book of Evidence. Oxford: Oxford University Press.CrossRefGoogle Scholar
Achinstein, Peter. 2010. “Mill's Sins or Mayo's Errors?” In Error and Inference: Recent Exchanges on Experimental Reasoning, Reliability, and the Objectivity and Rationality of Science, ed. Mayo, D. G. and Spanos, Aris, 170–88. Cambridge: Cambridge University Press.Google Scholar
Billingsley, Patrick. 1995. Probability and Measure. 3rd ed. New York: Wiley.Google Scholar
Cohen, Jacob. 1994. “The Earth Is Round (p < .05).” American Psychologist 49:997–1003.CrossRefGoogle Scholar
Cox, D. R., and Hinkley, D. V.. 1974. Theoretical Statistics. London: Chapman & Hall.CrossRefGoogle Scholar
Fisher, R. A. 1922. “On the Mathematical Foundations of Theoretical Statistics.” Philosophical Transactions of the Royal Society A 222:309–68.Google Scholar
Fisher, R. A.. 1925. “Theory of Statistical Estimation.” Proceedings of the Cambridge Philosophical Society 22:700–725.CrossRefGoogle Scholar
Fisher, R. A.. 1934. “Two New Properties of Maximum Likelihood.” Proceedings of the Royal Statistical Society A 144:285–307.Google Scholar
Fisher, R. A.. 1935. The Design of Experiments. Edinburgh: Oliver & Boyd.Google Scholar
Fisher, R. A.. 1955. “Statistical Methods and Scientific Induction.” Journal of the Royal Statistical Society B 17:69–78.Google Scholar
Hacking, Ian. 1965. Logic of Statistical Inference. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Hardy, I. C. W., ed. 2002. Sex Ratios: Concepts and Research Methods. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Howson, Colin. 1997. “A Logic of Induction.” Philosophy of Science 64:268–90.CrossRefGoogle Scholar
Howson, Colin. 2000. Hume's Problem. Oxford: Oxford University Press.CrossRefGoogle Scholar
Howson, Colin, and Urbach, Peter. 2005. Scientific Reasoning: The Bayesian Approach. 3rd ed. Chicago: Open Court.Google Scholar
Krämer, Walter, and Gigerenzer, Gerd. 2005. “How to Confuse with Statistics; or, The Use and Misuse of Conditional Probabilities.” Statistical Science 20:223–30.CrossRefGoogle Scholar
Lehmann, E. L. 1986. Testing Statistical Hypotheses. 2nd ed. New York: Wiley.CrossRefGoogle Scholar
Mayo, D. G. 1996. Error and the Growth of Experimental Knowledge. Chicago: University of Chicago Press.CrossRefGoogle Scholar
Mayo, D. G.. 1997a. “Error Statistics and Learning from Error.” Philosophy of Science 64:195–212.CrossRefGoogle Scholar
Mayo, D. G.. 1997b. “Response to Howson and Laudan.” Philosophy of Science 64:323–33.CrossRefGoogle Scholar
Mayo, D. G.. 2005. “Evidence as Passing Severe Tests: Highly Probable versus Highly Probed Hypotheses.” In Scientific Evidence: Philosophical Theories and Applications, ed. Achinstein, Peter, 95–127. Baltimore: Johns Hopkins University Press.Google Scholar
Mayo, D. G.. 2010. “Sins of the Epistemic Probabilist: Exchanges with Achinstein.” In Error and Inference: Recent Exchanges on Experimental Reasoning, Reliability, and the Objectivity and Rationality of Science, ed. Mayo, D. G. and Spanos, Aris, 189–201. Cambridge: Cambridge University Press.Google Scholar
Mayo, D. G., and Spanos, Aris. 2004. “Methodology in Practice: Statistical Misspecification Testing.” Philosophy of Science 71:1007–25.CrossRefGoogle Scholar
Mayo, D. G., and Spanos, Aris. 2006. “Severe Testing as a Basic Concept in a Neyman-Pearson Philosophy of Induction.” British Journal for the Philosophy of Science 57:323–57.CrossRefGoogle Scholar
Mayo, D. G., and Spanos, Aris. 2010. “Error Statistics.” In Philosophy of Statistics, Vol. 7, The Handbook of Philosophy of Science, ed. Gabbay, Dov, Thagard, Paul, and Woods, John, 151–96. North-Holland: Elsevier.Google Scholar
Neyman, J. 1956. “Note on an Article by Sir Ronald Fisher.” Journal of the Royal Statistical Society B 18:288–94.Google Scholar
Psillos, Stathis. 2007. Philosophy of Science A–Z. Edinburgh: Edinburgh University Press.Google Scholar
Spanos, Aris. 1999. Probability Theory and Statistical Inference: Econometric Modeling with Observational Data. Cambridge: Cambridge University Press.Google Scholar
Spanos, Aris. 2007. “Curve-Fitting, the Reliability of Inductive Inference, and the Error-Statistical Approach.” Philosophy of Science 74:1046–66.CrossRefGoogle Scholar
Spanos, Aris. 2009. “Model-Based Inference and the Frequentist Interpretation of Probability.” Working paper, Department of Economics, Virginia Tech.Google Scholar
Tversky, Amos, and Kahneman, Daniel. 1982. “Evidential Impact of Base Rates.” In Judgment under Uncertainty: Heuristics and Biases, ed. Kahneman, Daniel, Slovic, Paul, and Tversky, Amos, 153–60. Cambridge: Cambridge University Press.Google Scholar