Keynes’s Treatise on Probability

Early in his career, Keynes had a significant interest in probability and statistics, leading to his 1921 Treatise on Probability. ^{Footnote 4} In the part of the Treatise devoted to statistical inference, Keynes made the distinction between the descriptive function of the theory of statistics, which involved devising ways of representing and summarizing large amounts of data, and the inductive function, which “seeks to extend its descriptions of certain characteristics of observed events to the corresponding characteristics of other events that have not been observed.” This inductive function of statistics he called the “theory of statistical inference,” and he noted that it was currently understood to be “closely bound up with the theory of probability.” The theory of probability provided the basis for the calculations of probable errors for statistical measures like the mean and the correlation coefficient, and it justified the use of these probable errors in making inferences about unobserved phenomena on the basis of descriptive statistics calculated for a sample (Keynes Reference Keynes1921, p. 371).

Keynes then spent several chapters explaining why this approach to statistical inference was an unsound basis for drawing conclusions from statistical data. As he stated at one point, “To apply these methods to material, unanalyzed in respect to the circumstances of its origin, and without reference to our general body of knowledge, merely on the basis of arithmetic and those characteristics of our material with which the methods of descriptive statistics are competent to deal, can only lead to error and delusion” (Keynes Reference Keynes1921, p. 384).

The quote hints at one of Keynes’s central themes: that induction from statistical material, like all induction, required arguments by analogy. In the case of statistical induction, the key analogy was between the circumstances surrounding the generation of the data used to calculate descriptive statistics and the circumstances surrounding the phenomena about which one wished to draw conclusions. Assessing the extent of resemblance between these two sets of circumstances required knowledge beyond the sample, and the techniques of statistical inference based in probability theory offered no means for taking such knowledge into account.

Keynes accompanied his criticism with an outline of a more satisfactory approach to statistical induction, but it was a sketchy outline. At times, he seemed to be rejecting any role for probability theory in a separate “logic of statistical inference,” and arguing that the process of drawing inferences from statistical data should essentially follow the logic of ordinary induction. Other times, he seemed to be pointing towards a new theory of statistical inference that synthesized his own theory of induction by analogy with probability-based statistical procedures based on the work of the German statistician Wilhelm Lexis. For my purposes, however, what Keynes’s contemporaries believed him to be saying is more important than his intended message.

Keynes, Warren Persons, and the Statistics Texts of the 1920s

John Aldrich (Reference Aldrich2008) tells us that almost all the leading statistical theorists of the 1920s rejected Keynes’s arguments concerning statistical inference, and Mary Morgan (Reference Morgan1990) observes that the Treatise has seldom been cited by econometricians. As both Aldrich and Morgan note, however, one prominent American economist wholeheartedly embraced Keynes’s message, and that was Warren Persons.

Warren Persons was a highly original and skilled economic statistician who made important contributions to the analysis of time series in the period just prior to the 1920s. ^{Footnote 5} These contributions earned him the presidency of the American Statistical Association (ASA) in 1923, and in his Presidential Address, he spoke at length on statistical inference as it related to economic forecasting. One of Persons’s messages was that “statistical probabilities provide no aid in arriving at a statistical inference.” In defending that proposition, Persons borrowed many of Keynes’s arguments, and quoted liberally from Keynes’s Treatise.

Persons asserted that the time series data used in forecasting did not meet the conditions required for use of the theory of probability. Probability-based inference required a random sample of independent observations drawn from a well-defined population, but a time series could not be regarded as a random sample from a population in any but an “unreal, hypothetical sense,” and the individual elements of a time series were not independent of one another. Further, the logic behind using inferential procedures based in probability theory for forecasting assumed that the forecasting period was essentially another random draw from the same population as the sample used to calculate the forecast. Yet, the forecaster typically knew enough about the forecasting period to know ways in which it differed from the period covered by his sample, information that the probability-based measures could not incorporate.

Persons also borrowed some of Keynes’s arguments regarding the use of analogy in statistical induction: statistical results from time series data provided a stronger basis for inference if they were stable when calculated for different subperiods of the data, if they were consistent with results found in other periods and under different circumstances, and if the results “agree with, are supported by, or can be set in the framework of, related knowledge of a statistical or non-statistical nature” (Persons Reference Persons1924, pp. 4–5).

As noted above, others have pointed out Persons’s embrace of Keynes’s views on probability and statistical inference. What I would like to suggest is that Keynes’s arguments, as interpreted by Persons, helped to provide the intellectual justification for the attitude, widely held by leading empirical economists for over two decades, that “statistical inference” could and should be conducted without the apparatus of the classical theory of inference.

During the 1920s, there was a significant change in the way that economic researchers presented statistical data. In particular, there was a sizable jump in the percentage of empirical papers presenting summary statistics like means and correlation coefficients in addition to tables of numbers. This change coincided with a growing attention to training in statistical methods in American economics departments (Biddle 1999). Accompanying this statistical revolution was the publication, in 1924 and 1925, of first editions or new editions of four textbooks on statistical methods for economists. For the next several years, at least until the 1940s, the overwhelming majority of advanced economics students in the US were introduced to the subject by one of these books. And each one of them reflected Persons’s view of Keynes’s position on statistical inference.

The most prominent of the four was Frederick Mills’s Statistical Methods (Mills Reference Mills1924). Mills’s views on statistical inference are worth looking at not only because of the popularity of his textbook, but also because from the mid-1920s to the late 1930s, Mills was essentially the in-house authority on statistical theory at the National Bureau of Economic Research, which sponsored a non-trivial fraction of the empirical research being done in the US prior to WWII.

Mills’s text discussed the classical theory of statistical inference in a chapter entitled “Statistical Induction and the Problem of Sampling.” He adopted Keynes’s distinction between the descriptive and inductive functions of statistics, and, like Persons, regarded statistical inference as a synonym for statistical induction. Mills described the meaning of a representative sample, derived formulas for the probable errors of common statistical measures, and explained how those probable errors could measure the reliability of a sample statistic as an estimate of a characteristic of the population from which the sample was drawn. But he then strongly cautioned his readers against using these measures, arguing that the circumstances that justified their use were “rarely, if ever,” met in economic data, and that they could not account for the sampling biases and measurement errors that represented the most serious obstacles to reliable statistical inference. He recommended that instead of inferential techniques based on probability theory, the statistician use “actual statistical tests of stability,” such as the study of successive samples and comparisons of descriptive statistics across subsamples of the population—the same sorts of inferential procedures recommended by Keynes and Persons.

The general message was the same, but the debt to Keynes and Persons more explicit, in Edmund Day’s 1924 textbook, Statistical Analysis. Day asserted that the theory of probability was hardly applicable to most economic data, and that inferences should be based on “non-statistical tests of reasonableness” and the logic of analogy. In support of this, Day reproduced several paragraphs from Persons’s ASA Presidential Address. William Leonard Crum, senior author of An Introduction to the Methods of Economic Statistics (Crum and Patton Reference Crum and Patton1925), worked closely with Persons, and taught economic statistics at Harvard in the 1920s and 1930s. In a review of Day’s book, Crum endorsed Day’s discussion of inference (Crum Reference Crum1926, p. 545), while, in his own text, he barely mentioned probability theory as an inferential tool. Finally, a second edition of Horace Secrist’s An Introduction to Statistical Methods text appeared in 1925. He included new material on probable error measures, but argued, with the help of a quote from Persons, that these measures were largely inapplicable to economic data.

Statistical Inference in Agricultural Economics

Another source of economists’ views on statistical inference in the 1920s and 1930s is the literature of agricultural economics. As is well known, agricultural economists were at the forefront of the statistical revolution of the 1920s, and a good indication of the opinions of the period’s leading agricultural economists on a broad range of topics related to empirical research is provided by a two-volume report, “Research Method and Procedure in Agricultural Economics,” sponsored by the Social Science Research Council and published in 1928. The report was assembled by four prominent agricultural economists, based on contributions from a number of active researchers. ^{Footnote 6}

The editors’ stance on statistical inference in economics was made clear early in the section of the report devoted to “statistical method”: probability theory had little to contribute to statistical inference in economics, further development of the theory was not likely to soon change this situation, and the overreliance on mathematical formulas in analyzing statistical data was a mistake made by men of limited training and understanding (Advisory Committee 1928, p. 38).

The message that the data of agricultural economics did not meet the assumptions required for probability-based inference, and that statistical inference required both a priori analysis and empirical knowledge beyond that provided by the sample, was repeated by other contributors to the report. The section on sampling explained that most samples in agricultural economics did not meet the definition of random samples, a stance that is noteworthy, given that agricultural economists were more likely than empirical researchers in any other field of economics to have had a hand in the collection of the samples with which they worked. The section of the report specifically devoted to statistical inference was written largely by Elmer J. Working, the economist sometimes credited with producing the first complete and clear analysis of the identification problem. ^{Footnote 7} Working observed that statistical inference was sometimes narrowly defined as the process of generalizing from a sample to the universe from which the sample was drawn, with terms like ‘statistical induction’ or ‘statistical logic’ applied to the problem of generalizing from a sample to some more or less similar universe. Working saw little of value in this distinction, arguing that the universe of which an economist’s sample was truly representative was seldom the universe about which the economist wished to draw inferences. For this reason, the classical formulas for probable errors were of limited usefulness.

How, then, could an economist determine whether a correlation found in a sample was likely to provide a reliable basis for inference about the relationship between variables in that universe that really interested him? Like Keynes and Persons, Working argued that the tests of stability created by Wilhelm Lexis had a role to play—the relationship should at least be stable within the sample if it was to be trusted as an indication of what might be true beyond the sample. But one could have even more faith in a generalization based on a sample correlation if that correlation represented a true cause-and-effect relationship, and much knowledge of which correlations represented causal relationships could be provided by economic theory.

Producing such sample correlations was a tricky business, however, as the causal systems underlying economic phenomena involved many factors. Ideally, the economist would know this system of causal relations, and his sample would allow him to use multiple correlation analysis to account for all the relevant variables in the system. But these ideal conditions were almost never realized. Thus, an important part of statistical inference was being able to judge both how close one was to this ideal and the effects of deviations from the ideal.

Working provided a concrete example of this approach to inference by analyzing a multiple regression model explaining the price of butter. Combining theoretical considerations with a test of within-sample stability, he concluded that the sample estimates did not represent stable cause-and-effect relationships. He cautioned, however, that this sort of inferential process was difficult to standardize. Each case presented unique problems that would tax “to the utmost” the mental powers of the economist. ^{Footnote 8}

In a 1930 paper, Working made a number of these points again, and added an additional theme: his concern with a growing tendency among agricultural economists to substitute refined statistical methods for sound inferential procedures. Working endorsed the use of the most up-to-date multiple regression techniques in analyzing sample data, but believed that often those using refined techniques tended to be less careful and thorough in making inferences from their data, relying solely on statistical analysis rather than logical and theoretical analysis. Working suggested that this was because the greater precision and sophistication of the methods engendered greater confidence in the measurements they produced, while at the same time making it more difficult to understand exactly what those measures represented. That some of the refined statistical techniques that Working saw being substituted for “reasoned inference” were the tools of the classical theory of inference is clear from his comment that “the fact that the theory of sampling gives no valid ground for expecting statistical relationships of the past to hold true in the future has been pointed out repeatedly. Nevertheless, it appears that there continues to be widespread misunderstanding and disregard of this fact” (Working Reference Working1930, esp. pp. 122–125; the quote is from p. 124). ^{Footnote 9}

The 1930s

The 1930s saw further developments of the classical theory of inference, and the discussion of these developments in the journals read by economists (Biddle 1990). Morgan (Reference Morgan1990) identifies a handful of influential economists in the 1930s who were enthusiastic about applying the classical methods of inference to economic data, including Tjalling Koopmans, whose dissertation reinterpreted Ragnar Frisch’s confluence analysis in a way that made it amenable to the application of the classical theory of inference, and Harold Hotelling, who, throughout the 1930s, taught mathematical statistics to economics PhD students at Columbia. She also notes that this sort of work had little noticeable impact on the literature in the 1930s.

John Aldrich identifies Henry Schultz as one American economist who was willing to use probability-based inference in the 1930s, but this observation requires some qualification. Schultz, who worked primarily on the estimation of demand curves, was recognized as an authority on the application of regression techniques to time series data, and was the leading economic statistician at the University of Chicago from the late 1920s until his death in 1938. Although Schultz understood the classical theory of inference, measures derived from the theory played only a minor role in his attempts to draw conclusions from his estimates. His main strategy was to estimate several specifications of a demand curve, using different methods to control for the trend, different functional forms, and so forth. Then, using a variety of economic and statistical considerations, he would distill from these results a best estimate of the true demand curve. ^{Footnote 10}

In the 1930s, Schultz began a project of statistically testing the restrictions implied by the neoclassical theory of the consumer, and he used the standard errors of the coefficients of his estimated demand curves as a metric for determining whether the relationships between the coefficients were close enough to those implied by the theory to count as a confirmation of the theory (Schultz Reference Schultz1938, chs. XVIII, XIX). And, in the 1938 book summarizing his life’s work, he included an explanation of the fundamentals of the classical theory of hypothesis testing (Schultz Reference Schultz1938, pp. 732–734).

However, the same 1938 book includes a discussion of standard errors and significance testing that reveals serious reservations about the usefulness of the classical theory of inference in the analysis of time series data. Schultz argued that time series could not be considered as samples of a definable universe in any reasonable sense. Also, there were typically changing patterns of dependence between successive observations in a time series. So, standard errors derived from time series, Schultz wrote, “do NOT [and the emphasis is his] tell us the probability that the true value of a given parameter will be found between such and such limit.” But that, of course, was precisely the question that the classical theory of inference had been developed to answer. Instead, Schultz said, standard errors calculated for coefficients of a time series regression should be regarded as only one measure of goodness of fit (Schultz Reference Schultz1938, pp. 214–215).

In 1938, Frederick Mills issued a revised edition of his textbook (Mills Reference Mills1938). He noted in the preface that in the years since his last book, statistics had experienced “ferment and creative activity similar to that associated with Karl Pearson and his associates,” including the development of improved methods for testing hypotheses and a “sounder foundation” for statistical inference. As a result, the material on inferential methods derived from probability theory had been expanded (Mills Reference Mills1938, pp. vii–viii).

Although Mills’s revised chapter, “Statistical Induction and the Problem of Sampling,” now emphasized the importance of the classical measures of reliability, he did not tone down his warnings about the use of these measures to any significant extent. ^{Footnote 11} Still, students using Mills’s textbook were shown how to conduct Ronald Fisher’s analysis of variance, calculate standard errors for regression coefficients, and conduct T-tests when dealing with small samples. Mills himself used classical inferential methods in a 1926 paper that proposed and tested a hypothesis regarding the cause of a secular change in the duration of business cycles, although he was careful to note that he was testing only the likelihood that observed changes in cycle duration were due to “sampling fluctuation alone” (Mills Reference Mills1926).

Both Mills and Schultz were aware of, and concerned about, the stringent data requirements for the valid application of the classical inferential measures. However, they were also committed to making economics more like the natural sciences, in which theoretical hypotheses were tested empirically. One can conjecture that their enthusiasm for the potential of the classical theory of inference to provide economists with a more precise way of determining the extent to which statistical evidence supported theoretical assertions occasionally pushed them to “damn the torpedoes” and go full speed ahead with the classical inferential methods.

One goal of this section has been to establish that the era of “statistical inference without probability” preceding WWII was not rooted in an unreflective rejection of probability theory, and neither was it the result of a naiveté about the problems created for empirical research by the inexact nature of economic theory. Instead, leading empirical economists articulated a sophisticated, reasonable case against drawing conclusions from economic data using inferential procedures based on probability theory. But this case did not carry the day, and statistical inference in economics came to mean the use of the classical theory of inference. I now take up that part of the story.

The Early Textbooks on “Econometrics”

Prior to the early 1950s, there were, at least in the opinion of those enthusiastic about the Cowles econometric program, no adequate textbooks teaching the new econometrics. No one was teaching the material in any way to graduate students at Harvard or Yale; interested Harvard students studied Cowles monographs on their own. ^{Footnote 17} The first two textbooks teaching the new econometrics, including the use of techniques of inference based on the classical theory, were Gerhard Tintner’s Econometrics, published in 1952, and Lawrence Klein’s Textbook of Econometrics, which appeared the following year. Both Klein and Tintner had been affiliated with the Cowles Commission in the 1940s.

As I have noted, prominent empirical economists of the 1920s and 1930s considered and rejected the idea that the phrase “statistical inference” referred specifically to the use of inferential methods derived from probability theory, with some other phrase, like “statistical induction,” to be applied to other approaches to drawing conclusions from statistical data. However, it is clear from the first pages of “The Probability Approach” (1944) that Haavelmo accepted the now-standard identification of the phrase “statistical inference” with applications of probability theory, and both Tintner and Klein did likewise in their books.

Klein’s section on statistical inference—which he defined as “a method of inferring population characteristics on the basis of samples of information”—was a review of the basics of the classical theory of inference. The concept of the unbiased, minimum variance estimator was offered as an ideal. The construction of confidence intervals using the sample variance of an estimator was explained, as were the fundamentals of Neyman–Pearson hypothesis testing. In subsequent chapters, readers were shown how to apply these tools when estimating linear regression and simultaneous equations models. The reliability of forecasts from time series models was discussed almost entirely in terms of sampling error and the variance of the regression residual, although it was allowed that “structural change” might cause a forecast to be inaccurate, and that “a well rounded econometric approach to forecasting” would take outside information into account (Klein Reference Klein1953b, pp. 50–56, 252–264). Tintner also taught students how to apply the classical inferential tools in the context of regression analysis, although, somewhat surprisingly, the estimation of a simultaneous equations system was not discussed.

After a lull of about a decade, there appeared another set of econometrics texts destined for wide adoption, books by Arthur Goldberger (Reference Goldberger1963), Jack Johnston (Reference Johnston1963), and Edmond Malinvaud (Reference Malinvaud1966). In these books, which in many ways would set the pattern for subsequent twentieth-century graduate econometrics texts, statistical inference in economics meant an application of the classical theory of inference. ^{Footnote 18}

Both Malinvaud and Goldberger presented Haavelmo’s conceptualization of the sample-population relationship underlying statistical inference in terms of learning about the laws governing a stochastic process that generated the sample data. The econometrician’s stochastic model, which was derived from economic theory, reflected the basic form taken by these general laws. Malinvaud explained that statistical inference “proper” never questioned this model, but that in practice, economic theory lacked the precision to provide the econometrician with a framework he could rely upon—more than one model framework might be plausible. Typically, there was no “strictly inductive procedure” for choosing among models in such cases. It was unfortunate but true that the choice would have to be based on “an analysis in which our empirical knowledge will be less rigorously taken into account”: for example, applying the inferential tools to many different data sets to eliminate frameworks that did not seem to agree with the facts (Malinvaud Reference Malinvaud1966, pp. 64–65, 135).

Like Malinvaud, Goldberger argued that statistical inference proper began after the model was specified; it was only during model specification that a priori assumptions or information beyond the sample played a role in empirical analysis. Note the contrast between this conception of how theory and background information should be integrated into empirical analysis and Elmer Working’s view of the process of statistical inference. Like Goldberger, Working believed researchers should rely on theory in determining the relationships to be estimated, but Working saw a further role for theory and background knowledge after estimation, as an aid to interpreting the results and to diagnosing whether the estimates actually reflected stable cause-and-effect relationships.

Graduate Instruction in Econometrics

I believe that the early 1960s, in part due to the appearance of these textbooks, mark something of a turning point in the pedagogy of statistics/econometrics, after which graduate students in economics would routinely be taught to understand statistical estimation and inference in terms of the classical theory, whether in the context of a Cowles-style presentation of simultaneity, identification, and so forth, or simply the more prosaic instruction in testing regression coefficients for “statistical significance” and in constructing confidence intervals. ^{Footnote 19}

In the 1950s, however, the amount of systematic instruction in the application of the classical theory of inference available to economics graduate students depended on where they were being trained. For example, Frederick Mills was still teaching statistics to the Columbia graduate students in the 1950s, and it was not until 1962 that a departmental committee at Columbia recommended that econometrics be offered as a field for graduate students (Rutherford Reference Rutherford2004).

At the University of Wisconsin, graduate students in the 1940s were taught statistics by professors from the university’s School of Commerce in a fashion that Milton Friedman described as “inadequate” in an evaluation requested by a Wisconsin faculty member (Lampman Reference Lampman1993, p. 118). In 1947, the department hired Martin Bronfenbrenner, a student of Henry Schultz, with the expectation that he would teach statistics to graduate students. Bronfenbrenner appears to have taught the course only on an irregular basis during the 1950s, and a look at his own research would suggest that he offered students instruction in the classical inferential methods as one of many aids to statistical inference, probably not the most important. ^{Footnote 20} The picture at Wisconsin changed dramatically with the arrival of Guy Orcutt in 1958 and Arthur Goldberger in 1959, both strong advocates of the use of classical theory of inference (Lampman Reference Lampman1993, pp. 131, 171, 229). In the mid-1950s, before he came to Wisconsin, Goldberger was teaching econometrics to graduate students at Stanford, and Goldberger himself was trained in the early 1950s at the University of Michigan by Lawrence Klein (Lodewijks Reference Lodewijks2005; Kiefer Reference Kiefer1989).

At the University of Chicago, home of the Cowles Commission until 1955, the situation was complicated. Students could learn Cowles-style econometrics from teachers like Henri Thiel, Haavelmo, and Zvi Griliches. But these methods were eschewed for actual empirical work. Among Griliches’s students in the early 1960s was future econometrician Gangadharrao S. Maddala, who considered writing a dissertation in econometric theory before opting for a more empirically oriented topic. Maddala reports that neither he nor any other student in his cohort who was writing an empirical dissertation used anything more complicated than ordinary least squares regression (Krueger and Taylor Reference Krueger and Taylor2000; Lahiri Reference Lahiri1999).

Statistical Inference without Probability in the 1950s and 1960s: Some Important Examples

Another type of evidence on the timing of the emergence of a consensus behind the classical theory of inference is negative in nature: that is, examples of important empirical research studies of the 1950s and early 1960s that did not involve the classical tools of inference in any important way. The researchers involved were assessing the reliability of estimates, were drawing conclusions from data, were asserting and testing hypotheses, but they were not using the tools of the classical theory of inference to do so.

In 1957, Milton Friedman published his theory of the consumption function. Friedman certainly understood statistical theory and probability theory as well as anyone in the profession in the 1950s, and he used statistical theory to derive testable hypotheses from his economic model. But one will search his book almost in vain for applications of the classical methods of inference. ^{Footnote 21} Six years later, Friedman and Anna Schwartz published their Monetary History of the United States, a work packed with graphs and tables of statistical data, and generalizations based on that data. But the book contains no classical hypothesis tests, no confidence intervals, and no reports of statistical significance.

A colleague of Friedman’s at Chicago, and a student of Henry Schultz, H. Gregg Lewis was a member of the research staff of the Cowles Commission in the early 1940s. During the 1950s, Lewis worked with a number of students writing dissertations on the impact of labor unions on workers’ wages, measuring what came to be known as the ‘union/non-union wage gap.’ In the late 1950s, he set out to reconcile the wage-gap estimates from these and other studies. The resulting book, Unionism and Relative Wages in the United States, appeared in 1963. Using basic neoclassical reasoning and the statistical theory of regression, Lewis attempted to determine when differences in measured wage gaps were due to biases related to differences of data and method, and when they could be linked to economic factors. Lewis made little use of standard errors or significance testing in drawing his conclusions. For example, he would sometimes report a range for the likely value of the wage gap in a particular time period or labor market, but these ranges were not the confidence intervals of classical inference theory. Instead, they were Lewis’s judgments of the likely size of biases due to inadequacies of the data. Sampling error, the source of imprecision in estimation with which classical inferential methods were designed to deal, was hardly mentioned.

The human capital research program was also born at the University of Chicago in the late 1950s and early 1960s, with the key contributions being made by Theodore Schultz, Gary Becker, and Jacob Mincer. Theodore Schultz was the veteran of the three, and although he had written one of the early papers in the economics literature explaining Fisher-style hypothesis testing (Schultz Reference Schultz1933), he had since lost confidence in more complex empirical methods, and used cross-tabulations and estimation techniques borrowed from national income accounting to measure key human capital concepts and to establish general patterns. The young Gary Becker’s 1965 book, Human Capital, contained a fair amount of statistical data, some meant to test hypotheses derived from the book’s theoretical models. But Becker did not use the tools of classical inference theory. The empirical innovator of the three was Jacob Mincer, who, among other things, developed the “earnings regression” approach to measuring the rate of return to education. Mincer used probability theory as a modeling tool in his work on human capital theory. But the tools of classical inference theory played little role in his early empirical work on human capital, or, for that matter, in his 1974 magnum opus, Schooling, Experience, and Earnings (Mincer Reference Mincer1958, Reference Mincer1962, Reference Mincer1974).

Statistical inference without probability was not just a Chicago phenomenon in this period. Economists trained in the final decades of the twentieth century are familiar with Okun’s Law, an empirical generalization concerning the relationship between changes in unemployment and changes in the GNP. Arthur Okun, a policy-oriented macroeconomist, first presented the law and the empirical evidence behind it in 1962, and it was quickly accepted as a basic macroeconomic fact. The important point for my purpose, however, is the nature of the evidence he presented: results from three alternative specifications of a regression involving unemployment and GNP. There were no confidence intervals reported for the regression coefficient(s) that embodied the law, and neither was their “statistical significance” mentioned. ^{Footnote 22}

Then there is Simon Kuznets. In his 1949 Presidential Address to the American Statistical Association, Kuznets emphasized that the data used by social scientists were generated by social processes rather than by controlled experiments, and he discussed which methods were and were not well suited to the analysis of such data. And clearly on Kuznets’s “not well suited” list were the classical theory of inference and the Cowles Commission style of econometrics.

For example, Kuznets argued that properties of measurement errors in social data differed across data sets and across variables within data sets, so they could not be dealt with by any “broad and standard scheme.” At a minimum, one needed particular knowledge of the process behind each measurement. Further, recently developed inferential methods derived from probability theory presupposed that the analyst understood, and could control for, systematic patterns in the data. Yet, it was only through the analysis of data that we could gain knowledge of these patterns, or develop hypotheses regarding them. It was tempting, but inappropriate, to assume such knowledge at the beginning of the analysis in order to employ “the elegant apparatus developed for an entirely different situation” (Kuznets Reference Kuznets1950, p. 8). Kuznets said the same thing more pointedly a year later, arguing that statistical analysis in economics should be “divorced from the rather mechanistic approach that employs mathematical models as hasty simulacra of controlled experimentation (which they are not)” (Kuznets Reference Kuznets1951, p. 277).

At this time, Kuznets was beginning a major empirical research program on the causes of economic growth. Over the next two decades, he would analyze a great deal of statistical data, proposing hypotheses and drawing conclusions with varying degrees of confidence, including his conjecture of an inverted u-shaped relationship between a nation’s level of economic development and its level of income inequality, which became known as the “Kuznets curve.” And his analysis did not involve, and neither did his conclusions rest on, classical hypothesis tests.

My final example of an important empirical research program that made no use of the classical theory of inference is Wassily Leontief’s input-output analysis. During the 1940s and 1950s, there was great enthusiasm for input-output analysis among young, mathematically oriented economists who were rising to positions of prominence in the profession, partly because Leontief’s work provided a template for using theoretical models expressed as linear simultaneous equation systems as tools for empirical analysis and forecasting. ^{Footnote 23} But, as he made clear in the early 1950s, Leontief differed from many of his admirers on one point: Cowles-style econometrics and the classical theory of inference were not an important part of his vision of the best way forward for an empirically based economics.

To implement his model empirically, Leontief first needed to assign empirical values to its parameters, the input-output coefficients. He described his approach to this estimation task as “direct observation,” contrasting it with what he called the “indirect statistical inference” employed by “the modern school of statistical econometricians” (Leontief Reference Leontief1952, pp. 2–3, 7). He argued that the versions of the classical inferential techniques developed by the econometricians simply could not do the tasks being asked of them. In the search for the large samples required to make their methods valid, they had been driven towards what he called the “treacherous shoals of time series analysis,” where, in order to get a time series long enough to permit valid inference in the presence of autocorrelation, they were forced to assume “invariance in relationships which actually do change . . . over time.” The work of the modern econometricians was original and imaginative, but not the best use of the profession’s resources (Leontief Reference Leontief1952, pp. 2–3).

Leontief used his empirically specified model to estimate unknown price and quantity relationships and to forecast the effects of hypothetical economic changes, so he still faced the inferential problem of assessing the reliability of such estimates and forecasts, or, more correctly, of the empirical model that had produced them. He and his followers developed various techniques for doing so, but none were based in probability theory (e.g., Leontief Reference Leontief1951, pp. 216–218; Holzman in Leontief Reference Leontief1953).

Cowles econometricians predictably took issue with Leontief’s rejection of “indirect inference.” Both Lawrence Klein (Reference Klein1953a) and Leonid Hurwicz (Reference Hurwicz1955) reviewed the collection of input-output studies that included Leontief’s essay quoted above, and both found much to praise. However, Klein warned that without the use of probability theory to provide “scientific bases for separating good results from bad, … the path of empirical research in input-output study was dangerously close to the realm of arbitrary personal judgment” (Klein Reference Klein1953a, p. 262). Hurwicz also worried that, lacking the discipline imposed by the classical methods, the inferences drawn by the input-output researchers were prone to be influenced by the “author’s state of mind,” and he hoped for research that would reveal complementarities between Leontief’s empirical methods and those of the econometricians (Hurwicz Reference Hurwicz1955, pp. 635–636).

As it happens, input-output analysis has remained a form of empirical research with little connection to the classical theory of inference, although it is no longer a research tool employed at the frontiers of the profession (Biddle and Hamermesh Reference Biddle and Hamermesh2016). However, if you look at the post-1970 empirical research that built on the various other examples of inference without probability that I have offered, you will find classical hypothesis tests and indications of statistical significance taking a prominent place in the discussion of the empirical results.