This paper analyzes the theoretical, pragmatic, and substantive factors that have hampered the integration between psychology and psychometrics. Theoretical factors include the operationalist mode of thinking which is common throughout psychology, the dominance of classical test theory, and the use of “construct validity” as a catch-all category for a range of challenging psychometric problems. Pragmatic factors include the lack of interest in mathematically precise thinking in psychology, inadequate representation of psychometric modeling in major statistics programs, and insufficient mathematical training in the psychological curriculum. Substantive factors relate to the absence of psychological theories that are sufficiently strong to motivate the structure of psychometric models. Following the identification of these problems, a number of promising recent developments are discussed, and suggestions are made to further the integration of psychology and psychometrics.

I tend to agree with Professor Borsboom that psychology, and more generally the social sciences, could benefit from better psychometric modeling. However, if psychometric developments are to have more effect on everyday practice in psychology, psychometricians probably need to pay more attention to the substantive and methodological problems in various areas of psychology. For example, Professor Borsboom is critical of the Standards for educational and psychological testing (AERA, APA, NCME, 1999) for suggesting that group differences in testcriterion relationships are relevant to test bias. He bases his criticism on the finding that predictive invariance is not the same as measurement invariance. However, he fails to acknowledge the social, political, and ethical problems associated with failures of predictive invariance in high-stakes contexts (e.g., employment and admissions testing).

Borsboom (2006) attacks psychologists for failing to incorporate psychometric advances in their work, discusses factors that contribute to this regrettable situation, and offers suggestions for ameliorating it. This commentary applauds Borsboom for calling the field to task on this issue and notes additional problems in the field regarding measurement that he could add to his critique. It also chastises Borsboom for occasionally being unnecessarily perjorative in his critique, noting that negative rhetoric is unlikely to make converts of offenders. Finally, it exhorts psychometricians to make their work more accessible and points to Borsboom, Mellenbergh, and Van Heerden (2003) as an excellent example of how this can be done.

This is a reaction to Borsboom’s (2006) discussion paper on the issue that psychology takes so little notice of the modern developments in psychometrics, in particular, latent variable methods. Contrary to Borsboom, it is argued that latent variables are summaries of interesting data properties, that construct validation should involve studying nomological networks, that psychological research slowly but definitely will incorporate latent variable methods, and that the role of psychometrics in psychology is that of partner, not role model.

One basic reason that measurement in psychology requires statistics is that psychologists do not use copper instruments anymore, as they used to do in the nineteenth century. Instead, they determine test or total scores on the basis of miniature experiments with discrete outcomes, and use a variety of standard statististical techniques for reaching conclusions on the basis of observed data. Borsboom (2006) wants us to believe that psychologists are seriously misled in their hope that they can make progress this way, and recommends an invasion of psychometricians carrying IRT missiles and SEM guns into psychology.My prediction is that such an invasion would simply be ignored. That is not to say that whenever psychometric modeling really makes a difference, no attempts should be made to reach the mainstream of psychology. Indeed, many psychometric contributions that are obsolete according to Borsboom, like Cronbach’s alpha and exploratory factor analysis, in fact entered into the mainstream of psychology only because they tend to provide sensible answers to real problems, which cannot be easily surpassed.

First, I thank Sijtsma, Clark, Kane, and Heiser for taking the time and effort to provide a commentary on my paper, and the Editor for allowing me to respond to them. In general, the commentators agree with the thesis that psychometrics and psychology are, to an extent that must be deemed problematic, disconnected. They further agree with the upshot of this diagnosis: Psychometricians need to work harder to make a difference in psychology, and psychologists need to develop a greater awareness of important psychometric developments. However, the commentators also raise several points of disagreement and criticism.

Stevens postulated that we can use the responses of a participant in a ratio scaling experiment directly to construct a psychophysical function representing the participant’s sensations. Although Stevens’ methods of constructing measurement scales are widely used in the behavioral sciences, the problem of which scale type is appropriate to describe ratio scaling data is still unresolved. To deal with this problem, we develop a theoretical framework to specify the scale type attained by Stevens’ direct scaling methods. It is shown, under fairly mild background assumptions, that the behavioral axioms presented in this paper are necessary and sufficient for the psychophysical functions to be ordinal-, interval-, log-interval-, or ratio-scales. Furthermore, suggestions on how to test these behavioral axioms are provided.

The Candecomp/Parafac (CP) model decomposes a three-way array into a prespecified number R of rank-1 arrays and a residual array, in which the sum of squares of the residual array is minimized. The practical use of CP is sometimes complicated by the occurrence of so-called degenerate solutions, in which some components are highly correlated in all three modes and the elements of these components become arbitrarily large. We consider the real-valued CP model in which p × p × 2 arrays of rank p + 1 or higher are decomposed into p rank-1 arrays and a residual array. It is shown that the CP objective function does not have a minimum in these cases, but an infimum. Moreover, any sequence of CP approximations, of which the objective value approaches the infimum, will become degenerate. This result extends Ten Berge, Kiers, & De Leeuw (1988), who consider a particular 2 × 2 × 2 array of rank 3.

The assumptions of the model for factor analysis do not exclude a class of indeterminate covariances between factors and error variables (Grayson, 2003). The construction of all factors of the model for factor analysis is generalized to incorporate indeterminate factor-error covariances. A necessary and sufficient condition is given for indeterminate factor-error covariances to be arbitrarily small, for mean square convergence of the regression predictor of factor scores, and for the existence of a unique determinate factor and error variable. The determinate factor and error variable are uncorrelated and satisfy the defining assumptions of factor analysis. Several examples are given to illustrate the results.

The present paper shows that the usual factor analytic structured data dispersion matrix Λ Ψ Λ’ + Δ can readily arise from a set of scores y = Λ η + ε, where the “common” (η) and “unique” (ε) factors have nonzero covariance: Γ = Cov(ε,η) ≠ 0. Implications of this finding are discussed for the indeterminacy of factor scores, and for the issue of invariance of factor analytic covariance models. The size of the problem is explored with numerical examples.

This paper considers the problem of analysis of correlation coefficients from a multivariate normal population. A unified theorem is derived for the regression model with normally distributed explanatory variables and the general results are employed to provide useful expressions for the distributions of simple, multiple, and partial-multiple correlation coefficients. The inversion principle and monotonicity property of the proposed formulations are used to describe alternative approaches to the exact interval estimation, power calculation, and sample size determination for correlation coefficients.

A Bayesian approach is developed for analyzing nonlinear structural equation models with nonignorable missing data. The nonignorable missingness mechanism is specified by a logistic regression model. A hybrid algorithm that combines the Gibbs sampler and the Metropolis–Hastings algorithm is used to produce the joint Bayesian estimates of structural parameters, latent variables, parameters in the nonignorable missing model, as well as their standard errors estimates. A goodness-of-fit statistic for assessing the plausibility of the posited nonlinear structural equation model is introduced, and a procedure for computing the Bayes factor for model comparison is developed via path sampling. Results obtained with respect to different missing data models, and different prior inputs are compared via simulation studies. In particular, it is shown that in the presence of nonignorable missing data, results obtained by the proposed method with a nonignorable missing data model are significantly better than those that are obtained under the missing at random assumption. A real example is presented to illustrate the newly developed Bayesian methodologies.

By means of more than a dozen user friendly packages, structural equation models (SEMs) are widely used in behavioral, education, social, and psychological research. As the underlying theory and methods in these packages are vulnerable to outliers and distributions with longer-than-normal tails, a fundamental problem in the field is the development of robust methods to reduce the influence of outliers and the distributional deviation in the analysis. In this paper we develop a maximum likelihood (ML) approach that is robust to outliers and symmetrically heavy-tailed distributions for analyzing nonlinear SEMs with ignorable missing data. The analytic strategy is to incorporate a general class of distributions into the latent variables and the error measurements in the measurement and structural equations. A Monte Carlo EM (MCEM) algorithm is constructed to obtain the ML estimates, and a path sampling procedure is implemented to compute the observed-data log-likelihood and then the Bayesian information criterion for model comparison. The proposed methodologies are illustrated with simulation studies and an example.

In categorical data analysis, two-sample cross-validation is used not only for model selection but also to obtain a realistic impression of the overall predictive effectiveness of the model. The latter is of particular importance in the case of highly parametrized models capable of capturing every idiosyncracy of the calibrating sample. We show that for maximum likelihood estimators or other asymptotically efficient estimators Pearson’s X2 is not asymptotically chi-square in the two-sample cross-validation framework due to extra variability induced by using different samples for estimation and goodness-of-fit testing. We propose an alternative test statistic, X2xval, obtained as a modification of X2 which is asymptotically chi-square with C - 1 degrees of freedom in cross-validation samples. Stochastically, X2xval≤ X2. Furthermore, the use of X2 instead of X2xval with a χ2C - 1 reference distribution may provide an unduly poor impression of fit of the model in the cross-validation sample.

Several sufficient conditions are available for mean square convergence of factor predictors. A necessary and sufficient condition is given in the Heywood case with respect to (confirmatory) factor analysis. This condition generalizes that of Krijnen (2006) and performs better than a signal-to-noise type of condition (Schneeweiss & Mathes, 1995).