Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator’s consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.

Diagnostic classification models (DCMs) are widely used for providing fine-grained classification of a multidimensional collection of discrete attributes. The application of DCMs requires the specification of the latent structure in what is known as the $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document} matrix. Expert-specified $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document} matrices might be biased and result in incorrect diagnostic classifications, so a critical issue is developing methods to estimate $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document} in order to infer the relationship between latent attributes and items. Existing exploratory methods for estimating $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document} must pre-specify the number of attributes, K. We present a Bayesian framework to jointly infer the number of attributes K and the elements of $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document}. We propose the crimp sampling algorithm to transit between different dimensions of K and estimate the underlying $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document} and model parameters while enforcing model identifiability constraints. We also adapt the Indian buffet process and reversible-jump Markov chain Monte Carlo methods to estimate $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${\varvec{Q}}$$\end{document}. We report evidence that the crimp sampler performs the best among the three methods. We apply the developed methodology to two data sets and discuss the implications of the findings for future research.

Penalized factor analysis is an efficient technique that produces a factor loading matrix with many zero elements thanks to the introduction of sparsity-inducing penalties within the estimation process. However, sparse solutions and stable model selection procedures are only possible if the employed penalty is non-differentiable, which poses certain theoretical and computational challenges. This article proposes a general penalized likelihood-based estimation approach for single- and multiple-group factor analysis models. The framework builds upon differentiable approximations of non-differentiable penalties, a theoretically founded definition of degrees of freedom, and an algorithm with integrated automatic multiple tuning parameter selection that exploits second-order analytical derivative information. The proposed approach is evaluated in two simulation studies and illustrated using a real data set. All the necessary routines are integrated into the R package penfa.

We propose a two-step procedure to estimate structural equation models (SEMs). In a first step, the latent variable is replaced by its conditional expectation given the observed data. This conditional expectation is estimated using a James–Stein type shrinkage estimator. The second step consists of regressing the dependent variables on this shrinkage estimator. In addition to linear SEMs, we also derive shrinkage estimators to estimate polynomials. We empirically demonstrate the feasibility of the proposed method via simulation and contrast the proposed estimator with ML and MIIV estimators under a limited number of simulation scenarios. We illustrate the method on a case study.

In this paper, we consider the Rasch model and suggest novel point estimators and confidence intervals for the ability parameter. They are based on a proposed confidence distribution (CD) whose construction has required to overcome some difficulties essentially due to the discrete nature of the model. When the number of items is large, the computations due to the combinatorics involved become heavy, and thus, we provide first- and second-order approximations of the CD. Simulation studies show the good behavior of our estimators and intervals when compared with those obtained through other standard frequentist and weakly informative Bayesian procedures. Finally, using the expansion of the expected length of the suggested interval, we are able to identify reasonable values of the sample size which lead to a desired length of the interval.

Cognitive diagnostic models (CDMs) have arisen as advanced psychometric models in the past few decades for assessments that intend to measure students’ mastery of a set of attributes. Recently, quite a few studies attempted to extend CDMs to longitudinal versions, and they all tended to model transition probabilities from non-mastery to mastery or vice versa for each attribute separately, with an exception of a few studies (e.g., Chen et al. 2018; Madison & Bradshaw 2018). However, these pioneering works have not taken into consideration the attribute relationships and the ever-changing attributes in a learning period. In this paper, we consider a profile-level latent transition CDM (TCDM hereafter), which can not only identify transition probabilities across the same attributes over time, but also the transition pathways across different attributes. Two versions of the penalized expectation-maximization (PEM) algorithms are proposed to shrink the probabilities associated with impermissible transition pathways to 0 and, thereby, help explore attribute relationships in a longitudinal setting. Simulation results reveal that PEM with group penalty holds great promise for identifying learning trajectories.

Complex interactive test items are becoming more widely used in assessments. Being computer-administered, assessments using interactive items allow logging time-stamped action sequences. These sequences pose a rich source of information that may facilitate investigating how examinees approach an item and arrive at their given response. There is a rich body of research leveraging action sequence data for investigating examinees’ behavior. However, the associated timing data have been considered mainly on the item-level, if at all. Considering timing data on the action-level in addition to action sequences, however, has vast potential to support a more fine-grained assessment of examinees’ behavior. We provide an approach that jointly considers action sequences and action-level times for identifying common response processes. In doing so, we integrate tools from clickstream analyses and graph-modeled data clustering with psychometrics. In our approach, we (a) provide similarity measures that are based on both actions and the associated action-level timing data and (b) subsequently employ cluster edge deletion for identifying homogeneous, interpretable, well-separated groups of action patterns, each describing a common response process. Guidelines on how to apply the approach are provided. The approach and its utility are illustrated on a complex problem-solving item from PIAAC 2012.

Issues of model selection have dominated the theoretical and applied statistical literature for decades. Model selection methods such as ridge regression, the lasso, and the elastic net have replaced ad hoc methods such as stepwise regression as a means of model selection. In the end, however, these methods lead to a single final model that is often taken to be the model considered ahead of time, thus ignoring the uncertainty inherent in the search for a final model. One method that has enjoyed a long history of theoretical developments and substantive applications, and that accounts directly for uncertainty in model selection, is Bayesian model averaging (BMA). BMA addresses the problem of model selection by not selecting a final model, but rather by averaging over a space of possible models that could have generated the data. The purpose of this paper is to provide a detailed and up-to-date review of BMA with a focus on its foundations in Bayesian decision theory and Bayesian predictive modeling. We consider the selection of parameter and model priors as well as methods for evaluating predictions based on BMA. We also consider important assumptions regarding BMA and extensions of model averaging methods to address these assumptions, particularly the method of Bayesian stacking. Simple empirical examples are provided and directions for future research relevant to psychometrics are discussed.

In this paper, we propose a joint modeling approach to analyze dependency in parallel response data. We define two types of dependency: higher-level dependency and within-item conditional dependency. While higher-level dependency can be estimated with common latent variable modeling approaches, within-item conditional dependency is a unique kind of information that is often not captured with extant methods, despite its potential to shed new insights into the relationship between the two types of response data. We differentiate three ways of modeling within-item conditional dependency by conditioning on raw values, expected values, or residual values of the response data, which have different implications in terms of response processes. The proposed approach is illustrated with the example of analyzing parallel data on response accuracy and brain activations from a Theory of Mind assessment. The consequence of ignoring within-item conditional dependency is investigated with empirical and simulation studies in comparison to conventional dependency analysis that focuses exclusively on relationships between latent variables.

A social network comprises both actors and the social connections among them. Such connections reflect the dependence among social actors, which is essential for individuals’ mental health and social development. In this article, we propose a mediation model with a social network as a mediator to investigate the potential mediation role of a social network. In the model, the dependence among actors is accounted for by a few mutually orthogonal latent dimensions which form a social space. The individuals’ positions in such a latent social space are directly involved in the mediation process between an independent and dependent variable. After showing that all the latent dimensions are equivalent in terms of their relationship to the social network and the meaning of each dimension is arbitrary, we propose to measure the whole mediation effect of a network. Although individuals’ positions in the latent space are not unique, we rigorously articulate that the proposed network mediation effect is still well defined. We use a Bayesian estimation method to estimate the model and evaluate its performance through an extensive simulation study under representative conditions. The usefulness of the network mediation model is demonstrated through an application to a college friendship network.

This study revisits the parameter estimation issues in multidimensional item response theory more thoroughly and investigates some computation details that have seldom been addressed previously when implementing the expectation-maximization (EM) algorithm for finite mixtures (EM–FM). Two research questions are: Should we rescale after each EM cycle or after the final EM cycle? How to adapt the supplemented EM algorithm to the EM–FM framework to estimate standard errors (SEs) of all unknown parameters? Analytic details of the methods are provided, and a comprehensive simulation study is conducted to provide supporting evidence. Results reveal that rescaling after each EM cycle accelerates convergence without affecting the calibration accuracy. Moreover, the SEs of all model parameters, including item parameters and population mixing proportions, recover well when the sample size is relatively large (e.g., 2000).

In this article, we present the findings of an oral history project on the past, present, and future of psychometrics, as obtained through structured interviews with twenty past Psychometric Society presidents. Perspectives on how psychometrics should be practiced vary strongly. Some presidents are psychology-oriented, whereas others have a more mathematical or statistical approach. The originally strong relationship between psychometrics and psychology has weakened, and contemporary psychometrics has become a diverse and multifaceted discipline. The presidents are confident psychometrics will continue to be relevant but believe psychometrics needs to become better at selling its strong points to relevant research areas. We recommend for psychometrics to cherish its plurality and make its goals and priorities explicit.