Lyu et al. (Psychometrika, 2023) demonstrated that item-specific factors can cause spurious effects on the structural parameters of IRTree models for multiple nested response processes per item. Here, we discuss some boundary conditions and argue that person selection effects on item parameters are not unique to item-specific factors and that the effects presented by Lyu et al. (Psychometrika, 2023) may not generalize to the family of IRTree models as a whole. We conclude with the recommendation that IRTree model specification should be guided by theoretical considerations, rather than driven by data, in order to avoid misinterpretations of parameter differences.

Test items for which the item score reflects a sequential or IRTree modeling outcome are considered. For such items, we argue that item-specific factors, although not empirically measurable, will often be present across stages of the same item. In this paper, we present a conceptual model that incorporates such factors. We use the model to demonstrate how the varying conditional distributions of item-specific factors across stages become absorbed into the stage-specific item discrimination and difficulty parameters, creating ambiguity in the interpretations of item and person parameters beyond the first stage. We discuss implications in relation to various applications considered in the literature, including methodological studies of (1) repeated attempt items; (2) answer change/review, (3) on-demand item hints; (4) item skipping behavior; and (5) Likert scale items. Our own empirical applications, as well as several examples published in the literature, show patterns of violations of item parameter invariance across stages that are highly suggestive of item-specific factors. For applications using sequential or IRTree models as analytical models, or for which the resulting item score might be viewed as outcomes of such a process, we recommend (1) regular inspection of data or analytic results for empirical evidence (or theoretical expectations) of item-specific factors; and (2) sensitivity analyses to evaluate the implications of item-specific factors for the intended inferences or applications.

Factor copula models for item response data are more interpretable and fit better than (truncated) vine copula models when dependence can be explained through latent variables, but are not robust to violations of conditional independence. To circumvent these issues, truncated vines and factor copula models for item response data are joined to define a combined model, the so-called factor tree copula model, with individual benefits from each of the two approaches. Rather than adding factors and causing computational problems and difficulties in interpretation and identification, a truncated vine structure is assumed on the residuals conditional on one or two latent variables. This structure can be better explained as a conditional dependence given a few interpretable latent variables. On the one hand, the parsimonious feature of factor models remains intact and any residual dependencies are being taken into account on the other. We discuss estimation along with model selection. In particular, we propose model selection algorithms to choose a plausible factor tree copula model to capture the (residual) dependencies among the item responses. Our general methodology is demonstrated with an extensive simulation study and illustrated by analyzing Post-Traumatic Stress Disorder.

Item response tree (IRTree) models are widely used in various applications for their ability to differentiate sets of sub-responses from polytomous item response data based on a pre-specified tree structure. Lyu et al. (Psychometrika) article highlighted that item slopes are often lower for later nodes than earlier nodes in IRTree applications. Lyu et al. argued that this phenomenon might signal the presence of item-specific factors across nodes. In this commentary, I present a different perspective that conditional dependencies in IRTree data could explain the phenomenon more generally. I illustrate my point with an empirical example, utilizing the latent space item response model that visualizes conditional dependencies in IRTree data. I conclude the commentary with a discussion on the potential of exploring conditional dependencies in IRTree data that goes beyond identifying the sources of conditional dependencies.

The present article proposes and evaluates marginal maximum likelihood (ML) estimation methods for hierarchical multinomial processing tree (MPT) models with random and fixed effects. We assume that an identifiable MPT model with S parameters holds for each participant. Of these S parameters, R parameters are assumed to vary randomly between participants, and the remaining $S-R$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$S-R$$\end{document} parameters are assumed to be fixed. We also propose an extended version of the model that includes effects of covariates on MPT model parameters. Because the likelihood functions of both versions of the model are too complex to be tractable, we propose three numerical methods to approximate the integrals that occur in the likelihood function, namely, the Laplace approximation (LA), adaptive Gauss–Hermite quadrature (AGHQ), and Quasi Monte Carlo (QMC) integration. We compare these three methods in a simulation study and show that AGHQ performs well in terms of both bias and coverage rate. QMC also performs well but the number of responses per participant must be sufficiently large. In contrast, LA fails quite often due to undefined standard errors. We also suggest ML-based methods to test the goodness of fit and to compare models taking model complexity into account. The article closes with an illustrative empirical application and an outlook on possible extensions and future applications of the proposed ML approach.

Traditional measurement models assume that all item responses correlate with each other only through their underlying latent variables. This conditional independence assumption has been extended in joint models of responses and response times (RTs), implying that an item has the same item characteristics fors all respondents regardless of levels of latent ability/trait and speed. However, previous studies have shown that this assumption is violated in various types of tests and questionnaires and there are substantial interactions between respondents and items that cannot be captured by person- and item-effect parameters in psychometric models with the conditional independence assumption. To study the existence and potential cognitive sources of conditional dependence and utilize it to extract diagnostic information for respondents and items, we propose a diffusion item response theory model integrated with the latent space of variations in information processing rate of within-individual measurement processes. Respondents and items are mapped onto the latent space, and their distances represent conditional dependence and unexplained interactions. We provide three empirical applications to illustrate (1) how to use an estimated latent space to inform conditional dependence and its relation to person and item measures, (2) how to derive diagnostic feedback personalized for respondents, and (3) how to validate estimated results with an external measure. We also provide a simulation study to support that the proposed approach can accurately recover its parameters and detect conditional dependence underlying data.

Applications of structural equation modeling (SEM) may encounter issues like inadmissible parameter estimates, nonconvergence, or unsatisfactory model fit. We propose a new factor rotation method that reparameterizes the factor correlation matrix in exploratory factor analysis (EFA) such that factors can be either exogenous or endogenous. The proposed method is an oblique rotation method for EFA, but it allows directional structural paths among factors. We thus referred it to as FSP (factor structural paths) rotation. In particular, we can use FSP rotation to “translate” an SEM model to incorporate theoretical expectations on both factor loadings and structural parameters. We illustrate FSP rotation with an empirical example and explore its statistical properties with simulated data. The results include that (1) EFA with FSP rotation tends to fit data better and encounters fewer Heywood cases than SEM does when there are cross-loadings and many small nonzero loadings, (2) FSP rotated parameter estimates are satisfactory for small models, and (3) FSP rotated parameter estimates are more satisfactory for large models when the structural parameter matrices are sparse.

Standard response formats such as rating or visual analogue scales require respondents to condense distributions of latent states or behaviors into a single value. Whereas this is suitable to measure central tendency, it neglects the variance of distributions. As a remedy, variability may be measured using interval-response formats, more specifically the dual-range slider (RS2). Given the lack of an appropriate item response model for the RS2, we develop the Dirichlet dual response model (DDRM), an extension of the beta response model (BRM; Noel & Dauvier in Appl Psychol Meas, 31:47–73, 2007). We evaluate the DDRM’s performance by assessing parameter recovery in a simulation study. Results indicate overall good parameter recovery, although parameters concerning interval width (which reflect variability in behavior or states) perform worse than parameters concerning central tendency. We also test the model empirically by jointly fitting the BRM and the DDRM to single-range slider (RS1) and RS2 responses for two Extraversion scales. While the DDRM has an acceptable fit, it shows some misfit regarding the RS2 interval widths. Nonetheless, the model indicates substantial differences between respondents concerning variability in behavior. High correlations between person parameters of the BRM and DDRM suggest convergent validity between the RS1 and the RS2 interval location. Both the simulation and the empirical study demonstrate that the latent parameter space of the DDRM addresses an important issue of the RS2 response format, namely, the scale-inherent interdependence of interval location and interval width (i.e., intervals at the boundaries are necessarily smaller).

In this paper, the support of the joint probability distribution of categorical variables in the total population is treated as unknown. From a general total population model with unknown support, a general subpopulation model with its support equal to the set of all observed score patterns is derived. In maximum likelihood estimation of the parameters of any such subpopulation model, the evaluation of the log-likelihood function only requires the summation over a number of terms equal to at most the sample size. It is made clear that the parameters of a hypothesized total population model are consistently and asymptotically efficiently estimated by the values that maximize the log-likelihood function of the corresponding subpopulation model. Next, new likelihood ratio goodness-of-fit tests are proposed as alternatives to the Pearson chi-square goodness-of-fit test and the likelihood ratio test against the saturated model. In a simulation study, the asymptotic bias and efficiency of maximum likelihood estimators and the asymptotic performance of the goodness-of-fit tests are investigated.

This article presents a joint modeling framework of ordinal responses and response times (RTs) for the measurement of latent traits. We integrate cognitive theories of decision-making and confidence judgments with psychometric theories to model individual-level measurement processes. The model development starts with the sequential sampling framework which assumes that when an item is presented, a respondent accumulates noisy evidence over time to respond to the item. Several cognitive and psychometric theories are reviewed and integrated, leading us to three psychometric process models with different representations of the cognitive processes underlying the measurement. We provide simulation studies that examine parameter recovery and show the relationships between latent variables and data distributions. We further test the proposed models with empirical data measuring three traits related to motivation. The results show that all three models provide reasonably good descriptions of observed response proportions and RT distributions. Also, different traits favor different process models, which implies that psychological measurement processes may have heterogeneous structures across traits. Our process of model building and examination illustrates how cognitive theories can be incorporated into psychometric model development to shed light on the measurement process, which has had little attention in traditional psychometric models.

Multi-source functional block-wise missing data arise more commonly in medical care recently with the rapid development of big data and medical technology, hence there is an urgent need to develop efficient dimension reduction to extract important information for classification under such data. However, most existing methods for classification problems consider high-dimensional data as covariates. In the paper, we propose a novel multinomial imputed-factor Logistic regression model with multi-source functional block-wise missing data as covariates. Our main contribution is to establishing two multinomial factor regression models by using the imputed multi-source functional principal component scores and imputed canonical scores as covariates, respectively, where the missing factors are imputed by both the conditional mean imputation and the multiple block-wise imputation approaches. Specifically, the univariate FPCA is carried out for the observable data of each data source firstly to obtain the univariate principal component scores and the eigenfunctions. Then, the block-wise missing univariate principal component scores instead of the block-wise missing functional data are imputed by the conditional mean imputation method and the multiple block-wise imputation method, respectively. After that, based on the imputed univariate factors, the multi-source principal component scores are constructed by using the relationship between the multi-source principal component scores and the univariate principal component scores; and at the same time, the canonical scores are obtained by the multiple-set canonial correlation analysis. Finally, the multinomial imputed-factor Logistic regression model is established with the multi-source principal component scores or the canonical scores as factors. Numerical simulations and real data analysis on ADNI data show the proposed method works well.

Several measures of agreement, such as the Perreault–Leigh coefficient, the ${\mathrm{AC}}_1$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\textsc {AC}_{1}$$\end{document}, and the recent coefficient of van Oest, are based on explicit models of how judges make their ratings. To handle such measures of agreement under a common umbrella, we propose a class of models called guessing models, which contains most models of how judges make their ratings. Every guessing model have an associated measure of agreement we call the knowledge coefficient. Under certain assumptions on the guessing models, the knowledge coefficient will be equal to the multi-rater Cohen’s kappa, Fleiss’ kappa, the Brennan–Prediger coefficient, or other less-established measures of agreement. We provide several sample estimators of the knowledge coefficient, valid under varying assumptions, and their asymptotic distributions. After a sensitivity analysis and a simulation study of confidence intervals, we find that the Brennan–Prediger coefficient typically outperforms the others, with much better coverage under unfavorable circumstances.

We respond to the commentaries on Lyu, Bolt and Westby’s “Exploring the effects of item specific factors in sequential and IRTree models.” The commentaries raise important points that allow us to clarify our theoretical expectation for item specific factors in many educational and psychological test items. At the same time, we agree with the commentaries in acknowledging challenges associated with providing empirical evidence for their presence and reflect on strategies that might support their estimation. We maintain that the principal concern is the ambiguity item specific factors create in attempting to interpret or use the parameters beyond the first node.

In the current paper, we review existing tools for solving variable selection problems in psychology. Modern regularization methods such as lasso regression have recently been introduced in the field and are incorporated into popular methodologies, such as network analysis. However, several recognized limitations of lasso regularization may limit its suitability for psychological research. In this paper, we compare the properties of lasso approaches used for variable selection to Bayesian variable selection approaches. In particular we highlight advantages of stochastic search variable selection (SSVS), that make it well suited for variable selection applications in psychology. We demonstrate these advantages and contrast SSVS with lasso type penalization in an application to predict depression symptoms in a large sample and an accompanying simulation study. We investigate the effects of sample size, effect size, and patterns of correlation among predictors on rates of correct and false inclusion and bias in the estimates. SSVS as investigated here is reasonably computationally efficient and powerful to detect moderate effects in small sample sizes (or small effects in moderate sample sizes), while protecting against false inclusion and without over-penalizing true effects. We recommend SSVS as a flexible framework that is well-suited for the field, discuss limitations, and suggest directions for future development.

Signal detection theory (SDT; Tanner & Swets in Psychological Review 61:401–409, 1954) is a dominant modeling framework used for evaluating the accuracy of diagnostic systems that seek to distinguish signal from noise in psychology. Although the use of response time data in psychometric models has increased in recent years, the incorporation of response time data into SDT models remains a relatively underexplored approach to distinguishing signal from noise. Functional response time effects are hypothesized in SDT models, based on findings from other related psychometric models with response time data. In this study, an SDT model is extended to incorporate functional response time effects using smooth functions and to include all sources of variability in SDT model parameters across trials, participants, and items in the experimental data. The extended SDT model with smooth functions is formulated as a generalized linear mixed-effects model and implemented in the gamm4R package. The extended model is illustrated using recognition memory data to understand how conversational language is remembered. Accuracy of parameter estimates and the importance of modeling variability in detecting the experimental condition effects and functional response time effects are shown in conditions similar to the empirical data set via a simulation study. In addition, the type 1 error rate of the test for a smooth function of response time is evaluated.