Functional extended redundancy analysis (FERA) was recently developed to integrate data reduction into functional linear models. This technique extracts a component from each of multiple sets of predictor data in such a way that the component accounts for the maximum variance of response data. Moreover, it permits predictor and/or response data to be functional. FERA can be of use in describing overall characteristics of each set of predictor data and in summarizing the relationships between predictor and response data. In this paper, we extend FERA into the framework of generalized linear models (GLM), so that it can deal with response data generated from a variety of distributions. Specifically, the proposed method reduces each set of predictor functions to a component and uses the component for explaining exponential-family responses. As in GLM, we specify the random, systematic, and link function parts of the proposed method. We develop an iterative algorithm to maximize a penalized log-likelihood criterion that is derived in combination with a basis function expansion approach. We conduct two simulation studies to investigate the performance of the proposed method based on synthetic data. In addition, we apply the proposed method to two examples to demonstrate its empirical usefulness.

A penalized likelihood (PL) method for structural equation modeling (SEM) was proposed as a methodology for exploring the underlying relations among both observed and latent variables. Compared to the usual likelihood method, PL includes a penalty term to control the complexity of the hypothesized model. When the penalty level is appropriately chosen, the PL can yield an SEM model that balances the model goodness-of-fit and model complexity. In addition, the PL results in a sparse estimate that enhances the interpretability of the final model. The proposed method is especially useful when limited substantive knowledge is available for model specifications. The PL method can be also understood as a methodology that links the traditional SEM to the exploratory SEM (Asparouhov & Muthén in Struct Equ Model Multidiscipl J 16:397–438, 2009). An expectation-conditional maximization algorithm was developed to maximize the PL criterion. The asymptotic properties of the proposed PL were also derived. The performance of PL was evaluated through a numerical experiment, and two real data illustrations were presented to demonstrate its utility in psychological research.

Penalized factor analysis is an efficient technique that produces a factor loading matrix with many zero elements thanks to the introduction of sparsity-inducing penalties within the estimation process. However, sparse solutions and stable model selection procedures are only possible if the employed penalty is non-differentiable, which poses certain theoretical and computational challenges. This article proposes a general penalized likelihood-based estimation approach for single- and multiple-group factor analysis models. The framework builds upon differentiable approximations of non-differentiable penalties, a theoretically founded definition of degrees of freedom, and an algorithm with integrated automatic multiple tuning parameter selection that exploits second-order analytical derivative information. The proposed approach is evaluated in two simulation studies and illustrated using a real data set. All the necessary routines are integrated into the R package penfa.

Group-level variance estimates of zero often arise when fitting multilevel or hierarchical linear models, especially when the number of groups is small. For situations where zero variances are implausible a priori, we propose a maximum penalized likelihood approach to avoid such boundary estimates. This approach is equivalent to estimating variance parameters by their posterior mode, given a weakly informative prior distribution. By choosing the penalty from the log-gamma family with shape parameter greater than 1, we ensure that the estimated variance will be positive. We suggest a default log-gamma(2,λ) penalty with λ→0, which ensures that the maximum penalized likelihood estimate is approximately one standard error from zero when the maximum likelihood estimate is zero, thus remaining consistent with the data while being nondegenerate. We also show that the maximum penalized likelihood estimator with this default penalty is a good approximation to the posterior median obtained under a noninformative prior.

Our default method provides better estimates of model parameters and standard errors than the maximum likelihood or the restricted maximum likelihood estimators. The log-gamma family can also be used to convey substantive prior information. In either case—pure penalization or prior information—our recommended procedure gives nondegenerate estimates and in the limit coincides with maximum likelihood as the number of groups increases.

Rating scales as predictors in regression models are typically treated as metrically scaled variables or, alternatively, are coded in dummy variables. The first approach implies a scale level that is not justified, the latter approach results in a large number of parameters to be estimated. Therefore, when rating scales are dummy-coded, applications are often restricted to the use of a few predictors. The penalization approach advocated here takes the scale level serious by using only the ordering of categories but is shown to work in the high dimensional case. We consider the proper modeling of rating scales as predictors and selection procedures by using penalization methods that are tailored to ordinal predictors. In addition to the selection of predictors, the clustering of categories is investigated. Existing methodology is extended to the wider class of generalized linear models. Moreover, higher order differences that allow shrinkage towards a polynomial as well as monotonicity constraints and alternative penalties are introduced. The proposed penalization approaches are illustrated by use of the Motivational States Questionnaire.

Finite mixture models are widely used in the analysis of growth trajectory data to discover subgroups of individuals exhibiting similar patterns of behavior over time. In practice, trajectories are usually modeled as polynomials, which may fail to capture important features of the longitudinal pattern. Focusing on dichotomous response measures, we propose a likelihood penalization approach for parameter estimation that is able to capture a variety of nonlinear class mean trajectory shapes with higher precision than maximum likelihood estimates. We show how parameter estimation and inference for whether trajectories are time-invariant, linear time-varying, or nonlinear time-varying can be carried out for such models. To illustrate the method, we use simulation studies and data from a long-term longitudinal study of children at high risk for substance abuse.

Dynamic network models describe many important scientific processes, from cell biology and epidemiology to sociology and finance. Estimating dynamic networks from noisy time series data is a difficult task since the number of components involved in the system is very large. As a result, the number of parameters to be estimated is typically larger than the number of observations. However, a characteristic of many real life networks is that they are sparse. For example, the molecular structure of genes make interactions with other components a highly-structured and, therefore, a sparse process. Until now, the literature has focused on static networks, which lack specific temporal interpretations.

We propose a flexible collection of ANOVA-like dynamic network models, where the user can select specific time dynamics, known presence or absence of links, and a particular autoregressive structure. We use undirected graphical models with block equality constraints on the parameters. This reduces the number of parameters, increases the accuracy of the estimates and makes interpretation of the results more relevant. We illustrate the flexibility of the method on both synthetic and real data.

We propose a general partition-based strategy to estimate conditional density with candidate densities that are piecewise constant with respect to the covariate. Capitalizing on a general penalized maximum likelihood model selection result, we prove, on two specific examples, that the penalty of each model can be chosen roughly proportional to its dimension. We first study a classical strategy in which the densities are chosen piecewise conditional according to the variable. We then consider Gaussian mixture models with mixing proportion that vary according to the covariate but with common mixture components. This model proves to be interesting for an unsupervised segmentation application that was our original motivation for this work.

Even for a well-trained statistician the construction of a histogramfor a given real-valued data set is a difficult problem. It is evenmore difficult to construct a fully automatic procedure whichspecifies the number and widths of the bins in a satisfactory mannerfor a wide range of data sets. In this paper we compare severalhistogram construction procedures by means of a simulationstudy. The study includes plug-in methods, cross-validation,penalized maximum likelihood and the taut string procedure. Their performance on different test beds is measured by their ability to identify the peaks of an underlying density as well as by Hellinger distance.

Hypotheses concerning the diversification of notothenioid fishes have relied extensively on estimates of divergence times using molecular clock methods. The timing of diversification of the cold adapted antifreeze glycoprotein (AFGP)-bearing Antarctic notothenioid clade in the middle to late Miocene has been correlated with the onset of polar climatic conditions along the Antarctic Continental Shelf. Critical examination of the previous molecular clock analyses of notothenioids reveals several problems associated with heterogeneity of nucleotide substitution rates among lineages, the application of potentially inappropriate nucleotide substitution rates, and the lack of confidence intervals for divergence time estimates. In this study, the notothenioid partial gene mtDNA 12S-16S rRNA (PG-rRNA) molecular clock was reanalysed using a tree-based maximum likelihood strategy that attempts to account for rate heterogeneity of nucleotide substitution rates among lineages using the penalized likelihood method, and bootstrap resampling to estimate confidence intervals of divergence time estimates. The molecular clock was calibrated using the notothenioid fossil Proeleginops grandeastmanorum. Divergence time estimates for all nodes in the PG-rRNA maximum likelihood tree were substantially older than previous estimates. In particular, the estimated age of the AFGP-bearing Antarctic notothenioid clade predates the onset of extensive sea ice and development of polar conditions by at least 10 million years. Despite caveats involving the fossil calibration and limitations of the PG-rRNA dataset, these divergence time estimates provide initial observations for the development of a novel model of the diversification of cold adapted Antarctic notothenioid fishes.