Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-11-26T05:05:44.876Z Has data issue: false hasContentIssue false
  • English
  • Français

Multiple Imputation in Multilevel Models. A Revision of the Current Software and Usage Examples for Researchers

Published online by Cambridge University Press:  12 November 2020

Pablo García-Patos  and
Ricardo Olmos Open the ORCID record for Ricardo Olmos [Opens in a new window]
Pablo García-Patos
Affiliation:
Universidad Autónoma de Madrid (Spain)
Ricardo Olmos
Affiliation:
Universidad Autónoma de Madrid (Spain)
Get access Rights & Permissions [Opens in a new window]

Abstract

Although modern lines for dealing with missing data are well established from the 1970s, today there is a challenge when researchers encounter this problem in multilevel models. First, there is a variety of existing software to handle missing data based on multiple imputation (MI), currently pointed out by experts as the most promising strategy. Second, the two principal paradigms of MI are joint modelling (JM) and fully conditional specification (FCS), one more complication because they are not equally useful depending on the combination of multilevel model and the estimated parameters affected by missing data. Technical literature do not contribute to ease the number of decisions that researcher has to do. Given these inconveniences, the present paper has three objectives. (1) To present a thorough revision of the most recently developed software and functions about multiple imputation in multilevel models. (2) We derive a set of suggestions, recommendations, and guides for helping researchers to handle missing data. We list a number of key questions to consider when analyzing multilevel models. (3) Finally, based on the previous relevant questions, we present two detailed examples using the recommended R packages to be easy for the researcher applying multiple imputation in multilevel models.

Keywords

fully conditional specificationjoint modelingmultilevel modelsmultiple imputation
Type
Research Article
Information
The Spanish Journal of Psychology , Volume 23 , 2020 , e46
Copyright
© Universidad Complutense de Madrid and Colegio Oficial de Psicólogos de Madrid 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Conflicts of Interest: None.

Funding Statement: This research received no specific grant from any funding agency, commercial or not-for-profit sectors.

References

Allison, P.D. (2001). Missing data. Sage. https://doi.org/10.4135/9781412985079Google Scholar
Andridge, R. R. (2011). Quantifying the impact of fixed effects modeling of clusters in multiple imputation for cluster randomized trials. Biometrical Journal, 53(1), 57–74. http://doi.org/10.1002/bimj.201000140CrossRefGoogle ScholarPubMed
Asparouhov, T., & Muthén, B. (2010). Multiple imputation with Mplus (Version 2) [Data set]. Mplus. http://statmodel.com/download/Imputations7.pdfGoogle Scholar
Audigier, V., & Resche-Rigon, M. (2018). Package micemd: Multivariate imputation by Chained Equations.Google Scholar
Audigier, V., White, I. R, Jolani, S., Debray, T. P. A, Quartagno, M., Carpenter, J., van Buuren, S., & Resche-Rigon, M. (2018). Multiple imputation for multilevel data with continuous and binary variables. Statistical Science, 33(2), 160–183. http://doi.org/10.1214/18-STS646CrossRefGoogle Scholar
Carpenter, J., & Kenward, M. (2013). Multiple imputation and its application (1stEd.). Wiley. http://doi.org/10.1002/9781119942283CrossRefGoogle Scholar
Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6, 330–351. http://doi.org/10.1037//1082-989X.6.4.330CrossRefGoogle ScholarPubMed
Drechsler, J. (2015). Multiple imputation of multilevel missing data-rigor versus simplicity. Journal of Educational and Behavioral Statistics, 4(1), 69–95. http://doi.org/10.3102/1076998614563393CrossRefGoogle Scholar
Enders, C. K. (2010). Applied missing data analysis. Methodology in the social sciences. Guilford Press.Google Scholar
Enders, C. K. (2017). Multiple imputation as a flexible tool for missing data handling in clinical research. Behaviour Research and Therapy, 98, 4–18. http://doi.org/10.1016/j.brat.2016.11.008CrossRefGoogle ScholarPubMed
Enders, C. K., Hayes, T., & Du, H. (2018). A comparison of multilevel imputation schemes for random coefficient models: Fully conditional specification and joint model imputation with random covariance matrices. Multivariate Behavioral Research, 53(5), 695–713. https://doi.org/10.1080/00273171.2018.1477040CrossRefGoogle ScholarPubMed
Enders, C. K., Keller, B. T., & Levy, R. (2018). A fully conditional specification approach to multilevel imputation of categorical and continuous variables. Psychological Methods, 23(2), 298–317. https://doi.org/10.1037/met0000148CrossRefGoogle ScholarPubMed
Enders, C. K., Mistler, S. A., & Keller, B. T. (2016). Multilevel multiple imputation: A review and evaluation of joint modeling and chained equations imputation. Psychological Methods, 21, 222–240. https://doi.org/10.1037/met0000063CrossRefGoogle ScholarPubMed
Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press. http://doi.org/10.1017/CBO9780511790942Google Scholar
Gelman, A., & Rubin, D. B. (1992). A single series from the Gibbs sampler provides a false sense of security. Bayesian Statistics, 4, 625–631.Google Scholar
Goldstein, H. (2003). Multilevel statistical models (3ª Ed.). Halstead Press.Google Scholar
Goldstein, H., Carpenter, J. R., & Browne, W. J. (2014). Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non-linear terms. Journal of Royal Statistical Society Series A, 177(2), 553–564. https://doi.org/10.1111/rssa.12022CrossRefGoogle Scholar
Graham, J. W. (2003). Adding missing-data-relevant variables to fiml-based structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 10, 80–100. http://doi.org/10.1207/S15328007SEM1001_4CrossRefGoogle Scholar
Graham, J. W., Olchowski, A. E., & Gilreath, T. D. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8(3), 206–213. http://doi.org/10.1007/s11121-007-0070-9CrossRefGoogle ScholarPubMed
Grund, S., Lüdtke, O., & Robitzsch, A. (2016). Multiple imputation of missing covariate values in multilevel models with random slopes: A cautionary note. Behavior Research Methods, 48(2), 640–649. http://doi.org/10.3758/s13428-015-0590-3CrossRefGoogle ScholarPubMed
Grund, S., Lüdtke, O., & Robitzsch, A. (2018). Multiple imputation of missing data for multilevel models: Simulations and recommendations. Organizational Research Methods, 21(1), 111–149. http://doi.org/10.1177/1094428117703686CrossRefGoogle Scholar
Grund, S., Robitzsch, A., & Lüdtke, O. (2019). ‘Mitml‘: Tools for multiple imputation in multilevel modeling (R package version 0.3–6) [Data set] . CRAN. https://cran.r-project.org/web/packages/mitml/mitml.pdfGoogle Scholar
Keller, B. T., & Enders, C. K. (2019). Blimp User’s Guide (Version 2.1.) [Computer software]. http://www.appliedmissingdata.com/blimpusermanual-2-1.pdfGoogle Scholar
Hox, J. J. (2010). Multinivel analysys. Techniques and applications (2nd Ed.). Routledge.Google Scholar
Hughes, R. A., White, I. R., Seaman, S. R., Carpenter, J. R., Tilling, K., & Sterne, J. A. C. (2014). Joint modeling rationale for chained equations. BMC Medical Research Methodology, 14, Article 28. https://doi.org/10.1186/1471-2288-14-28CrossRefGoogle Scholar
Jolani, S., Debray, T. P. A., Koffijberg, H., van Buuren, S., & Moons, K. G. M. (2015). Imputation of systematically missing predictors in an individual participant data meta-analysis: A generalized approach using MICE. Statistics in Medicine, 34(11), 1841–1863. https://doi.org/10.1002/sim.6451CrossRefGoogle Scholar
Kunkel, D., & Kaizar, E. E. (2017). A comparison of existing methods for multiple imputation in individual participant data meta-analysis. Statistics in Medicine, 36(22), 3507–3532. http://doi.org/10.1002/sim.7388CrossRefGoogle ScholarPubMed
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2 nd Ed.). Wiley.CrossRefGoogle Scholar
McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114–140. https://doi.org/10.1037/met0000078CrossRefGoogle ScholarPubMed
Quartagno, M., & Carpenter, J. (2020, August, 12). jomo: Multilevel joint modelling multiple imputation (Version 2.7–2.) [Data set]. CRAN. https://cran.r-project.org/web/packages/jomo/jomo.pdfGoogle Scholar
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchial linear models: Applications and data analysis methods (2 nd Ed.). Sage.Google Scholar
Raykov, T. (2011). On testability of missing data mechanisms in incomplete data sets. Structural Equation Modeling: A Multidisciplinary Journal, 18(3), 419–429. https://doi.org/10.1080/10705511.2011.582396CrossRefGoogle Scholar
Resche-Rigon, M., & White, I. R. (2016). Multiple imputation by chained equations for systematically and sporadically missing multilevel data. Statistical Methods in Medical Research, 27, 1634-1649. http://doi.org/10.1177/0962280216666564CrossRefGoogle ScholarPubMed
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581–592. http://doi.org/10.1093/biomet/63.3.581CrossRefGoogle Scholar
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. Wiley. http://doi.org/10.1002/9780470316696CrossRefGoogle Scholar
Rubin, D. B. (1996). Multiple imputation after 18+ years. Journal of the American Statistical Association, 91(434), 473–489. http://doi.org/10.1080/01621459.1996.10476908CrossRefGoogle Scholar
Schafer, J. L. (1997). Analysis of incomplete multivariate data. Chapman & Hall/CRC. http://doi.org/10.1201/9781439821862CrossRefGoogle Scholar
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of art. Psychological Methods, 7, 147–177. https://doi.org/10.1037/1082-989X.7.2.147CrossRefGoogle Scholar
Schafer, J. L., & Yucel, R. M. (2002). Computational strategies for multivariate linear mixed effects models with missing data. Journal of Computational and Graphical Statistics, 11, 437–457.CrossRefGoogle Scholar
Scott, M. A., Shrout, P. E., & Weinberg, S. L. (2013). Multilevel model notation—establishing the commonalities. In The SAGE handbook of multilevel modeling (pp. 21–38). SAGE Publications Inc. http://doi.org/10.4135/9781446247600.n2CrossRefGoogle Scholar
van Buuren, S. (2011). Multiple imputation of multilevel data. In Hox, J. J. (Ed.), Handbook of advanced multilevel analysis (pp. 173–196). Routledge.Google Scholar
van Buuren, S. (2018). Flexible imputation of missing data. CRC Press.CrossRefGoogle Scholar
van Buuren, S., & Groothuis-Oudshoorn, K. (2011). MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–68. http://doi.org/10.18637/jss.v045.i03CrossRefGoogle Scholar
van Buuren, S., Groothuis-Oudshoorn, K., Robitzsch, A., Vink, G., Doove, L., & Jolani, S. (2015). Package ‘mice’ [Computer software]. CRAN. https://mran.microsoft.com/snapshot/2014-11-17/web/packages/mice/mice.pdfGoogle Scholar
Yucel, R. M. (2008). Multiple imputation inference for multivariate multilevel continuous data with ignorable non-response. Philosophical Transactions of the Royal Society of London Series A, Mathematical and Physical Sciences, 366, 2389–2403. https://doi.org/10.1098/rsta.2008.0038Google ScholarPubMed
Yucel, R. M. (2011). Random covariances and mixed-effects models for imputing multivariate multilevel continuous data. Statistical Modelling, 11(4), 351–370. http://doi.org/10.1177/1471082X1001100404CrossRefGoogle ScholarPubMed