Hostname: page-component-5cf477f64f-fcbfl Total loading time: 0 Render date: 2025-04-08T01:07:30.568Z Has data issue: false hasContentIssue false
  • English
  • Français

A Solution to Separation in Binary Response Models

Published online by Cambridge University Press:  04 January 2017

Christopher Zorn
Christopher Zorn*
Affiliation:
Law and Social Science Program, National Science Foundation, 4201 Wilson Boulevard, Suite 995, Arlington, VA 22230. e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

A common problem in models for dichotomous dependent variables is “separation,” which occurs when one or more of a model's covariates perfectly predict some binary outcome. Separation raises a particularly difficult set of issues, often forcing researchers to choose between omitting clearly important covariates and undertaking post—hoc data or estimation corrections. In this article I present a method for solving the separation problem, based on a penalized likelihood correction to the standard binomial GLM score function. I then apply this method to data from an important study on the postwar fate of leaders.

Type
Research Article
Information
Political Analysis , Volume 13 , Issue 2 , Spring 2005 , pp. 157 - 170
Copyright
Copyright © The Author 2005. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agresti, Alan. 2002. Categorical Data Analysis, 2nd ed. New York: Wiley.CrossRefGoogle Scholar
Albert, A., and Anderson, J. A. 1984. “On the Existence of Maximum Likelihood Estimates in Logistic Regression Models.” Biometrika 71(1): 1–10.Google Scholar
Aldrich, John H., and Nelson, Forrest D. 1984. Linear Probability, Logit, and Probit Models. Newbury Park, CA: Sage.CrossRefGoogle Scholar
Barndorff-Nielsen, O. E., and Cox, David R. 1994. Inference and Asymptotics. London: Chapman and Hall.CrossRefGoogle Scholar
Blalock, Hubert M. 1979. Social Statistics, rev. 2nd ed. New York: McGraw-Hill.Google Scholar
Brown, Lawrence D., Tony, Cai T., and DasGupta, Anirban. 2001. “Interval Estimation for a Binomial Proportion.” Statistical Science 16(2): 101–133.Google Scholar
Cameron, Lisa J. 2000. “Limiting Buyer Discretion: Effects on Performance and Price in Long-Term Contracts.” American Economic Review 90 (March): 265–281.Google Scholar
Clogg, Clifford C., Rubin, D. B., Schenker, N., Schultz, B., and Weidman, L. 1991. “Multiple Imputation of Industry and Occupation Codes in Census Public-Use Samples Using Bayesian Logistic Regression.” Journal of the American Statistical Association 86: 68–78.Google Scholar
Collett, Dave. 2002. Modeling Binary Data, 2nd ed. London: Chapman and Hall.Google Scholar
Cox, David R. 1970. Analysis of Binary Data. New York: Wiley.Google Scholar
Davidson, R., and MacKinnon, J. G. 1993. Estimation and Inference in Econometrics. New York: Oxford University Press.Google Scholar
Eisenberg, Theodore, Goerdt, John, Ostrom, Brian, Rottman, David, and Wells, Martin T. 1997. “The Predictability of Punitive Damages.” Journal of Legal Studies 26 (June): 623–661.CrossRefGoogle Scholar
Firth, David. 1993. “Bias Reduction of Maximum Likelihood Estimates.” Biometrika 80(1): 27–38.Google Scholar
Galindo-Garre, Francisca, Vermunt, Jeroen K., and Bergsma, Wicher P. 2005. “Bayesian Posterior Estimation of Logit Parameters with Small Samples.” Sociological Methods and Research: forthcoming.Google Scholar
Gibler, Douglas M., and Vasquez, John R. 1998. “Uncovering the Dangerous Alliances, 1495–1980.” International Studies Quarterly 42 (December): 785–807.Google Scholar
Gill, Jeff. 2002. Bayesian Methods: A Social and Behavioral Sciences Approach. Boca Raton, FL: Chapman & Hall.Google Scholar
Goemans, Hein. 2000. “Fighting for Survival: The Fate of Leaders and the Duration of War.” Journal of Conflict Resolution 44 (October): 555–579.Google Scholar
Greene, William H. 2003. Econometric Analysis, 5th ed. Upper Saddle River, NJ: Prentice-Hall.Google Scholar
Haldane, John Burden Sanderson. 1956. “The Estimation and Significance of the Logarithm of a Ratio of Frequencies.” Annals of Human Genetics 20 (July): 309–311.Google Scholar
Heinze, Georg, and Ploner, Meinhard. 2003. “Fixing the Nonconvergence Bug in Logistic Regression with SPLUS and SAS.” Computer Methods and Programs in Biomedicine 71: 181–187.Google Scholar
Heinze, Georg, and Ploner, Meinhard. 2004. “Technical Report 2/2004: A SAS Macro, S-Plus Library and R Package to Perform Logistic Regression Without Convergence Problems.” Section of Clinical Biometrics, Department of Medical Computer Sciences, Medical University of Vienna, Vienna.Google Scholar
Heinze, Georg, and Schemper, Michael. 2003. “A Solution to the Problem of Separation in Logistic Regression.” Statistics in Medicine 21(21): 2409–2419.Google Scholar
Hill, Mark E. 2000. “Color Differences in the Socioeconomic Status of African American Men: Results of a Longitudinal Study.” Social Forces 78 (June): 1437–1460.Google Scholar
Ibrahim, Joseph G., and Laud, Purushottam W. 1991. “On Bayesian Analysis of General Linear Models Using Jeffreys’ Prior.” Journal of the American Statistical Association 86: 981–986.CrossRefGoogle Scholar
Iyengar, S., and Greenhouse, J. B. 1988. “Selection Models and the File-Drawer Problem.” Statistical Science 3: 109–135.Google Scholar
Jacobsen, J. 1989. “Existence and Unicity of MLEs in Discrete Exponential Family Distributions.” Scandinavian Journal of Statistics 16: 335–349.Google Scholar
Jeffreys, H. 1946. “An Invariant Form of the Prior Probability in Estimation Problems.” Proceedings of the Royal Society of London, Series A 186: 453–461.Google Scholar
Jianakoplos, Nancy A., and Menchik, Paul L. 1997. “Wealth Mobility.” Review of Economics and Statistics 79 (February): 18–31.Google Scholar
Kass, R. E. 1989. “The Geometry of Asymptotic Inference.” Statistical Science 4 (August): 188–219.Google Scholar
King, Elizabeth N., and Ryan, Thomas P. 2002. “A Preliminary Investigation of Maximum Likelihood Logistic Regression versus Exact Logistic Regression.” American Statistician 56 (August): 163–170.Google Scholar
Krain, Matthew. 1997. “State-Sponsored Mass Murder: The Onset and Severity of Genocides and Politicides.” Journal of Conflict Resolution 41 (June): 331–360.Google Scholar
Lesaffre, E., and Albert, A. 1989. “Partial Separation in Logistic Discrimination.” Journal of the Royal Statistical Society, Series B 51: 109–116.Google Scholar
Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage.Google Scholar
McCullagh, P., and Nelder, J. A. 1989. Generalized Linear Models, 2nd ed. London: Chapman & Hall.Google Scholar
Mehta, C. R., and Patel, R. 1995. “Exact Logistic Regression: Theory and Examples.” Statistics in Medicine 14: 2143–2160.CrossRefGoogle Scholar
Peterson, Geoff, and Wrighton, Mark J. 2000. “Expressions of Distrust: Third-Party Voting and Cynicism in Government.” Political Behavior 20 (March): 17–34.Google Scholar
Poirier, Dale J. 1994. “Jeffreys’ Prior for Logit Models.” Journal of Econometrics 63 (August): 327–339.Google Scholar
Reiter, Dan. 1999. “Military Strategy and the Outbreak of International Conflict: Quantitative Empirical Tests, 1903–1992.” Journal of Conflict Resolution 43 (June): 366–387.Google Scholar
Rotolo, Thomas. 2000. “A Time to Join, A Time to Quit: The Influence of Life Cycle Transitions on Voluntary Association Membership.” Social Forces 78 (March): 1133–1161.Google Scholar
Silvapulle, M. J. 1981. “On the Existence of Maximum Likelihood Estimates for the Binomial Response Models.” Journal of the Royal Statistical Society, Series B 43: 310–313.Google Scholar
Venables, W. N., and Ripley, B. D. 2002. Modern Applied Statistics with S, 4th ed. New York: Springer.Google Scholar
Wedderburn, R. W. M. 1976. “On the Existence and Uniqueness of the Maximum Likelihood Estimates for Certain Generalized Linear Models.” Biometrika 63 (April): 27–32.Google Scholar
Werner, Suzanne. 1996. “Absolute and Limited War: The Possibilities of a Foreign Imposed Regime Change.” International Interactions 22(1): 67–88.Google Scholar
Wibbels, Erik. 2000. “Federalism and the Politics of Macroeconomic Policy and Performance.” American Journal of Political Science 44 (October): 687–702.Google Scholar