Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-02-04T17:39:01.921Z Has data issue: false hasContentIssue false
  • English
  • Français

Genetic Optimization Using Derivatives

Published online by Cambridge University Press:  04 January 2017

Jasjeet S. Sekhon  and
Walter R. Mebane Jr.
Get access Rights & Permissions [Opens in a new window]

Abstract

We describe a new computer program that combines evolutionary algorithm methods with a derivative-based, quasi-Newton method to solve difficult unconstrained optimization problems. The program, called GENOUD (GENetic Optimization Using Derivatives), effectively solves problems that are nonlinear or perhaps even discontinuous in the parameters of the function to be optimized. When a statistical model's estimating function (for example, a log-likelihood) is nonlinear in the model's parameters, the function to be optimized will usually not be globally concave and may contain irregularities such as saddlepoints or discontinuous jumps. Optimization methods that rely on derivatives of the objective function may be unable to find any optimum at all. Or multiple local optima may exist, so that there is no guarantee that a derivative-based method will converge to the global optimum. We discuss the theoretical basis for expecting GENOUD to have a high probability of finding global optima. We conduct Monte Carlo experiments using scalar Normal mixture densities to illustrate this capability. We also use a system of four simultaneous nonlinear equations that has many parameters and multiple local optima to compare the performance of GENOUD to that of the Gauss-Newton algorithm in SAS's PROC MODEL.

Type
Research Article
Information
Political Analysis , Volume 7 , 1998 , pp. 187 - 210
Copyright
Copyright © Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Amemiya, Takeshi. 1985. Advanced Econometrics. Cambridge: Harvard University Press.Google Scholar
Billingsley, Patrick. 1986. Probability and Measure. New York: Wiley.Google Scholar
Bollen, Kenneth A. 1989. Structural Equations with Latent Variables. New York: Wiley.Google Scholar
Bright, H. S., and Enison, R. L. 1979. “Quasi-random Number Sequences from a Long-period TLP Generator with Remarks on Application to Cryptography.” Computing Surveys 11: 357–70.Google Scholar
Davis, Lawrence, ed. 1991. Handbook of Genetic Algorithms. New York: Van Nostrand Reinhold.Google Scholar
De Jong, Kenneth A. 1993. “Genetic Algorithms Are Not Function Optimizers.” In Foundations of Genetic Algorithms 2, ed. Darrell Whitley, L. San Mateo, CA: Morgan Kaufmann.Google Scholar
Efron, Bradley, and Tibshirani, Robert J. 1993. An Introduction to the Bootstrap. New York: Chapman and Hall.Google Scholar
Fahrmeir, Ludwig, and Tutz, Gerhard. 1994. Multivariate Statistical Modeling Based on Generalized Linear Models. New York: Springer-Verlag.Google Scholar
Feller, William. 1970. An Introduction to Probability Theory and Its Applications. Vol. 1, 3d ed., revised printing. New York: Wiley.Google Scholar
Filho, Jose L. Ribeiro, Treleaven, Philip C., and Alippi, Cesare. 1994. “Genetic Algorithm Programming Environments.” Computer 27: 28–43.CrossRefGoogle Scholar
Gallant, A. Ronald. 1987. Nonlinear Statistical Models. New York: Wiley.Google Scholar
Gill, Philip E., Murray, Walter, and Wright, Margaret H. 1981. Practical Optimization. San Diego: Academic Press.Google Scholar
Goldberg, David E. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley.Google Scholar
Grefenstette, John J. 1993. “Deception Considered Harmful.” In Foundations of Genetic Algorithms 2, ed. Darrell Whitley, L. San Mateo, CA: Morgan Kaufmann.Google Scholar
Grefenstette, John J., and Baker, James E. 1989. “How Genetic Algorithms Work: A Critical Look at Implicit Parallelism.” In Proceedings of the Third International Conference on Genetic Algorithms, 20–27. San Mateo, CA: Morgan Kaufmann.Google Scholar
Heckman, James J., and Singer, Burton, ed. 1985. Longitudinal Analysis of Labor Market Data. New York: Cambridge University Press.CrossRefGoogle Scholar
Holland, John H. 1975. Adaptation in Natural and Artificial Systems. Ann Arbor: University of Michigan Press.Google Scholar
Jöreskog, Karl G. 1973. “A General Method for Estimating a Linear Structural Equation System.” In Structural Equation Models in the Social Sciences, ed. Goldberger, Arthur S. and Dudley Duncan, Otis, New York: Seminar Press.Google Scholar
Kemighan, Brian W., and Ritchie, Dennis M. 1988. The C Programming Language. 2d ed. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Lancaster, Tony. 1990. The Econometric Analysis of Transition Data. New York: Cambridge University Press.Google Scholar
Lewis, T. G., and Payne, W. H. 1973. “Generalized Feedback Shift Register Pseudorandom Number Algorithm.” Journal of the Association for Computing Machinery 20: 456–68.CrossRefGoogle Scholar
Maddala, G. S. 1983. Limited-Dependent and Qualitative Variables in Econometrics. Cambridge: Cambridge University Press.Google Scholar
Marron, J. P., and Wand, M. P. 1992. “Exact Mean Integrated Squared Error.” Annals of Statistics. 20: 712–36.Google Scholar
McCullagh, P., and Nelder, J. A. 1992. Generalized Linear Models. New York: Chapman & Hall.Google Scholar
Mebane, Walter R. Jr. 1999. “Congressional Campaign Contributions, District Service and Electoral Outcomes in the United States: Statistical Tests of a Formal Game Model with Nonlinear Dynamics.” In Political Complexity: Nonlinear Models of Politics, ed. Richards, Diana. Ann Arbor: University of Michigan Press.Google Scholar
Mebane, Walter R. Jr., and Sekhon, Jasjeet. 1996. GENOUD: GENetic Optimization Using Derivatives. Computer program.Google Scholar
Mebane, Walter R. Jr., and Sekhon, Jasjeet. 1998. GENBLIS: GENetic optimization and Bootstrapping of Linear Structures. Computer program.Google Scholar
Mebane, Walter R. Jr., and Wand, Jonathan. 1997. “Markov Chain Models for Rolling Cross-section Data: How Campaign Events and Political Awareness Affect Vote Intentions and Partisanship in the United States and Canada.” Paper presented at the 1997 Annual Meeting of the Midwest Political Science Association, April 10-12, Palmer House Hilton, Chicago, IL.Google Scholar
Mebane, Walter R. Jr., and Wawro, Gregory J. 1996. ” Ghadha, Gramm-Rudman-Hollings and Impoundments: Testing a Spatial Game Model of Congressional Delegation and Institutional Change.” Manuscript.Google Scholar
Michalewicz, Zbigniew. 1992. Genetic Algorithms + Data Structures = Evolution Programs. New York: Springer-Verlag Google Scholar
Michalewicz, Zbigniew, and Logan, Thomas D. 1993. “GA Operators for Convex Continuous Parameter Spaces.” Manuscript.Google Scholar
Michalewicz, Zbigniew, Swaminathan, Swarnalatha, and Logan, Thomas D. 1993. GENOCOP (version 2.0). C language computer program source code.Google Scholar
Nix, Allen E., and Vose, Michael D. 1992. “Modeling Genetic Algorithms with Markov Chains.” Annals of Mathematics and Artificial Intelligence 5: 79–88.Google Scholar
SAS Institute, Inc. 1988. SAS/ETS User's Guide, Version 6, First Edition. Cary, NC: SAS Institute, Inc.Google Scholar
SAS Institute, Inc. 1989. “SAS (r) PROC MODEL.” Proprietary Software Release 6.09. Cary, NC: SAS Institute, Inc.Google Scholar
Shao, Jun, and Tu, Dongsheng. 1995. The Jackknife and Bootstrap. New York: Springer-Verlag.Google Scholar
Wawro, Gregory J. 1996. Legislative Entrepreneurship in the U.S. House of Representatives. Unpublished Ph.D. dissertation, Cornell University.Google Scholar
Vose, Michael D. 1993. “Modeling Simple Genetic Algorithms.” In Foundations of Genetic Algorithms 2 2, ed. L. Darrell Whitley. San Mateo, CA: Morgan Kauftnann Publishers.Google Scholar