Hostname: page-component-586b7cd67f-2brh9 Total loading time: 0 Render date: 2024-11-25T10:09:18.987Z Has data issue: false hasContentIssue false
  • English
  • Français

Performance of various BFGS implementations with limited precision second-order information

Published online by Cambridge University Press:  17 February 2009

D. Byatt ,
I. D. Coope  and
C. J. Price
D. Byatt
Affiliation:
Department of Mathematics and Statistics, University of Canterbury, Private Bag 4800, Christchurch, New Zealand; email: [email protected].
I. D. Coope
Affiliation:
Department of Mathematics and Statistics, University of Canterbury, Private Bag 4800, Christchurch, New Zealand; email: [email protected].
C. J. Price
Affiliation:
Department of Mathematics and Statistics, University of Canterbury, Private Bag 4800, Christchurch, New Zealand; email: [email protected].
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The BFGS formula is arguably the most well known and widely used update method for quasi-Newton algorithms. Some authors have claimed that updating approximate Hessian information via the BFGS formula with a Cholesky factorisation offers greater numerical stability than the more straightforward approach of performing the update directly. Other authors have claimed that no such advantage exists and that any such improvement is probably due to early implementations of the DFP formula in conjunction with low accuracy line searches.

This paper supports the claim that there is no discernible advantage in choosing factorised implementations (over non-factorised implementations) of BFGS methods when approximate Hessian information is available to full machine precision. However the results presented in this paper show that a factorisation strategy has clear advantages when approximate Hessian information is available only to limited precision. These results show that a conjugate directions factorisation outperforms the other methods considered in this paper (including Cholesky factorisation).

Type
Research Article
Information
The ANZIAM Journal , Volume 45 , Issue 4 , April 2004 , pp. 511 - 522
Copyright
Copyright © Australian Mathematical Society 2004

References

[1]Brodlie, K. W., Gourlay, A. R. and Greenstadt, J., “Rank-one and rank-two corrections to positive definite matrices expressed in product form”, J. Inst. Math. Appl. 11 (1973) 73–82.CrossRefGoogle Scholar
[2]Byatt, D., Coope, I. D. and Price, C. J., “Conjugate grids for unconstrained optimisation”, Comput. Optim. Appl. 29 (2004), to appear.CrossRefGoogle Scholar
[3]Byatt, D., Coope, I. D. and Price, C. J., “Performance of various BFGS and DFP implementations with limited precision second order information”, Research Report UCDMS2003/1, Univ. of Canterbury, Christchurch, New Zealand, Jan 2003.Google Scholar
[4]Coope, I. D., “A conjugate direction implementation of the BFGS algorithm with automatic scaling”, J. Austral. Math. Soc. Ser. B 31 (1989) 122–134.CrossRefGoogle Scholar
[5]Davidon, W. C., “Optimally conditioned optimization algorithms without line searches”, Math. Program. 9 (1975) 1–30.CrossRefGoogle Scholar
[6]Fletcher, R., Practical methods of optimization, 2nd ed. (John Wiley & Sons, New York, 1987).Google Scholar
[7]Gill, P. E. and Murray, W., “Quasi-Newton methods for unconstrained optimization”, J. Inst. Math. Appl. 9 (1972) 91–108.CrossRefGoogle Scholar
[8]Gill, P. E. and Murray, W., “Modification of matrix factorizations after a rank one update”, in The state of the art in numerical analysis (ed. Jacobs, D.), Institute of Mathematics and its Applications, (Academic Press, London, 1976) 55–83.Google Scholar
[9]Grandinetti, L., “Factorization versus non-factorization in quasi-Newtonian algorithms for differentiable optimization”, in Third Symposium on Operations Research (University of Mannheim, Mannheim, 1978), Section I, (Hain, Konigstein, 1979) 255–274.Google Scholar
[10]Moré, J. J., Garbow, B. S. and Hillstrom, K. E., “Testing unconstrained optimization software”, ACM Trans. Math. Software 7 (1981) 17–41.CrossRefGoogle Scholar
[11]Osborne, M. R. and Saunders, M. A., “Descent methods for minimization”, in Optimization (eds. Anderssen, R. S., Jennings, L. D. and Ryan, D. M.), (Univ. of Queensland Press, St. Lucia, 1972) 221–237.Google Scholar
[12]Powell, M. J. D., “Updating conjugate directions by the BFGS formula”, Math. Program. 38 (1987) 29–46.CrossRefGoogle Scholar
[13]Schnabel, R. B. and Eskow, E., “A revised modified Cholesky factorization algorithm”, SIAM J. Optim. 9 (1999) 1135–1148.CrossRefGoogle Scholar