Hostname: page-component-586b7cd67f-tf8b9 Total loading time: 0 Render date: 2024-11-26T03:25:05.837Z Has data issue: false hasContentIssue false
  • English
  • Français

Sharp upper bounds on perfect retrieval in the Hopfield model

Part of: Time-dependent statistical mechanics (dynamic and nonequilibrium) Special processes

Published online by Cambridge University Press:  14 July 2016

Anton Bovier
Anton Bovier*
Affiliation:
Weierstraß–Institut für Angewandte Analysis und Stochastik
*
∗Postal address: WIAS, Mohrenstrasse 39, D-10117 Berlin, Germany. Email address: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

We prove a sharp upper bound on the number of patterns that can be stored in the Hopfield model if the stored patterns are required to be fixed points of the gradient dynamics. We also show corresponding bounds on the one-step convergence of the sequential gradient dynamics. The bounds coincide with the known lower bounds and confirm the heuristic expectations. The proof is based on a crucial idea of Loukianova (1997) using the negative association properties of some random variables arising in the analysis.

Keywords

Hopfield modelstorage capacitygradient dynamicssequential dynamics

MSC classification

Primary: 82C32: Neural nets 60K35: Interacting random processes; statistical mechanics type models; percolation theory
Type
Short Communications
Information
Journal of Applied Probability , Volume 36 , Issue 3 , September 1999 , pp. 941 - 950
Copyright
Copyright © Applied Probability Trust 1999 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Burshtein, D. (1994). Non-direct convergence radius and number of iterations of the Hopfield associative memory. IEEE Trans. Inf. Theory 40, 838–847.CrossRefGoogle Scholar
Bovier, A. and Picco, P. (eds.) (1998). Mathematical Aspects of Spin Glasses and Neural Networks. (Progress in Probability 41). Birkhäuser, Boston, MA.Google Scholar
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. USA 79, 2554–2558.CrossRefGoogle ScholarPubMed
Joag-Dev, K., and Proschan, F. (1983). Negative association of random variables, with applications. Ann. Statist. 11, 286–295.CrossRefGoogle Scholar
Komlós, J., and Paturi, R. (1988). Convegence results in an autoassociative memory model. Neural Networks 1, 239–250.CrossRefGoogle Scholar
Loukianova, D. (1997). Lower bounds on the restitution error in the Hopfield model. Prob. Theor. Rel. Fields 107, 161–176.CrossRefGoogle Scholar
Löwe, M. (1998). Storage capacity of the Hopfield model with correlated patterns. Ann. Appl. Prob. 8, 1216–1250.CrossRefGoogle Scholar
Martinez, S. (1996). Introduction to neural networks. In Disordered Sysytems (Travaux en Cours 53). Hermann, Paris. Preprint, Temuco.Google Scholar
McEliece, R. J., Posner, E. C., Rodemich, E. R., and Venkatesh, S. S. (1987). The capacity of the Hopfield associative memory. IEEE Trans. Inf. Theory 33, 461–482.CrossRefGoogle Scholar
Newman, C. M. (1988). Memory capacity in neural network models: rigorous results. Neural Networks 1, 223–238.CrossRefGoogle Scholar
Petritis, D. (1996). Thermodynamic formalism of neural computing. Dynamics of Complex Interacting Systems (Nonlinear Phenomena and Complex Systems 2). Kluwer, Dordrecht, pp. 81–146.CrossRefGoogle Scholar
Talagrand, M. (1998). Rigorous results for the Hopfield model with many patterns. Prob. Theory Rel. Fields 110, 177–276.CrossRefGoogle Scholar
Vermet, F. (1994). Étude asymptotique d'un réseau neuronal: le modèle de mémoire associative de Hopfield. Thèse, Université de Rennes I.Google Scholar
Weisbuch, G. and Fogelman-Soulié, F. (1985). Scaling laws for the attractor Hopfield networks. J. Phys. Lett. 46, 623–630.CrossRefGoogle Scholar