Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-11-26T09:54:01.379Z Has data issue: false hasContentIssue false
  • English
  • Français

A simple model from a powerful framework that spans levels of analysis

Published online by Cambridge University Press:  11 December 2008

Timothy T. Rogers  and
James L. McClelland
Timothy T. Rogers
Affiliation:
University of Wisconsin-Madison, Department of Psychology, Madison, WI 53706; [email protected]://concepts.psych.wisc.edu
James L. McClelland
Affiliation:
Center for Mind, Brain and Computation, and Department of Psychology, Stanford University, Stanford, CA 94305. [email protected]://psychology.stanford.edu/~jlm
Get access Rights & Permissions [Opens in a new window]

Abstract

The commentaries reflect three core themes that pertain not just to our theory, but to the enterprise of connectionist modeling more generally. The first concerns the relationship between a cognitive theory and an implemented computer model. Specifically, how does one determine, when a model departs from the theory it exemplifies, whether the departure is a useful simplification or a critical flaw? We argue that the answer to this question depends partially upon the model's intended function, and we suggest that connectionist models have important functions beyond the commonly accepted goals of fitting data and making predictions. The second theme concerns perceived in-principle limitations of the connectionist approach to cognition, and the specific concerns these perceived limitations raise for our theory. We argue that the approach is not in fact limited in the ways our critics suggest. One common misconception, that connectionist models cannot address abstract or relational structure, is corrected through new simulations showing directly that such structure can be captured. The third theme concerns the relationship between parallel distributed processing (PDP) models and structured probabilistic approaches. In this case we argue that there the difference between the approaches is not merely one of levels. Our PDP approach differs from structured statistical approaches at all of Marr's levels, including the characterization of the goals of cognitive computations, and of the representations and algorithms used.

Type
Authors' Response
Information
Behavioral and Brain Sciences , Volume 31 , Issue 6 , December 2008 , pp. 729 - 749
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ackley, D. H., Hinton, G. E. & Sejnowski, T. J. (1985) A learning algorithm for Boltzmann machines. Cognitive Science 9:147–69.Google Scholar
Anderson, J. R. (1991) The adaptive nature of human categorization. Psychological Review 98(3):409–29.CrossRefGoogle Scholar
Bengio, S. & Bengio, Y. (2000) Taking on the curse of dimensionality in joint distributions using neural networks. IEEE Transactions on Neural Networks (special issue on data mining and knowledge discovery) 11:550–57.CrossRefGoogle ScholarPubMed
Bengio, Y., Lamblin, P., Popovici, D. & Larochelle, H. (2006) Greedy layer-wise training of deep networks. In: Advances in neural information processing systems (NIPS). MIT Press.Google Scholar
Borges, J. L. (1998) On the exactitude of science. In: Collected fictions, trans. Hurley, A., p. 325. Penguin.Google Scholar
Bybee, J. (2001) Phonology and language use. Cambridge University Press.CrossRefGoogle Scholar
Chomsky, N. (1957) Syntactic structure. Mouton.CrossRefGoogle Scholar
Christiansen, M. H. & Ellefson, M. (2002) Linguistic adaptation without linguistic constraints: The role of sequential learning in language evolution. In: The transition to language, ed. Wray, A., pp. 335–58. Oxford University Press.CrossRefGoogle Scholar
Cohen, J. D., Dunbar, K. & McClelland, J. L. (1990) On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review 97:332–61.CrossRefGoogle ScholarPubMed
Collins, A. M. & Quillian, M. R. (1969) Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior 8:240–47.CrossRefGoogle Scholar
Cree, G., McRae, K. & McNorgan, C. (1999) An attractor model of lexical conceptual processing: Simulating semantic priming. Cognitive Science 23(4):371–414.CrossRefGoogle Scholar
Dienes, Z., Altmann, G. & Gao, S. (1999) Mapping across domains without feedback: A neural-network model of transfer of implicit knowledge. Cognitive Science 23(1):53–82.CrossRefGoogle Scholar
Dilkina, K., McClelland, J. L. & Boroditsky, L. (2007) How language affects thought in a connectionist model. In: Proceedings of the 29th Annual Cognitive Science Society Conference, pp. 215. Erlbaum.Google Scholar
Dilkina, K., McClelland, J. L. & Plaut, D. C. (2008) A single-system account of semantic and lexical deficits in five semantic dementia patients. Cognitive Neuropsychology 25(2):136–64.CrossRefGoogle ScholarPubMed
Ellis, H. D. & Lewis, M. B. (2001) Capgras delusion: A window on face recognition. Trends in Cognitive Science 5(4):149–56.CrossRefGoogle ScholarPubMed
Elman, J. L. (1990) Finding structure in time. Cognitive Science 14:179–211.CrossRefGoogle Scholar
Farah, M. & McClelland, J. L. (1991) A computational model of semantic memory impairment: Modality-specificity and emergent category-specificity. Journal of Experimental Psychology: General 120:339–57.CrossRefGoogle ScholarPubMed
Fodor, J. (1998) Concepts: Where cognitive science went wrong. Oxford University Press.CrossRefGoogle Scholar
Fodor, J. A. & Pylyshyn, Z. W. (1988) Connectionism and cognitive architecture: A critical analysis. Cognition 28:3–71.CrossRefGoogle ScholarPubMed
Gentner, D. (1983) Structure-mapping: A theoretical framework for analogy. Cognitive Science 7(2):155–70.Google Scholar
Hare, M. & Elman, J. (1995) Learning and morphological change. Cognition 56(1):61–98.CrossRefGoogle ScholarPubMed
Hinton, G. E. (1981) Implementing semantic networks in parallel hardware. In: Parallel models of associative memory, ed. Hinton, G. E. & Anderson, J. A., pp. 161–87. Erlbaum.Google Scholar
Hinton, G. E. (1986) Learning distributed representations of concepts. In: Proceedings of the 8th Annual Conference of the Cognitive Science Society, pp. 1–12. Erlbaum.Google Scholar
Hinton, G. E. (1989) Learning distributed representations of concepts. In: Parallel distributed processing: Implications for psychology and neurobiology, ed. Morris, R. G. M., pp. 46–61. Clarendon Press.Google Scholar
Hinton, G. E. & McClelland, J. L. (1988) Learning representations by recirculation. In: Neural information processing systems, ed. Anderson, D. Z., pp. 358–66. American Institute of Physics.Google Scholar
Hinton, G. E. & Salakhutdinov, R. R. (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507.CrossRefGoogle ScholarPubMed
Hinton, G. E. & Sejnowski, T. J. (1983) Optimal perceptual inference. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 1983, pp. 448–53. Computer Society of the IEEE/IEEE Press.Google Scholar
Hofstadter, D. (1995) Fluid concepts and creative analogies. Basic Books.Google Scholar
Keil, F. C. (1981) Constraints on knowledge and cognitive development. Psychological Review 88(3):197–227.CrossRefGoogle Scholar
Lambon Ralph, M. A., Lowe, C. & Rogers, T. T. (2007) The neural basis of category-specific semantic deficits for living things: Evidence from semantic dementia, HSVE and a neural network model. Brain 130:1127–37.CrossRefGoogle Scholar
Lupyan, G. & McClelland, J. L. (2003) Did, made, had, said: Capturing quasi-regularity in exceptions. In: Proceedings of the 25th Annual Conference of the Cognitive Science Society, ed. Alterman, R. & Kirsh, D., pp. 740–45. Erlbaum.Google Scholar
MacKay, D. J. (1992) A practical Bayesian framework for backpropagation networks. Neural Computation 4:448–72.CrossRefGoogle Scholar
MacWhinney, B. (2006) Emergentism: Use often and with care. Applied Linguistics 27:729–40.CrossRefGoogle Scholar
Marcus, G. F. (2001) The algebraic mind: Integrating connectionism and cognitive science. MIT Press.CrossRefGoogle Scholar
Marr, D. (1982) Vision. Freeman.Google Scholar
McClelland, J. L. (1995) A connectionist perspective on knowledge and development. In: Developing cognitive competence: New approaches to process modeling, ed. Simon, T. J. & Halford, G. S., pp. 157–204. Erlbaum.Google Scholar
McClelland, J. L. (1998) Connectionist models and Bayesian inference. In: Rational models of cognition, ed. Oaksford, M. & Chater, N., pp. 21–53. Oxford University Press.Google Scholar
McRae, K. & Cree, G. (2002) Factors underlying category-specific deficits. In: Category specificity in brain and mind, ed. Forde, E. M. E. & Humphreys, G., pp. 211–49. Psychology Press.Google Scholar
McRae, K., Cree, G., Seidenberg, M. & McNorgan, C. (2005) Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, Instruments, and Computers 37(4):547–59.CrossRefGoogle ScholarPubMed
McRae, K., De Sa, V. & Seidenberg, M. (1997) On the nature and scope of featural representations of word meaning. Journal of Experimental Psychology: General 126(2):99–130.CrossRefGoogle ScholarPubMed
Miikkulainen, R. & Dyer, M. G. (1987) Building distributed representations without microfeatures. (Technical Report No. UCLA-AI-87–17). University of California–Los Angeles, Department of Computer Science.Google Scholar
Murphy, G. L. (2000) Explanatory concepts. In: Explanation and cognition, ed. Keil, F. & Wilson, R. A., pp. 361–92. MIT Press.CrossRefGoogle Scholar
O'Reilly, R. (1996) The LEABRA model of neural interactions and learning in the neocortex. Unpublished doctoral dissertation, Department of Psychology, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
Perfors, A., Tenenbaum, J. & Regier, T. (2006) Poverty of the stimulus? A rational approach. Paper presented at the 28th Annual Conference of the Cognitive Science Society, Vancouver, British Columbia.Google Scholar
Plaut, D. C. & Shallice, T. (1993) Deep dyslexia: A case study of connectionist neuropsychology. Cognitive Neuropsychology 10(5):377–500.CrossRefGoogle Scholar
Rajat, R., Battle, A., Lee, H., Packer, B. & Ng, A. Y. (2007) Self-taught learning: Transfer learning from unlabeled data. In: Proceedings of the 24th ACM International Conference on Machine Learning, 2007. ACM International Conference Proceedings Series, vol. 227, pp. 759–66.Google Scholar
Ranzato, M., Poultney, C., Chopra, A. & LeCun, Y. (2007) Efficient learning of sparse representations with an energy-based model. In: Advances in neural information processing systems (NIPS), ed. Schölkopf, B., Platt, J. & Hoffman, T., pp. 1137–44. MIT Press.Google Scholar
Rogers, T. T., Lambon Ralph, M., Garrard, P., Bozeat, S., McClelland, J. L., Hodges, J. R. & Patterson, K. (2004) The structure and deterioration of semantic memory: A computational and neuropsychological investigation. Psychological Review 111 (1):205–35.CrossRefGoogle Scholar
Rogers, T. T. & McClelland, J. L. (2004) Semantic cognition: A parallel distributed processing approach. MIT Press.CrossRefGoogle Scholar
Rogers, T. T. & Patterson, K. (2007) Object categorization: Reversals and explanations of the basic-level advantage. Journal of Experimental Psychology: General 163(3):451–69.CrossRefGoogle Scholar
Rohde, D. L. T. (2002) A connectionist model of sentence comprehension and production. Unpublished doctoral dissertation, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA. (Available as Technical Report CMU-CS-02–105.)Google Scholar
Rosch, E. R., Mervis, C. B., Gray, W., Johnson, D. & Boyes-Braem, P. (1976) Basic objects in natural categories. Cognitive Psychology 8:382–439.CrossRefGoogle Scholar
Rumelhart, D. E. (1977) Toward an interactive model of reading. In: Attention and performance, vol. VI, ed. Dornic, S., pp. 573–603. Erlbaum.Google Scholar
Rumelhart, D. E. (1990) Brain style computation: Learning and generalization. In: An introduction to neural and electronic networks, ed. Zornetzer, S. F., Davis, J. L. & Lau, C., pp. 405–20. Academic Press.Google Scholar
Rumelhart, D. E. & McClelland, J. L. (1986) PDP models and general issues in cognitive science. In: Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1, ed. Rumelhart, D. E., McClelland, J. L. & the PDP Research Group, pp. 110–46. MIT Press.CrossRefGoogle Scholar
Rumelhart, D. E., McClelland, J. L. & the PDP Research Group, eds. (1986b) Parallel distributed processing: Explorations in the microstructure of cognition, vol. 2. MIT Press.CrossRefGoogle Scholar
Rumelhart, D. E., Smolensky, P., McClelland, J. L. & Hinton, G. E. (1986c) Schemata and sequential thought processes in PDP models. In: Parallel distributed processing: Explorations in the microstructure of cognition, vol. 2, ed. McClelland, J. L., Rumelhart, D. E. & the PDP Research Group, pp. 7–57. MIT Press.CrossRefGoogle Scholar
Rumelhart, D. E. & Todd, P. M. (1993) Learning and connectionist representations. In: Attention and performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience, ed. Meyer, D. E. & Kornblum, S., pp. 3–30. MIT Press.CrossRefGoogle Scholar
Schapiro, A. C. & McClelland, J. L. (in press) A connectionist model of a continuous developmental transition in the balance scale task. Cognition.Google Scholar
Siegler, R. S. & Chen, Z. (1998) Developmental differences in rule learning: A microgenetic analysis. Cognitive Psychology 36(3):273–310.CrossRefGoogle ScholarPubMed
Smolensky, P. (1988) On the proper treatment of connectionism. Behavioral and Brain Sciences 11(1):1–23; discussion 23–74.CrossRefGoogle Scholar
St. John, M. F. & McClelland, J. L. (1990) Learning and applying contextual constraints in sentence comprehension. Artificial Intelligence 46:217–57.CrossRefGoogle Scholar
Tanaka, J. & Taylor, M. (1991) Object categories and expertise: Is the basic level in the eye of the beholder? Cognitive Psychology 23:457–82.CrossRefGoogle Scholar
Tenenbaum, J. B., Griffiths, T. & Kemp, C. (2006) Theory-based bayesian models of inductive learning and reasoning. Trends in Cognitive Science 10(7):309–18.CrossRefGoogle ScholarPubMed