Department of Psychology

Storage capacity of the linear associator : beginnings of a theory of computational memory

Dean C. Mumme et al. — Wed, 23 May 2012 07:33:12 PDT

Abstract: "This paper presents a characterization of a simple connectionist-system, the linear-associator, as both a memory and a classifier. Toward this end, a theory of memory based on information-theory is devised. The principles of the information-theory of memory are then used in conjunction with the dynamics of the linear-associator to discern its storage capacity and classification capabilities as they scale with system size. To determine storage capacity, a set of M vector-pairs called 'items' are stored in an associator with N connection-weights. The number of bits of information stored by the system is then determined to be about (N/2) log[subscript 2]M.The maximum number of items storable is found to be half the number of weights so that the information capacity of the system is quantified to be (N/2)log[subscript 2]N. Classification capability is determined by allowing vectors not stored by the associator to appear at its input. Conditions necessary for the associator to make a correct response are derived from constraints of information-throughput of the associator, the amount of information that must be present in an input-vector and the number of vectors that can be classified by an associator of a given size with a given storage load.Figures of merit are obtained that allow comparison of capabilities of general memory/classifier systems. For an associator with a simple non-linearity on its output, the merit figures are evaluated and shown to be suboptimal. Constant attention is devoted to relative parameter size required to obtain the derived performance characteristics. Large systems are shown to perform nearest the optimum performance limits and suggestions are made concerning system architecture needed for best results. Finally, avenues for extension of the theory to more general systems are indicated."

AHA! : a connectionist perspective on problem solving

Craig A. Kaplan et al. — Wed, 23 May 2012 07:33:02 PDT

Abstract: "The AHA! model is proposed as a demonstration of what connectionism might have to offer the study of problem solving. Bridging the Gestalt and Problem Space theories of problem solving, AHA! simulates serial search at a macro-level while incorporating (at a micro-level) the Gestaltist idea of a dynamic interaction between parts of the problem and the goals and knowledge of the problem solver. AHA! exhibits a number of problem solving phenomena including insight, directed search, goal fixedness, einstellung, functional fixedness, and responsiveness to the salience of problem features. It provides not only qualitative fits to human data from functional fixedness experiments, but also a framework for exploring the potential outcomes of variations which could be carried out in future experiments."

On the control of automatic processes : a parallel distributed processing model of the Stroop effect

Jonathan Cohen et al. — Wed, 23 May 2012 07:32:51 PDT

Abstract: "A growing body of evidence suggests that traditional views of automaticity are in need of revision. For example, automaticity has often been treated as an all-or-none phenomenon, and traditional theories have held that automatic processes are independent of attention. Yet recent empirical data suggest that automatic processes are continuous, and furthermore are subject to attentional control. In this paper we present a model of attention which addresses these issues. Using a parallel distributed processing framework we propose that the attributes of automaticity depend upon the strength of a process and that strength increases with training. Using the Stroop effect as an example, we show how automatic processes are continuous and emerge gradually with practice.Specifically, we present a computational model of the Stroop task which simulates the time course of processing as well as the effects of learning. This is done by combining the cascade mechanism described by McClelland (1979) with the backpropagation learning algorithm (Rumelhart, Hinton, & Williams, 1986). The model is able to simulate performance in the standard Stroop task, as well as aspects of performance in variants of this task which manipulate SOA, response set, and the number of competing words in the display. These simulations demonstrate that when two processes are in competition, the weaker process can take on several of the attributes that have previously been associated with controlled processing: susceptibility to interference and a requirement for the allocation of attention. This suggests that the traditional distinction between controlled and automatic processing is in need of reconsideration."

Developmental differences in scientific discovery processes

Kevin Dunbar et al. — Wed, 23 May 2012 07:32:40 PDT

Information processing approaches to cognitive development

David Klahr et al. — Wed, 23 May 2012 07:32:29 PDT

Abstract: "This chapter reviews the history and current status of information-processing approaches to cognitive development. Because the approach is so pervasive, it is useful to characterize research in terms of distinctive features, and to organize the features according to whether they are 'soft-core' or 'hard-core' aspects of the information processing approach."

Problem formulation and alternative generation in the decision making process

Herbert Alexander Simon et al. — Wed, 23 May 2012 07:32:17 PDT

Abstract: "Classical and neoclassical economic theory, as well as statistical decision theory, through their neglect of human bounded rationality -- the vast disparity between human computing capabilities and the complexity of our world -- both give a seriously distorted picture of human decision making and omit at least three components of the decision making process that are of central importance. In this paper, I will outline what is known, today, about these neglected aspects of human decision-making. A great deal is known, mainly as a result of the progress of cognitive science in the last generation. Economics can make rapid progress by drawing upon this storehouse of new knowledge to reconstruct and expand its foundations."

Context effects in letter perception : a comparison of two theories

Howard B. Richman et al. — Wed, 23 May 2012 07:32:05 PDT

Abstract: "The purpose of this study is to test whether EPAM (Elementary Perceiver and Memorizer) can explain context effects in letter recognition. EPAM, a model of learning and recognition in the form of a computer program, has successfully explained many aspects of learning and perception in a range of task environments. In 1984, Barsalou and Bower claimed that EPAM could not explain the phenomena in the tachistoscopic perception experiments successfully simulated by the Interactive Activiation Model (IAM) of word perception (McClelland & Rumelhart, 1981; Rumelhart & McClelland, 1982).In this study, using EPAM IV, a revision of the most recent version of EPAM, we show that the human data modeled by the IAM are at least as accurately simulated by EPAM. Close examination of the performance of the two programs shows that the fact that one (EPAM) processes perceptions serially, while the other (IAM) processes them in parallel, plays no essential role in producing the context effects that are observed. The main effects are produced, in both programs, by a feedback of information from word recognition to the process of recognizing letters."

Foundations of cognitive science : overview

Herbert Alexander Simon et al. — Wed, 23 May 2012 07:31:53 PDT

Abstract: "Intelligence is closely related with adaptivity -- with problem-solving, learning, and evolution. A science of intelligent systems has to be a science of adaptive systems, with all which that entails for the difficulty of finding genuine invariants. Some of the invariance in intelligence is imposed by the structure of the inner environment -- the limits, for example, of human short term memory. Some of it is imposed by the outer environment, the need to search very large spaces selectively. Some of the invariance is to be found in the structure of learning systems, rather than in the highly adapted performance systems they produce. But, in cognitive science we must be prepared to recognize that the invariants in an adaptive system are likely to be limited to specific times and places; that in the long run, almost any aspect of them can change adaptively."

Using rules and task division to augment connectionist learning

William L. Oliver et al. — Wed, 23 May 2012 07:31:40 PDT

Abstract: "Learning as a function of task complexity was examined in human learning and two connectionist simulations. An example task involved learning to map basic input/output digital logic functions for six digital gates (AND OR, XOR and negated versions) with 2- or 6-inputs. Humans given instruction learned the task in about 300 trials and showed no effect of the number of inputs. Backpropagation learning in a network with 20 hidden units required 68,000 trials and scaled poorly, requiring 8 times as many trials to learn the 6-input gates as to learn the 2-input gates. A second simulation combined backpropagation with task division based upon rules humans use to perform the task. The combined approach improved the scaling of the problem, learning in 3,100 trials and requiring about 3 times as many trials to learn the 6-input gates as to learn the 2-input gates. Issues regarding scaling and augmenting connectionist learning with rule-based instruction are discussed."

The role of practice in dual-task performance : toward workload modelling in a connectionist control architecture

Walter Schneider et al. — Wed, 25 Apr 2012 13:22:07 PDT

Abstract: "The literature on practice effects and transfer from single- to dual-task performance and part-whole task learning are briefly reviewed. The results suggest that single-task training produces limited transfer to dual-task performance. Past theoretical frameworks for multi-task performance are reviewed. A connectionist/control architecture for skill acquistion is presented. The architecture involves neural-like units at the microlevel, with information transmitted on vectors between modules at the macrolevel. The simulation of the model exhibits five phases of skill acquisition. Dual-task interference and performance are predicted as a function of the phase of practice the skill has reached.Seven compensatory activities occur in the model during dual-task training that do not appear in single-task training: 1) task shedding, delay and buffer pre-loading; 2) letting go of high-workload strategies; 3) utilizing noncompeting resources; 4) time multiplexing; 5) shortening transmissions; 6) converting interference from concurrent transmissions; and 7) chunking transmissions. Future research issues suggested by the architecture include: Mapping out the marginal utility of single- to multi-task transfer; investigating the classificationof multi-task compensatory activities; evaluating the role of part-task trainers for multi-task skills; and developing and testing quantitative models of skill acquisition."

Prediction and prescription in systems modeling

Herbert Alexander Simon et al. — Wed, 25 Apr 2012 13:21:52 PDT

Abstract: "In modeling, as in any other human activity, there is a certain amount of historical inertia. Predictive models bulked large in the early history of modeling. In our enthusiatic efforts to explore the potential of computers we tended to conceptualize computers as number crunchers, and were not immediately able to see the potential for qualitative and symbolic modeling that did not use numbers.Before we begin a modeling task, we need to answer the following: Whether we need temporal detail, and if so, what amount can be supported by the kinds of data and theories that we have available; whether a good understanding of steady states may be more important to us than tracing paths; whether we can simplify the systems we are modeling by making use of their hierarchical properties to aggregate, or in other way [sic]; are there aspects of the situation of interest that are better modeled symbolically, in words or pictures, rather than numberically [sic]?"

Learning and applying contextual constraints in sentence comprehension

Mark St. John et al. — Wed, 25 Apr 2012 13:21:37 PDT

Abstract: "A parallel distributed processing model is described that learns to comprehend single clause sentences. Specifically, it assigns thematic roles to sentence constituents, disambiguates ambiguous words, instantiates vague words, and elaborates implied roles. The sentences are pre-segmented into constituent phrases. Each constituent is processed in turn to update an evolving representation of the event described by the sentence. The model uses the information derived from each constituent to revise its on-going interpretation of the sentence and to anticipate additional constituents. The network learns to perform these tasks through practice on processing example sentence/event pairs.The learning procedure allows the model to take a long-range statistical approach to solving the bootstrapping problem of learning the syntax and semantics of a language from the same data. The model performs very well on the corpus of sentences on which it was trained, but learns slowly."

Using the expert's diagrams as a specification of expertise

Stephen H. Casner et al. — Wed, 25 Apr 2012 13:21:20 PDT

Abstract: "This work explores the use of diagrams in generating executable specifications of expert knowledge. We make the observation that experts frequently use diagrams as an efficient means of communicating detailed information. For some types of information diagrams might offer the expert an alternative to the high cost of understanding existing knowledge representation formalisms. We are interested in accomplishing three things: 1) understanding the diagramming techniques used by domain experts to encode detailed information in a restricted type of diagram called a relational diagram; 2) characterizing a set of notions that experts frequently encode in relational diagrams; 3) developing an environment that allows experts to partially construct a formal specification of problem domain knowledge by drawing relational diagrams.We describe BOS, a diagramming tool that allows domain experts to build a customized set of diagramming conventions suitable to their problem domain. Diagrams drawn with BOS generate formal specifications that reduce the need to establish the diagram's meaning through accompanying text or verbal explanation. BOS is currently able to generate frames and rules from an interesting set of relational diagrams that allow the use of spatial arrangement and connectivity to represent notions about problem domain entities, part of relations, constraints, temporal ordering, and procedural steps."

Communicating with high-level plans

Jeffrey Bonar et al. — Wed, 25 Apr 2012 13:21:06 PDT

Abstract: "We discuss our experience with an interface that gives users the ability to directly represent and manipulate goals at several levels of detail. The interface is built into Bridge, a tutorial environment for novice programmers. The name comes from our intended 'bridge' between novice and expert conceptions of programming. In order to understand student designs and partial programs, Bridge provides languages that allow a student to talk about his or her high-level designs and partial work. We call the vocabulary of these languages plans. Plans are bundles of knowledge about the standard subtasks in a domain, designed and organized based on a typical user's point of view."

The place of cognitive architectures in a rational analysis

John R. Anderson et al. — Wed, 25 Apr 2012 13:20:51 PDT

Abstract: "It is argued that human cognition can be predicted from the assumption that it is optimized to the information-processing demands that are placed on it. Results that are taken in support of particular architectures (PDP, ACT*, SOAR) are shown to be consequences of this rationality principle of human cognitions. Implications of this rationality principle for cognitive architectures are discussed."

Parallel distributed processing : implications for cognition and development

James L. McClelland et al. — Wed, 25 Apr 2012 13:20:37 PDT

Abstract: "This paper provides a brief overview of the connectionist or parallel-distributed processing framework for modeling cognitive processes, and considers the application of the connectionist framework to problems of cognitive development. Several aspects of cognitive development might result from the process of learning as it occurs in multi-layer networks. This learning process has the characteristic that it reduces the discrepancy between expected and observed events. As it does this, representations develop on hidden units which dramatically change both the way in which the network represents the environment from which it learns and the expectations that the network generates about environmental events.The learning process exhibits relatively abrupt transitions corresponding to stage shifts in cognitive development. These points are illustrated using a network that learns to anticipate which side of a balance beam will go down, based on the number of weights on each side of the fulcrum and their distance from the fulcrum on each side of the beam. The network is trained in an environment in which weight more frequently governs which side will go down. It recapitulates the states of development seen in children, as well as the stage transitions, as it learns to represent weights and distance information."

Soar : an architecture for general intelligence

Laird et al. — Wed, 25 Apr 2012 13:20:22 PDT

Abstract: "The ultimate goal of work in cognitive architecture is to provide a foundation for a system capable of general intelligent behavior. That is, the goal is to provide the underlying structure that would enable a system to perform the full range of cognitive tasks, employ the full range of prolem-solving methods and representations appropriate for the tasks, and learn about all aspects of the task and its performance on them. In this article we present Soar, an implemented proposal for such an architecture. We describe its organizational principles, the system as currently implemented, and demonstrations of its capabilities."

Knowledge level learning in Soar

Paul S. Rosenbloom et al. — Wed, 25 Apr 2012 13:20:05 PDT

Abstract: "In this article we demonstrate how knowledge level learning can be performed within the Soar architecture. That is, we demonstrate how Soar can acquire new knowledge that is not deductively implied by its existing knowledge. This demonstration employs Soar's chunking mechanism -- a mechanism which acquires new productions from goal-based experience -- as its only learning mechanism. Chunking has previously been demonstrated to be a useful symbol level learning mechanism, able to speed up the performance of existing systems, but this is the first demonstration of its ability to perform knowledge level learning. Two simple declarative-memory tasks are employed for this demonstration: recognition and recall."

The chunking of skill and knowledge

Paul S. Rosenbloom et al. — Wed, 25 Apr 2012 13:19:49 PDT

Abstract: "This article describes recent work that utilizes the concept of chunking as the basis for an integrated model of the acquisition of both skill and knowledge. We look at results in the areas of practice (skill) and verbal learning (knowledge). The approach is based on viewing task performance as a problem solving process and chunking as a learning process that stores away information about the results of problem solving. In practice tasks, chunks acquired during the solution of one problem can be used during later problems to speed up the system's performance. This chunking process produces the same type of power-law practice curves that appear so ubiquitously in human practice. In verbal learning tasks, chunks acquired during training are used at test time to determine how to respond. This psychological model is a manifestation of a set of processes that provide the basis of a general architecture. Such an architecture is not only interesting in its own right, but provides support for the more narrowly based psychological phenomena."

Varieties of learning in Soar : 1987

D. M. Steier et al. — Wed, 25 Apr 2012 13:19:33 PDT

Abstract: "Soar is an architecture for intelligence that integrates learning into all of its problem-solving behavior. The learning mechanism, chunking, has been studied experimentally in a broad range of tasks and situations. This paper summarizes the research on chunking in Soar, covering the effects of chunking in different tasks, task-independent applications of chunking and our theoretical analysis of effects and limits of chunking. We discuss what and when Soar has been able to learn so far. The results demonstrate that the variety of learning in Soar arises from variety in problem solving, rather than from variety in architectural mechanisms."