1. Connectionist and dynamical systems approaches
to cognition (McClelland, et al., 2010)
Jennifer D’Souza
Donald Kretz November 17, 2010

2.  Definitions and main concepts
 Connectionist and probabilistic models
 General arguments against probabilistic models
 Cognitive areas where models are useful
◦ Language
◦ Development
◦ Semantics
 Conclusions
 Questions and Discussion

4.  Connectionism - neurons are the basic information
processing structures in the brain, and every sort of
information the brain processes occurs in networks of
interconnected neurons. Models knowledge and knowledge
acquisition as adjusting strengths on connections of networks
of “neuron-like processing units”
Neural Networks

5.  Structured probabilistic models

6.  Semantic Cognition
 Universal grammar

7. Explore the claim
Arguments against
probabilistic models
Cognitive Domain –
Emergent Phenomena Models
Conclusion

8.  Main idea: How low-level, highly localized,
non-cognitive processes can combine to
produce cognition
 Claim: only models that account for how
neurons connect (“mechanism”) can fully
explain cognition

10.  Do the ants possess a blueprint for creating
this structure?

11.  “Human thoughts, language and behavior
have a rich and complex structure that is the
emergent consequence of a large number of
simpler processes.”

12. Looking + discriminating locations + posture control
+ motor planning
Reaching Action
Emergent Consequence

13. Reading 15
Griffiths, T.L. et al. (2010) Probabilistic models
of cognition: Exploring the laws of thought

14. Cognizing
Agents
Hypothesis Space with a
prior probability
distribution
Observations
Result with highest
posterior
probability
InferenceInput
Evaluation

15. Why is consideration of the structured probabilistic
approach to cognition dangerous?

16.  Computational Level – what does the system do?
 Algorithmic Level – how does the system do what it
does?
 Implementational Level – how is the system
physically realized?

17. Structured Probabilistic Inference
Model
Chomsky’s competence-based
approach to linguistics
Probabilistic Inference Problem Goal is to characterize language
user’s knowledge
Select the correct knowledge
structure
Grammar as representation that
explain the facts
Abstraction from cognitive tasks –
Marr’s computational level
Theory is pitched at a competence
level
Cognitive performance remains as a “promissory note”.

18.  Problem formulation is not neutral.
 Is there a generalized knowledge
representation technique?
◦ Propositional Logic
◦ First-order Logic
“There is someone who can be fooled every time.”

19.  Treating levels of analysis as independent is
counter-productive.
 Level of description and competence /
performance approaches also introduce a
comfortable extra degree of freedom w.r.t.
data.
High level computational theories Behavior

20.  Explicit inferences in contingency learning
task
Ample time to make
response
Quick response – Time
constrained
Exploit the causal
framing scenario to make
normatively correct,
explicit inferences.
Learning process –
simple connection weight
adjustments.
How can the statistical structure or the computational-level analysis of
what would be optimal be the same?

21.  NOT about probabilities – both approaches
emphasize statistics
 NOT about bottom-up over top-down – both
are important
 IS about cognition being a choice of statistical
models

22.  The utility of the structured probabilistic
approach depends in part on the validity of
the units as descriptions of linguistic
structure. Herein lies the problem.
◦ Hypothesis space in the case of language would be
characterized by discrete units such as phonemes,
morphemes and sentences.

23.  From citation 37 
◦ As far as phonemes are concerned:-
◦ High frequency words – just, went or don’t all have
the final /t/ or /d/ deleted than,
◦ Low frequency words – innocent, interest or attract
◦ High frequency words -
 Every – 2 syllable word
◦ Lower frequency words –
 Mammary, Summary – 3 syllable words
 Memory, family – anywhere between 2 to 3 syllables

24.  From citation 36 
◦ As far as morphemes are concerned:-
◦ Derived forms that are more frequent than their
base should be less decomposable, than derived
forms that are less frequent than their base.
◦ Conclusion – Relative frequency matters more than
absolute frequency.

25. Arguments against probabilistic approaches
 No real basis – not representative of the actual processes – use
of probabilistic models is unnecessary and dangerous
 Don’t account for (explain) the development of cognitive abilities
 What if the high-level models are wrong?
Source of disagreement
 NOT about probabilities – both approaches emphasize statistics
 NOT about bottom-up over top-down – both are important
 IS about cognition being a choice of statistical models

26.  Language
◦ Tense, word reading, sentence processing
 Development
◦ Stage transitions, walking
 Semantics
◦ Representing living vs. nonliving things

27. For each cognitive area, the authors present:
 Their interpretation of probabilistic modeling
approaches
 Examples of probabilistic models gone awry
 An explanation of how a connectionist model
better accounts for cognitive development
and activity

28.  Probabilistic:
◦ Problem formulation
◦ A priori assignment of outcomes and probabilities
◦ Abstraction from mechanistic details
 Chomsky’s universal grammar
◦ Formulation: characterizing knowledge of language
user assuming that is the user’s goal
◦ Commitment: selecting grammar that explains such
knowledge but may not select the grammar that
convergence would
◦ Abstracting at the competence level but may not
map directly to behavior or neurophysiological
details

29.  Probabilistic:
◦ Characterized in discrete units
 Elemental structure
◦ Phonemes, morphemes, sentences but these are
matters of degree and may be misleading
approximations

30.  Connectionist:
◦ No fixed vocabulary of representational units
◦ Graded patterns of distinctness, compositionality,
and context sensitivity

31.  Probabilistic:
◦ Stages of development (Piaget)
◦ Object permanence
 A-not-B task
◦ Objects exist independent of one’s own action
◦ Not an explicit focus of research in probabilistic
modeling

32.  Connectionist:
◦ Dynamic Field Theory – integrates multiple sources
of relevant information
◦ Situation (events, past reaches, object positions)
◦ Motor planning (direction of next reach)

33.  Probabilistic:
◦ Acquiring semantic knowledge represents a choice
among alternatives
◦ Requires knowledge of hypothesis space, space of
possible choices, prior distributions
 Taxonomic hierarchy of nature
◦ Separate branches such as birds, fishes, and
mammals do not account for partial homologies
◦

34.  Connectionist:
◦ Learn a set of weights
◦ Discrepancies between predicted and observed
outcomes serve as feedback to weight adjustment
◦ Related items evoke similar but differentiated
internal representations

35.  Cognition depends fundamentally on
underlying mechanism – abstract models will
miss important aspects
 Connectionist modeling efforts have led to
advances in cognitive theories
 Authors advocate an integrated approach
where high-level models are informed by
knowledge about underlying neural
mechanisms

36. 1. Why are we interested in modeling cognition?
2. Is there always a need to choose one modeling
approach over the other?
3. Did the authors convince you that higher cognitive
abilities can be modeled at the connectionist level?
4. “Understanding how each and every neuron
functions still tells us absolutely nothing about how
the brain manufactures a mental state.” (Gazzaniga,
2010)
5. Are there other types of symbolic models (other
than probabilistic) that may be appropriate for
cognitive modeling?

37. Emergence as patterns observed from activations and
inhibitions across connections of neurons.
 Note that the term "emergent" was coined by the
pioneer psychologist G. H. Lewes, who wrote:
"Every resultant is either a sum or a difference of the co-
operant forces; their sum, when their directions are the
same -- their difference, when their directions are
contrary. Further, every resultant is clearly traceable in its
components, because these are homogeneous and
commensurable. It is otherwise with emergents, when,
instead of adding measurable motion to measurable
motion, or things of one kind to other individuals of their
kind, there is a co-operation of things of unlike kinds.
The emergent is unlike its components insofar as these
are incommensurable, and it cannot be reduced to their
sum or their difference." (Lewes, G. H. (1875), Problems
of Life and Mind (First Series), 2, London: Trübner)