Representations Overheads

KNOWLEDGE REPRESENTATION WHAT ARE MENTAL REPRESENTATIONS? Mental representations are the FORM for what you KNOW (inside your mind) about things, ideas, events, etc… … that exist outside of your mind Mental representations are the mental units that we use to STORE and PROCESS information. Cognitive activities involve the MANIPULATION and TRANSFORMATION of these mental representations.

WHAT FORM(S) DO OUR MENTAL REPRESENTATIONS TAKE? DUAL-CODE THEORY (Allan Paivio, 1969) We use both IMAGE and VERBAL codes for representing information. MENTAL IMAGES are ANALOGUE codes 

ANALOGUES are representations of specific objects that preserve the features and properties of the realworld object



May represent things that have never been observed by your senses at any time (even things that do not exist at all outside your mind!)

VERBAL information is represented in SYMBOLIC code 

SYMBOLIC representations are arbitrary: there is nothing about the symbol that depicts the actual properties of the object it represents



Use of symbols needs to be governed by rules



Information associated with words is abstract and general and may apply to classes or categories

Experiments have demonstrated that we do process verbal information differently from imaginal information! Brooks, 1968  a task involving visual perception interfered with a visual imaging task



response involving verbal expression interfered with a verbal processing task

PROPOSITIONAL THEORY: One Universal Format Knowledge is mentally represented in terms of their underlying meanings, or as abstract propositions. Images and words are mere byproducts of more basic cognitive processes involving propositions. A PROPOSITION 

is the smallest possible meaningful assertion about the world that can be evaluated as true or false (Boole again!)



uses shorthand “predicate calculus” to express meanings, stripping away superficial differences in how we express ideas

(relationship between elements) ([subject] [object])

Where did this idea come from? The mind-as-computer metaphor: Thinking IS computation! Computers can process many different kinds of information using a universal binary format… … as the human mind is an even more intelligent information-processing machine, then at its deepest level of processing, we must also be using a basic, universal mental format

Evidence that propositional codes may shape or override imaginal codes: 

Carmichael, Hogan & Walter (1932)



Chambers & Reisberg (1985)

MENTAL IMAGERY I see meaning as the blue-grey tip of a kind of scoop, which has a bit of yellow above it (probably a part of the handle), and which is just digging into a dark mass of what appears to be plastic material. I was educated on classical lines; and it is conceivable that this picture is an echo of the oft-repeated admonition to “dig out the meaning” of some passage of Greek or Latin. I do not know; but I am sure of the image. And I am sure that others have similar images. (Edward Titchener) What does a person mean when he closes his eyes…and says, “I see the house where I was born…I can even see my mother as she tucks me in…” Touching, of course, but sheer bunk. We are merely dramatizing. The behaviorist finds no proof of imagery in all this. We have put these things in words long, long ago and we constantly rehearse those scenes verbally whenever the occasion arises. (John Watson)

My particular ability does not lie in mathematical calculation, but rather in visualizing effects, possibilities, and consequences. (Albert Einstein)

What is a mental image? Are mental images like “pictures” in our mind? Where are these pictures located? Who is “looking at” these pictures? (the physical eye cannot be turned inward!)

What do we DO with mental images? Is thinking in images more like computing with graphics or more like computing with a database of propositions? Are images different from other forms of thought/representation? FUNCTIONAL EQUIVALENCE Hypothesis: Mental imagery and visual perception use the same operations

The same brain processes and parts (the visual cortex) are involved in visual perception and mental imagery:   

interference studies brain damage studies analogous effects when images are used in place of sensory input

Images are ANALOGUE representations: Every part of the image corresponds to parts of the actual stimulus Capture the geometry of the stimulus, not just its meaning

Functional Properties of Mental Images We possess a well-stocked mental toolbox of graphic operations, such as zooming, shrinking, panning, scanning, tracing, and coloring!

Mental Rotations Shepard & Metzler, 1971 Findings are functionally equivalent to what we might expect if the participants had been rotating physical objects in space 

rotating objects at larger angles of rotation takes longer, whether clockwise, counterclockwise, or 3D

Image Scanning Kosslyn, Ball & Reiser, 1978 The relative distances in physical stimuli are replicated in the mental image.



There is a linear relationship between the relative distances of objects on the map and the amount of time for participants to “scan” or travel from one point to another

Image Scaling Kosslyn, 1975 On our “mental screen” for visual images, our “screen resolution” is more detailed for objects that are larger. 

we “zoom in” to see parts of objects imaged to be smaller (and blurrier), and thus take longer to respond

more quickly verify non-distinct properties of animals if large-scaled (does a cat have knees?), versus distinct properties of animals that are small-scaled (does a cat have claws?) 

Imagery is a wonderful faculty, BUT…  





Images are fragmented Images are slaves to the organization of memory and conceptual knowledge Images cannot serve to represent our concepts Images are ambiguous

Arguments against mental imagery as a form of representation: Imagery is a by-product of symbolically encoded propositions and meanings Imagery is cognitively penetrable: it is at the mercy of propositions and is NOT a default mechanism of the cognitive system Thus, images cannot contribute to our knowledge of how the mind works! Kosslyn argues back… Useless to argue whether images are an irreducible form of mental representation Simply, mental images have properties that distinguish it as a distinctive and functional form of representation!

Propositional Representations of Underlying Meanings Type of Representation Relationshi in Words p Actions A mouse bit a cat.

Propositional Representation Bite [action] (mouse [agent of action], cat [object of action])

Attributes

Mice are furry.

[external surface characteristic] (furry [attribute], mouse [object])

Spatial positions

A cat is under the table.

[vertically higher position] (table, cat)

Class A cat is an animal. membership

[category membership] (animal [category], cat [member])

Imaginal Representati on