Learning Chapter 3

Chapter 3 Pavlovian Conditioning

Pavlovian Responses: It’s not just about salivation..

Classical conditioning applies to responses that we do not experience as voluntary, purposeful, or under our willful control.

They involve “passive” responses---responses that “happen to us,” or that we undergo. We salivate, but we don’t “do salivation” like we “do our chores.” salivation, eye blinks, startle, nausea, tears-

but also….

pain

attraction

anxiety

arousal

fear

happiness

disgust

amusement

sadness

relaxed state

repulsion

love (the kind you fall into)

anger A neurological view of Pavlovian responses has also been proposed---

The Nervous System

Operant Conditioning Classical Conditioning

Autonomic Nervous System (ANS)

Sympathetic NS “Arouses” (fight-or-flight) Parasympathetic NS “Calms” (rest and digest)

The basic form of classical cond. A new stimulus gets paired with



another stimulus that produces a predictable, automatic response (reflex or autonomic activity). The new stimulus can then come to elicit the original automatic response.

The basic form of classical cond. Loud thunder automatically causes startle/fear + After repeated pairing of lightning with thunder, lightning itself produces the response

Fears/Phobias Being stuck with a needle causes pain/fear

+

Pain/Fear/Stress

Fear/Stress

Now the sight of a syringe cause fear/stress

Attraction Closeness/Caress produces attraction

+

Arousal/Attraction

Arousal/Attraction

Security blankets, fetishes, inappropriate attractions (pedophilia)

Food/Taste Aversions Alcohol causes nausea, dizziness, vomiting

+

Nausea/Vomiting

Sight, taste, smell

Revulsion/Disgust

Sight, taste, smell

Same with foods. The illness might even be due to other conditions, such as viruses, flu, etc

Musical Responses Theme from Jaws

Tubular Bells

“Our Song”

+

+

+

Fear

Fear

Happy

Unpopular Mustache

Ivan Pavlov (1849-1936) Russian Physiologist b. Ryazan, Russia Poor: Father a priest Started as seminarian. Left to study physical sciences, U. St. Petersburg Specialty: Digestion

Nobel Prize, 1904, for his work on the digestive system

Pavlov in his lab with his favorite experimental subject

Pavlov’s accidental discovery 



Wanted to measure quantity and composition of saliva in response to food in mouth. Problem: dogs began to salivate before food was administered, to everyone’s annoyance. 





Salivated to sight of researchers, door opening, footsteps, bowl

Pavlov called this phenomenon “Psychic Secretions” He realized he had discovered something psychological, and, after much debate, directed his research to

Pavlov’s Experiments Before conditioning, food produces salivation. However, a tone (neutral stimulus) does not.

Pavlov’s Experiments During conditioning, the neutral stimulus (tone) and the food are paired, resulting in salivation. After conditioning, the neutral stimulus (tone) elicits salivation.

Formal Definitions 

An unconditioned stimulus (US) 

A stimulus that triggers an unconditioned response. 



Examples: food, loud noises, painful stimuli

An unconditioned response (UR) 

An unlearned response to an unconditioned stimulus. 

Examples: salivation to food, jumping when hearing a loud noise, moving away from something painful

Formal Definitions 

A conditioned stimulus (CS) 

A neutral stimulus (an event) that comes to evoke a classically conditioned (learned) response due to being presented shortly before the US. 



In Pavlov’s experiments, the CS was the tone.

A conditioned response (CR) 

A learned response to a classically conditioned stimulus. 

In Pavlov’s experiments, salivation to the bell was the CR.

Learning the 4 Key Terms CS

US UR

CR

First, identify the existing “reflex” This reflexpair pair is always the USUR 

The rest is easy….

Nice Notation….  



US  UR (identify the reflex pair) CS : US  UR (pair a neutral stim. with the US) CS  CR (CS elicits response; now called CR)   

Food  Salivation Tone : Food  Salivation Tone  Salivation

Of what use is Pavlovian conditioning? You are walking through the woods when you 











hear an eery rattling noise arising from several spots around you. You pause, listen carefully, and keep walking, wondering what the noise might be. After a few steps, you feel a sharp pain on your ankle. Then you see the snake, still hooked onto you. You are terrified, but manage to pull the snake loose. You walk back to your car, call a nearby hospital for advice, and drive to the hospital for treatment. You’ll be fine, but you go through several of the most painful days of your life. What will happen next time you hear the rattling noise in the woods?

What use? 





Based on our experiences, classical condition builds a set of signals or predictive cues for potentially important things to come. Builds an early “warning” system: even warning us of good things to come. We become physiologically prepared for what might be coming next.

Factors that affect Pairing CS and US (4 ways) conditioning    

Contiguity (closeness in time) Contingency (predictive value of CS) Stimulus Features  



Prior Experience  

 

Overshadowing: Salience, intensity, Sensory preconditioning Latent Inhibition Blocking

Number of Pairings Intertrial Interval

Just relax……

Puff  Blink Bell : Puff  Blink Bell  Blink

Contemporary human eyeblink preparation

Pairing: Trace Conditioning The CS begins and ends before the US is presented.

Delayed Conditioning The CS and US overlap— i.e., the US is presented while the CS is still being presented

Simultaneous Conditioning The CS and US coincide exactly in time.

Backward Conditioning The CS comes after the US

Which pairing procedure works best for creating learning? 

Given our notion that classical conditioning establishes signals or cues for important future events, which procedures might be best? Which worst?



Think of the rattle snake.

Which pairing works best? 



 

All things being equal, delayed conditioning is most effective. Followed very closely by trace conditioning. Simultaneous conditioning is weak. Backward conditioning is the weakest, One of Pavlov’s students found that when the smell of vanilla (CS) came almost impossible. before the US (acid in the mouth), conditioning of salivation occurred in 20 trials. But when the smell came after the US, conditioning did not occur even after 427 pairings (after which he gave up).



Why are the last two so ineffective? They do not signal what is coming up.

Another Factor: CS-US Contiguity 





Contiguity—closeness in time between two events. In classical conditioning, CS-US contiguity refers to the interval of time between the CS and US. This time is called the Interstimulus Interval (ISI) 

Defined differently for trace and delayed conditioning.

CS-US Contiguity Interstimulus Interval (ISI)

Interstimulus Interval (ISI)

Delay Conditioning Short-Delay  US begins shortly (a second or less) after the CS begins.

Long-Delay  US begins several seconds or minutes after the CS begins.

CS-US Contiguity 





In general, the shorter the ISI, the quicker the learning. With simultaneous conditioning, in which there is no interval at all, learning is very slow. Rule of Thumb: Make the interval as short as possible, just short of simultaneous presentations. 

NB: This rule of thumb has many exceptions, depending on the species of organism, the kind of response being

Another Factor: Contingency 

Contingency is related to prediction— does one event, A, predict another event, B 



In classical conditioning, it has to do with the consistency of pairing CS and US. 





If A, then B

If CS, then US (follows)

If the CS is always followed by US, you have perfect contingency. But what if the CS is followed by the US inconsistently?

Contingency: Rescorla (1968)    

Rats Shock (US) Fear (UR) CS = Tone Three conditions   



CS presented without US 10% of time 20% of time 40% of time Results:

Contingency: Rescorla (1968) Greater contingency, More learning

Stimulus Features 



Almost any stimulus that can be detected can become a CS, but some are more conditionable than others. Compound Stimuli Studies of Pavlov 

Two or more CSs presented at the same time just before the US is presented. 



[Tone & Light] : Food  Salivation

Each CS then tested individually 



Often, only one of the stumuli shows strong conditioning Overshadowing – if one CS is more intense or salient, the other CS may be “ignored”  

Strong light better than weak tone Loud tone better than weak light

Sensory Preconditioning • Sensory preconditioning is another example of stimuli influenced by compound events. • Sensory Preconditioning- two stimuli such as light and tone are repeatedly presented together without the occurrence of a US (preconditioning). • Later, only one of these stimuli (e.g., tone) is paired with a US (e.g., a shock). • Then other stimulus (light) is tested for conditioning. • Even though the second stimulus (light) was never directly associated with the US

Prior Experience with CS & US 





 

Suppose you present a CS (e.g., a tone) repeatedly all by itself (never with a US like food or shock). You then start pairing the CS with a US, trying to establish conditioning. How will the conditioning compare to standard procedures (where the CS isn’t at first repeatedly presented alone)? In general, learning is slower. Latent Inhibition: The repeated appearance of the CS without the US seems to inhibit the ability of the CS to elicit the conditioned response.

Latent Inhibition: the more pre-exposures of the CS without the US, the slower the learning.

Why does latent inhibition occur? Remember contingency effects: learning is 







stronger when the CS predicts the UCS (If CS, then US) a greater proportion of the time. In latent inhibition, there is no contingency during the initial pre-exposure period. Basically, since the CS seems unrelated to the US for a while, it takes more contingent presentations (CS  US) before the organism “believes” there is really a relationship. A completely new stimulus is a better CS than an older stimulus that didn’t seem to

Prior Experience: Blocking in Compound Stimulus Studies 





Leon Kamin discovered that sometimes a new CS cannot be established effectively. This occurs when the new CS is part of a compound stimulus study with a previously established CS. A picture is worth a thousand words….

Kamin Blocking

1. Light is established as a CS for eliciting salivation.

2. Now the light and a new CS (tone) are presented together before the US, and the compound stimulus elicits salivation.

3. Now test the original CS (light): it still elicits salivation. 4. Now try the tone by itself: conditioning does not occur. The tone was blocked by the previous conditioning of the light

Blocking 









Blocking resembles overshadowing, in which one CS in a compound stimulus gets overshadowed by another stronger or more salient CS. In overshadowing, both stimuli (CSs) in the compound are new stimuli. But in blocking, one of the CSs is previously established and is known to elicit the CR. In blocking, the new CS often doesn’t add anything more to the established CS in predicting the US. If an employee can predict the stock market with 100% accuracy, would you hire

Number of Pairings 

Each pairing of a CS with a UCS is a trial. 





The more trials conducted, the stronger the learning (sort of obvious). But the amount of learning that occurs over repeated trials is not linear—i.e., growth in learning is not constant over trials. Learning occurs more rapidly during early trials, but there appear to be diminishing returns for one’s efforts after a while.

Number of Pairings  

Acquisition (learning) curve Non-linear Asymptote

asymptote

CR Strength



Conditioning Trials

Intertrial Interval 







The intertrial interval (ITI) is the time between each CS-US pairing (i.e., between trials). Recall that the shorter the interstimulus interval (ISI), i.e., when the CS is closer to the US in time, the better the learning. But what about the interval between trials (ITI)? Is learning stronger when the time between trials is brief or longer? Counter-intuitively, more time between trials often produces better learning.

Other Variables Affecting Conditioning Age: Older people do not condition as readily as younger people.

Stress: People and other organisms condition more readily when under stress. Note: Stress hormones consolidate memories. (e.g., flashbulb memories)

Learning, then losing it:  Extinction Continued pairing of CS (tone) with US 

(food) maintains the CR (salivation), i.e., the learning is maintained. Repeated presentation of the CS without US leads to a weakening and stopping of the CR: this is called Extinction of the CR.

Pavlov’s data: After repeated trials of showing dogs food then giving it to them, the mere sight of food eventually elicits salivation. But when food was shown to dogs repeatedly without giving the food to them, then, over time, the sight of food no longer produced salivation---the response was extinguished.

Spontaneous Recovery   

After extinction, let time pass Present CS again by itself Temporary, small return of CR

Reacquisition   



Establish a CSCR connection. Extinguish CR by presenting CS alone. Try to establish the CSCR connection again. Conditioning the second time around is much quicker. Fewer trials required than the original learning prior to extinction.

All Together Now….

Strength of CR

Acquisition

CS&US

Extinction

CS alone

Trials/Time

Spontaneous Recovery Reacquisition

CS alone

CS&US

Higher Order Conditioning 





Association of CS and US is First-Order Conditioning However, CSs can be associated with other, established CSs

Second-order conditioning second-order CS

Light (CS2)

first-order CS

tone (CS1)

salivation (CR)

food (US)

Higher Order Conditioning 

Among humans, language is saturated with higher order conditioning. 

Staats & Staats (1957) 



Students observe non-sense syllables on a screen (e.g., Laj, Qug, Yof, etc). At the same time, Ss also repeat words spoken by the experimenter. 



 

Sometimes repeated positive words (joy, peace, love). Sometimes repeated negative words (sad, thief, foe) No natural US involved, just words Students then rated nonsense syllables on a scale from pleasant to unpleasant.

Students rated the “nonsense” according to the Politics and Advertising words with which they had been paired. 

Theories of Classical Conditioning 

Substitution Theory (Pavlov) 



The conditioned stimulus becomes a substitute for the unconditioned stimulus Learning depends only the number of conditioned/unconditioned stimulus pairings

Pavlov’s Substitution Theory

Pavlov believed that conditioning depended only on temporal contiguity: (c)At the start of conditioning, activity in the UCS center automatically causes activation of the UCR center. At this time activity of the CS center does not affect the UCS center. (d)(b) After sufficient pairings of the CS and UCS, their simultaneous activity causes the growth of a connection between the CS and UCS centers. Afterward, activity in the CS center will flow to the UCS center and therefore excite the UCR center.

Problems with Substitution  If substitution of CS for US is what is Theory 





really happening, then UR should be very similar to CR A lot of time, this does happen, but not always Sometimes, UR and CR look different Also, stimulus substitution theory assumes that simple contiguity (repeated pairing of CS and US) establishes conditioning.

Situation where StimulusSubstitution does not explain events Shock (US)  Jump, freeze, +heart rate (UR)

Tone: Shock (US)  Jump, freeze, +heart rate (UR) Tone (CS)  Freeze only, -heart rate (CR)

Jump (UR)

If the CR differs from the UR, simple substitution does not seem to be the full story.

Freeze (CR)

Rescorla-Wagner Model: A Mathematical Model of Classical Conditioning  Proposes that there is a limit to how much learning can occur through the pairing of a CS and US. •

A CS acquires a limited amount of associative strength on each trial (drawn from the limited pool available). –

Associative strength – amount of learning. –

–

Notation: V = associative strength

Maximum associative strength – –

Notation: λ = Vmax = Upper limit of associative strength (V)

V and Vmax (λ) λ=

Associative Strength

10

VMAX

8

Si=0.25

JVMAX=10.00

6

VSUM= 0.00 4

2

V 0 0

2

4

6

Trials

8

10

The R-W equation for a single  ΔV = c( λ – V ) CS n



n-1

ΔV = amount of change in associative strength n

on trial n  

V = associative strength on the previous trial λ = Vmax = maximum possible associative n-1

strength 

c = salience/intensity of the CS (varies from 01, with higher values meaning greater salience/intensity).



Using this equation, we can plot V (associative strength) acquired at each trial, which gives a

Varying λ (i.e., Vmax) c = .50 λ = 100

c = .50 λ = 65

R-W:Varying c (i.e., salience/intensity) Higher salience, faster learning

c = .50 λ = 100

c = .10 λ = 100

Extinction of a CS US always follows CS c = .30 λ = 100

c=.30 λ = 100 Beginning with trial 10, CS presented without US (like a bell without food)

R-W and Contingency US always follows CS c = .30 λ = 100

US only sometimes follows CS c = .30 λ = 100

Note that learning is slower — never reaches λ = 100 over same number of trials.

R-W and compound stimuli 

Competitive learning: The total learning available, λ , must be shared by each stimulus in a compound. Thus, the amount of learning to each stimulus is less in a compound than if that stimulus is alone.

R-W and Compound Stimuli: Overshadowing Two stimuli, a tone (CS1) and a light (CS2), presented simultaneously over all trials. same salience

c 1= .25 c 2 =.25 λ = 100

The more salient stimulus overshadows the other. c 1= .25 c 2 =.10 λ = 100

R-W and Blocking same salience; Both CSs present on all trials

c 1= .25 c 2 =.25 λ = 100

same salience; c 1= .25 CS1 by itself for six trials c 2 =.25 Then both CS present thereafter λ = 100 So, when one CS established first, it blocks the second from becoming established

Rescorla-Wagner 

Good mathematical model, accounting for      



Nonlinear learning curves with upper limit Salience/intensity of CS Extinction Contingency Overshadowing Blocking

But, doesn’t account for   

CS-US contiguity (time between CS and US) Latent Inhibition Spontaneous Recovery