Cognitive Capability
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Cognitive Capability as PDF for free.
More details
- Words: 901
- Pages: 31
Episodic Memory and Cognitive Capabilities
Andrew Nuxoll 24 May 2007
Outline
Review and Introduction
Definitions Research Goals TankSoar Domain
Demonstrating Cognitive Capabilities
Action Modeling Virtual Sensors Learning from Past Success and Failure
2
Long Term Memory Memory
Long Term Memory
Declarative Memory
Semantic Memory
Short Term Memory
Procedural Memory
Episodic Memory
Episodic Memory
Memories of specific events in your past 3
Current Implementation Encoding Initiation?
Long-term Procedural Memory Production Rules
Storage Retrieval Output
Input
Working Memory
Cue
Retrieved
When the agent takes an action. 4
Current Implementation Encoding Initiation Content? Storage
Long-term Procedural Memory Production Rules
Retrieval Output
Input
Working Memory
Cue
Retrieved
A portion of working memory is stored in the episode 5
Current Implementation Encoding Initiation Content Storage Episode Structure? Retrieval
Long-term Procedural Memory Production Rules
Output
Working Memory
Episodic Memory
Cue
Episodic Learning Input
Retrieved
Episodes are stored in a separate memory 6
Current Implementation Encoding Initiation Content Storage Episode Structure Retrieval Initiation/Cue?
Long-term Procedural Memory Production Rules
Output
Working Memory
Episodic Memory
Cue
Episodic Learning Input
Retrieved
Cue is placed in an architecture specific buffer. 7
Current Implementation Encoding Initiation Content Storage Episode Structure Retrieval Initiation/Cue Retrieval
Long-term Procedural Memory Production Rules
Output
Working Memory
Episodic Memory
Cue
Episodic Learning Input
Retrieved
The closest partial match is retrieved. 8
Research Goals
Explore the cognitive capabilities granted to an agent with an episodic memory Explore what’s necessary to build an effective episodic memory for a general cognitive architecture
Domain independence Performance
Take inspiration from cognitive psychology
9
Cognitive Capabilities: How do we use Episodic Memory?
Sensing
Detecting Repetition Virtual Sensing Noticing Unusual Input Sense of Identity
Reasoning
Action Modeling Recording Previous Successes/Failures Modeling the Environment Managing Long Term Goals
Learning
Retroactive Learning Reanalyzing with new Knowledge Explaining Behavior “Boosting” other Learning Mechanisms
10
TankSoar Domain
Tanks in a maze Sub-goals
Shoot other tanks Don’t get shot Don’t run out of
Energy Missiles
Multiple sensors and actions 11
Action Modeling
Cognitive Capability: Action Modeling
Action Modeling
Definition: Learning the immediate effect of an action Analog: Case-Based Reasoning
Task: Conserve Energy
Selecting proper radar setting to minimize energy consumption
13
Agent Implementation Radar Setting: 1
Evaluation Result: Radar Setting: 5
Radar Setting: 2 X=11 Y=3 Dir=south
… Radar Setting: max
Agent’s State
Agent Confirms Location and Direction Match
te y a re r C mo e e M Cu
Episodic Retrieval X=11 Y=3 Dir=south
X=11 Y=3 Dir=south
14
Initial Performance Radar Tank Performance 1 0.9 Fraction Correct
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Random
0 1
101
201
301
401
501
601
701
801
901
Radar Settings (time)
15
Nuggets
Episodic memory is an effective medium for action modeling
Coal
Requires agent have knowledge of whether a retrieved episode is useful
16
Virtual Sensors
Cognitive Capability: Virtual Sensors
Virtual Sensors
Definition: Retrieving past sensing that is relevant to the current task
?
Task: Locate the Battery
Using episodic memories to construct a path
18
Agent Implementation Move North
Evaluation Result: Move North
Move West Move East Agent’s State
te y a e r Cr mo e e M Cu
Episodic Retrieval
19
Performance Energy Search Time
Number of Moves
1000
100 Random Ep. Mem. 10
1 1
2
3
4
5
6
7
8
9
10
11 12
Searches
20
Paths to Battery
21
Nuggets
Episodic memory can be used as a virtual sensor
Coal
Limited investigation of integration of episodic and semantic memory
22
Learning from Past Success and Failure
Cognitive Capability: Learning from Past Success and Failure
Learning from Past Success and Failure
Definition: Using past performance to guide future behavior More emphasis on long term
Task: Combat
Using episodic memory to determine best tactics in the “attack” subgoal 24
Agent Implementation Move Forward
Evaluation Result: Move Forward + Fire = 2 points
Move Forward + Fire Missile
… Turn Right Agent’s State
Compare Memories In the Sequence
te ea Cue r C ry mo e M
Episodic Retrieval
Retrieve the Next Memory in Sequence
…
Retrieve the Next Memory in Sequence
25
Performance 20 10
Margin of Victory
0 1
11 21 31 41 51
61 71 81 91 101 111 121 131 141 151 161
-10 -20 -30 -40 -50 -60 Successive Games
26
Tactics Learned
Fight or flight Back away and shoot Dodging
27
Without Heuristic Cue Performance without Heuristic Cue Selection 20
Average Margin of Victory
10 0 -10
1
11 21 31 41 51 61 71
81 91 101 111 121 131 141 151 161
-20 -30 -40 -50 -60 Successive Games
28
Without Discount Factor No Discount Factor 20
Average Margin of Victory
10 0 -10
1
12 23 34 45
56 67 78
89 100 111 122 133 144 155 166 177
-20 -30 -40 -50 -60 Successive Games
29
Nuggets
Episodic memory can be effective at learning long term tactics
Coal
Requires the use of a discount factor Requires heuristic cue selection
30
Summary
Episodic memory enables multiple cognitive capabilities including:
Action modeling Virtual sensing Learning from past success and failure
31
Andrew Nuxoll 24 May 2007
Outline
Review and Introduction
Definitions Research Goals TankSoar Domain
Demonstrating Cognitive Capabilities
Action Modeling Virtual Sensors Learning from Past Success and Failure
2
Long Term Memory Memory
Long Term Memory
Declarative Memory
Semantic Memory
Short Term Memory
Procedural Memory
Episodic Memory
Episodic Memory
Memories of specific events in your past 3
Current Implementation Encoding Initiation?
Long-term Procedural Memory Production Rules
Storage Retrieval Output
Input
Working Memory
Cue
Retrieved
When the agent takes an action. 4
Current Implementation Encoding Initiation Content? Storage
Long-term Procedural Memory Production Rules
Retrieval Output
Input
Working Memory
Cue
Retrieved
A portion of working memory is stored in the episode 5
Current Implementation Encoding Initiation Content Storage Episode Structure? Retrieval
Long-term Procedural Memory Production Rules
Output
Working Memory
Episodic Memory
Cue
Episodic Learning Input
Retrieved
Episodes are stored in a separate memory 6
Current Implementation Encoding Initiation Content Storage Episode Structure Retrieval Initiation/Cue?
Long-term Procedural Memory Production Rules
Output
Working Memory
Episodic Memory
Cue
Episodic Learning Input
Retrieved
Cue is placed in an architecture specific buffer. 7
Current Implementation Encoding Initiation Content Storage Episode Structure Retrieval Initiation/Cue Retrieval
Long-term Procedural Memory Production Rules
Output
Working Memory
Episodic Memory
Cue
Episodic Learning Input
Retrieved
The closest partial match is retrieved. 8
Research Goals
Explore the cognitive capabilities granted to an agent with an episodic memory Explore what’s necessary to build an effective episodic memory for a general cognitive architecture
Domain independence Performance
Take inspiration from cognitive psychology
9
Cognitive Capabilities: How do we use Episodic Memory?
Sensing
Detecting Repetition Virtual Sensing Noticing Unusual Input Sense of Identity
Reasoning
Action Modeling Recording Previous Successes/Failures Modeling the Environment Managing Long Term Goals
Learning
Retroactive Learning Reanalyzing with new Knowledge Explaining Behavior “Boosting” other Learning Mechanisms
10
TankSoar Domain
Tanks in a maze Sub-goals
Shoot other tanks Don’t get shot Don’t run out of
Energy Missiles
Multiple sensors and actions 11
Action Modeling
Cognitive Capability: Action Modeling
Action Modeling
Definition: Learning the immediate effect of an action Analog: Case-Based Reasoning
Task: Conserve Energy
Selecting proper radar setting to minimize energy consumption
13
Agent Implementation Radar Setting: 1
Evaluation Result: Radar Setting: 5
Radar Setting: 2 X=11 Y=3 Dir=south
… Radar Setting: max
Agent’s State
Agent Confirms Location and Direction Match
te y a re r C mo e e M Cu
Episodic Retrieval X=11 Y=3 Dir=south
X=11 Y=3 Dir=south
14
Initial Performance Radar Tank Performance 1 0.9 Fraction Correct
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Random
0 1
101
201
301
401
501
601
701
801
901
Radar Settings (time)
15
Nuggets
Episodic memory is an effective medium for action modeling
Coal
Requires agent have knowledge of whether a retrieved episode is useful
16
Virtual Sensors
Cognitive Capability: Virtual Sensors
Virtual Sensors
Definition: Retrieving past sensing that is relevant to the current task
?
Task: Locate the Battery
Using episodic memories to construct a path
18
Agent Implementation Move North
Evaluation Result: Move North
Move West Move East Agent’s State
te y a e r Cr mo e e M Cu
Episodic Retrieval
19
Performance Energy Search Time
Number of Moves
1000
100 Random Ep. Mem. 10
1 1
2
3
4
5
6
7
8
9
10
11 12
Searches
20
Paths to Battery
21
Nuggets
Episodic memory can be used as a virtual sensor
Coal
Limited investigation of integration of episodic and semantic memory
22
Learning from Past Success and Failure
Cognitive Capability: Learning from Past Success and Failure
Learning from Past Success and Failure
Definition: Using past performance to guide future behavior More emphasis on long term
Task: Combat
Using episodic memory to determine best tactics in the “attack” subgoal 24
Agent Implementation Move Forward
Evaluation Result: Move Forward + Fire = 2 points
Move Forward + Fire Missile
… Turn Right Agent’s State
Compare Memories In the Sequence
te ea Cue r C ry mo e M
Episodic Retrieval
Retrieve the Next Memory in Sequence
…
Retrieve the Next Memory in Sequence
25
Performance 20 10
Margin of Victory
0 1
11 21 31 41 51
61 71 81 91 101 111 121 131 141 151 161
-10 -20 -30 -40 -50 -60 Successive Games
26
Tactics Learned
Fight or flight Back away and shoot Dodging
27
Without Heuristic Cue Performance without Heuristic Cue Selection 20
Average Margin of Victory
10 0 -10
1
11 21 31 41 51 61 71
81 91 101 111 121 131 141 151 161
-20 -30 -40 -50 -60 Successive Games
28
Without Discount Factor No Discount Factor 20
Average Margin of Victory
10 0 -10
1
12 23 34 45
56 67 78
89 100 111 122 133 144 155 166 177
-20 -30 -40 -50 -60 Successive Games
29
Nuggets
Episodic memory can be effective at learning long term tactics
Coal
Requires the use of a discount factor Requires heuristic cue selection
30
Summary
Episodic memory enables multiple cognitive capabilities including:
Action modeling Virtual sensing Learning from past success and failure
31
Related Documents
Cognitive Capability
December 2019 18
Cognitive
November 2019 38
Cognitive
December 2019 36
Nem Capability
May 2020 32
Process Capability
June 2020 5