CS 312: Algorithm Analysis
Announcements No homework due today Project #6: Linear Programming Due: today How’s H ’ it going? i ?
Project #7: TSP
Lecture #31: Backtracking Search: Graph Coloring and 8 Queens
Improvement Improvement Proposal ingredients: 1. 2. 3.
Specify a problem, selected from one of the projects Describe your proposed solution, an improvement to the algorithm used originally in the project Include analysis of the new, improved asymptotic bound Probably one page Due: Friday – submit online Feedback: by Monday at 5pm
Improvement Project
Once proposal is approved, implement Run experiments for many orders of magnitude of n (if necessary) Hopefully results show an improvement in the empirical performance Write up the outcome: Probably one or two pages Submit online That’s it!
Graph Coloring Problem Assign colors to the vertices of a graph so that no adjacent vertices share the same color Vertices i and j are adjacent if there is an edge from vertex i to vertex j.
Find all m-colorings of a graph Find all ways to color a graph with at most m colors.
Guidelines & Framework: available soon Help session: Thursday Ensuing lectures will provide insight
Objectives Formulate algorithmic solutions to intractable problems as state-space search Related to graph search
Use backtracking search (in a state-space) to find solutions to the graph (& map) coloring problems Gain insight into carefully describing and expanding states for more efficient backtracking search Formulate the n-Queens problem as a permutation selection problem
Recognize the need for feasibility functions in backtracking search
State-Space Search What to store with each state (node)? How to travel between states (nodes)? i.e., how to expand a state into successor states
What to analyze about the state-space? Is it a graph with shared sub-structure? Is it simply a tree?
When to backtrack? i.e., how to search?
1
Example: Graph Coloring State:
m=3 Red Yellow Blue
Example: Graph Coloring
Transition:
• Enumerate all possible states; check for solutions at the leaves. • Naively, non-solutions (not shown here) will be generated. • More on that later.
Aside: Coloring a Map Assign colors to countries so that no two bordering regions are the same color. Map coloring is planar graph coloring. Idaho Wyoming
Nevada
Utah Arizona
Colorado
Four Color Theorem How many colors do you need for a planar map? Four.
Haken and Appel using a computer program and 1,200 hours of run time in 1976. Checked 1,476 graphs (i.e., cases in the proof). First proposed in 1852.
New Mexico
Simple Recursive Backtracking Algorithm procedure explore(int k, state s) Input: Depth in the search tree k; current state s Output: none; prints solutions
for each s’ ∈ extensions_of(s) if k==n and criterion(s’) then output “Solution:” s’ // if possible solution not yet fully specified if (k < n) then explore(k+1, s’)
Segue … Let’s take a step back from back-tracking for a moment. Consider how large the search space could be if we don’t don t approach the formulation of a problem carefully.
Outer-most call: explore(0, init_state())
2
n-Queens Problem Given: n-queens and an nxn chess board Find: A way to place all n queens on the board s.t. no queens are attacking another queen. queen
First Solution Idea Consider every placement of each queen 1 at a time. How many placements are there?
• How could you formulate an algorithmic solution to this problem? • How would you represent a state?
First Solution Idea Consider every placement of each queen 1 at a time.
Second Solution Idea Don’t place 2 queens in the same row. Now how many positions must be checked?
How many placements are there? ⎛ n 2 ⎞ ⎛ 64 ⎞ ⎜ ⎟ ⇒ ⎜⎜ ⎟⎟ = 4,426,165,368 ⎜n⎟ ⎝ ⎠ ⎝8⎠
⎛n⎞ n! ⎜ ⎟= k !( k n − k )! ⎝ ⎠
Recall:
The Eight Queens Problem
The Eight Queens Problem
1
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8 1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
3
The Eight Queens Problem
Second Solution Idea Don’t place 2 queens in the same row.
1 2
Now how many positions must be checked?
3
Represent a positioning as a vector [x1, …, x8] Where each element is an integer 1, …, 8.
4
n n = 88 = 16, 777, 216
5 6 7 8 1
2
3
4
5
6
7
8
Third Solution Idea Don’t place 2 queens in the same row or in the same column. Generate all permutations of (1,2…8) Now N h how many positions iti mustt b be checked? h k d?
1 2 3 4 5 6 7 8 1
2
3
4
5
6
7
8
7
8
(1,2,3,4,5,6,7,8)
1
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8 1
2
3
4
5
6
(1,2,3,4,5,6,8,7)
7
8
1
2
3
4
5
6
(1,2,3,4,5,8,6,7)
4
Third Solution Idea
1 2
Don’t place 2 queens in the same row or in the same column. Generate all permutations of (1,2…8)
3 4
Now how many positions must be checked?
5
n ! = 8! = 40,320 40 320
6
We went from C(n2, n) to nn to n!
7
And we’re happy about it!
8 1
2
3
4
5
6
7
8
We applied explicit constraints in the enumeration of successor states to shrink our search space.
(1,2,3,4,5,8,7,6)
Permutation Generation It is easy to enumerate all permutations Given a permutation you know the “next” one to check
Lessons So far, efficiency of Backtracking Algorithms in state-space search depends on: 1 The time to generate the next possible 1. states in the search tree 2. The number of next possible states satisfying the explicit constraints
Toward More Lessons
Four Queens Problem
We’ve only been checking a solution after placing all 8 queens. A criterion function (or solution test) A binary test
A better idea is to check after each placement of a queen. A partial-criterion function (or feasibility test) A binary test Next time, we will replace with “bounding function”, a numerical goodness measure
1
2
3
4
Easier to manage in a lecture.
Idea: Determine we are on a dead-end road without exploring the entire road
5
Four Queens Problem
Four Queens Problem
x1 = 1
x1 = 1 x2 = 2
x2 = 2 x3 = 3
x3 = 3
x4 = 4 1
2
3
4
x4 = 4 1
2
3
4
Each branch of the tree represents a decision to place a queen. The criterion function can only be applied to leaf states.
Is this leaf state a solution?
Four Queens Problem
Four Queens Problem
x1 = 1 x2 = 2 x3 = 3
x3 = 4
x4 = 4 1
2
3
x4 = 3
4
Is this leaf state a solution? Using the criterion function, this is the search tree.
Four Queens Problem
Four Queens Problem
x1 = 1
x1 = 1
x2 = 2 x3 = 3 x4 = 4 1
2
3
x2 = 2 x3 = 3
x3 = 4 x4 = 3
4
A better question: Is any child of this state ever going to be a solution?
x4 = 4 1
2
3
x3 = 4 x4 = 3
4
A better question: Is any child of this state ever going to be a solution? No. 1 is attacking 2. Adding queens won’t fix the problem.
6
Four Queens Problem
Four Queens Problem
x1 = 1
x1 = 1
x2 = 2
1
2
3
x2 = 2
4
1
The partial criterion or feasibility function checks that a state may eventually have a solution. Will this state have a solution? no.
2
3
4
Will this state have a solution? Maybe.
Four Queens Problem
Four Queens Problem
x1 = 1 x2 = 2
x1 = 1 x2 = 2
x2 = 3
x3 = 2
x3 = 2 1
2
3
x2 = 3
4
1
2
3
x2 = 3 x3 = 4
4
Will this state have a solution? No. Etc.
Will this state have a solution? No. Etc.
Four Queens Problem
Four Queens Problem
x1 = 1 x2 = 2 x2 = 3 x3 = 2
1
2
3
x2 = 4 x3 = 4
4
And so on … Using the feasibility function, this is the search tree.
7
Efficiency of Backtracking Algorithms depends on:
Recursive Backtracking Algorithm (updated with feasibility check) procedure explore(int k, state s) Input: Depth in the search tree k; current state s Output: Solution or {} (failure)
for each s’ ∈ extensions_of(s) if partial_criterion(s’) partial criterion(s’) then if k==n and criterion(s’) then output “Solution:” s’ // if state not yet fully specified if (k < n) then explore(k+1, s’)
1. The time to generate the next possible states in the search tree 2. The number of next possible states satisfying the explicit constraints 3. The time to compute the partial criterion (feasibility) 4 The proportion of next possible states satisfying the 4. partial criterion function
A feasibility function is considered good, if it substantially reduces the number of states that are generated.
But, feasibility functions take more time to evaluate.
Outer-most call: explore(0, init_state())
Conclusions Can formulate just about any problem as state-space search. What about MST? Sure. Hopefully H f ll as you considered id d th the statet t space, you’d get smart and settle in on something that looks greedy. Other state-spaces might look a lot like D&C or DP. Not so for the toughest problems
Assignment Homework #22 – linked to schedule State-space search! due Monday
Read R dS Sec. 9 9.1.2 12
8