Print Blackwell John
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Print Blackwell John as PDF for free.
More details
- Words: 1,774
- Pages: 20
Successfully Implementing Predictive Analytics in Direct Marketing John Blackwell
Copyright © 2007, SAS Institute Inc. All rights reserved.
A New York Times Science supplement study analyzed “feline high-rise syndrome” During the spring cats have a tendency to fall from high places. Analysis based on data collected from animal medical centers Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
1
From an analysis of the data they saw: −Below 7 stories the survival rate was 92% −Above 7 stories the rate jumped to 96% –Above 10 stories 100% survived.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Their conclusion: • Cats survival increases the higher the fall
Their explanation: • Until terminal velocity a cat extends it legs increasing possibility of injury. • At terminal velocity it can relax and stretch its legs and distribute impact. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
2
The Nature Conservancy works around the world to protect ecologically important lands and waters for nature and people.
Copyright © 2007, SAS Institute Inc. All rights reserved.
The Nature Conservancy TNC has over 3,000 employees across offices in 50 states and 30 countries. The Nature Conservancy has protected more than 117 million acres of land and 5,000 miles of river around the world. TNC relies on fundraising from its membership base. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
3
The Nature Conservancy Membership grew steadily throughout the 1990's with double digit growth in revenue. This rapid growth came to an abrupt halt in the early 2000's due to several factors. • Weak economy • Increased competition • Negative Press Copyright © 2007, SAS Institute Inc. All rights reserved.
The Challenge To maintain its reputation and continue its mission The Nature Conservancy needed to increase it revenues through increased efficiencies.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
4
The Challenge TNC were not effectively utilizing a potentially valuable asset • large database containing demographic and historical giving data.
Lacked the ability to turn the data into actionable insights.
Copyright © 2007, SAS Institute Inc. All rights reserved.
The Solution The creation of an in-house analytics team. Encourage organization wide adoption of analytics in determining strategy. Today all levels of the organization have information to make better business decisions. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
5
How TNC Raises Money Philanthropy – Donors who are capable of giving gifts of 100k or more. • Pool of about 30,000 potential givers − Personal contact
Membership – Solicits donors for smaller gifts • Audience of 2.5 million/ Average gift ~$60 − Direct mail − Telemarketing − Web/ Email − 30 million solicitations per year Copyright © 2007, SAS Institute Inc. All rights reserved.
Sample Decile Chart for Membership Appeal Model Size of Resp. Rev. Per Rank Group Rate Piece 1 2 3 4 5 6 7 8 9 10
96,715 96,740 96,294 96,778 97,917 95,817 104,412 89,794 95,245 97,424
6.9% 5.4% 4.4% 3.8% 3.6% 2.9% 2.4% 1.8% 1.5% 1.3%
$7.46 $2.67 $1.70 $1.24 $0.98 $0.75 $0.55 $0.41 $0.30 $0.20
Profit $668,961 $206,185 $111,441 $68,075 $42,595 $19,859 $594 ($11,857) ($22,643) ($32,891)
Cost Per $ Raised $0.07 $0.20 $0.32 $0.43 $0.55 $0.72 $0.99 $1.32 $1.79 $2.67
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
6
The Results Our approach has led to significant increases over the “business as usual” approach. Improvements in direct marketing campaigns have ranged from 5% to 75%
Select Model Business as Usual Model % Lift
Average Gift Resp. Rate Rev. Per Piece $57.52 4.7% $2.70 $47.61 3.3% $1.55 21% 42% 74%
Copyright © 2007, SAS Institute Inc. All rights reserved.
Success at The Nature Conservancy Today TNC raises over $400 million annually from its membership base. Forbes calculated fundraising efficiency at 88%. “Four-star rating” from Charity Navigator. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
7
How we got there? Making the case for the creation of the analytics team Choosing the right software Developing modeling algorithms Copyright © 2007, SAS Institute Inc. All rights reserved.
Making the case Bringing some outsourced dataprocessing tasks in-house lead to cost savings. Direct mail/ Telemarketing vendors often charge a lot for fairly routine tasks. This alone was about enough to cover the additional expense of the team. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
8
Choosing the software We needed a solution that could: • connect directly to the membership database • allow for easy data manipulation for file segmentation • provide excellent data-mining/ predictive modeling capability Copyright © 2007, SAS Institute Inc. All rights reserved.
The role of the analyst Some vendors implied that their software could build excellent models with the push of a button. Stories of Artificial Intelligence may imply computers are capable of doing almost anything. It was a decade ago that “Deep Blue” beat Kasparov in chess. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
9
The role of the analyst Advancements in software will help us to make better models Technology cannot replace the role of the analyst.
Copyright © 2007, SAS Institute Inc. All rights reserved.
The role of the analyst A Chinese game “Go” is one example where humans are still supreme.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
10
The role of the analyst In games there are defined rules and a defined playing surface Data mining is messier • New techniques are developed • Rules change • There is no grid Copyright © 2007, SAS Institute Inc. All rights reserved.
The role of the analyst Business Knowledge
Statistics
The overlap of these 3 skills is important IT/ Data Manipulation
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
11
Knowing your subject
Copyright © 2007, SAS Institute Inc. All rights reserved.
Knowing your subject Newspaper columnist George Will asserted: • “…[T]he states with the lowest per pupil spending… are among the states with the top SAT scores.” • “The public education lobby's crumbling last line of defense is the miseducation of the public.”
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
12
Knowing your subject
Copyright © 2007, SAS Institute Inc. All rights reserved.
From 1952 to 1976: • If the American League won the World Series, a Republican would win the presidential election. • If the National League won, a Democrat would win.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
13
From 1936 until 2004 the following held true: • A Washington Redskins win the week of the election meant a win for the incumbent party. • A Redskins loss would result in a victory for the challenging party.
Copyright © 2007, SAS Institute Inc. All rights reserved.
“As the old saw has it, garbage in, garbage out. The difficulty comes when you do not know what garbage looks like.”
*The Economist August 18, 2007 p.69
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
14
Knowing your subject CAS offer data appends and state: • “With over 500 demographics, psychographics, and lifestyle elements available for append, you can understand…”
How can you minimize the chance of selecting the wrong variables? Copyright © 2007, SAS Institute Inc. All rights reserved.
Knowing your subject It is not easy to tell which correlations are by chance Look at predictive variable across multiple data sets Create randomized input variables Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
15
Data Mining in Direct Marketing Finding patterns in large historical data sets that can be usefully applied to predict future customer behavior Observation Performance Present Date Date Future
Past
Observation Period
Performance Period
Development
Implementation
Copyright © 2007, SAS Institute Inc. All rights reserved.
Data Mining in Direct Marketing An example
Prev. Mail Date
Past
Observation Period
End of Current Responses Mail Date Future
Performance Period
Development
Implementation
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
16
Leaks from the future… Leaks are not always obvious. Certain data may be captured based on a triggering event. At TNC we capture appended phone numbers for donors at a certain giving level. Copyright © 2007, SAS Institute Inc. All rights reserved.
Leaks from the future… A recently published book tells readers to not “worry too much about how the data got there”. You should always know how the data “got there”. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
17
Understanding the data Effective analytics can not be achieved without an understanding of where the data comes from. In the “high-rise syndrome” study were the correct conclusions drawn?
Copyright © 2007, SAS Institute Inc. All rights reserved.
Understanding the data The data came from a clinic People will bring in hurt animals Will an owner always take a pet to a clinic who died? Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
18
Choosing the best model
Copyright © 2007, SAS Institute Inc. All rights reserved.
Choosing the best model
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
19
Choosing the best model The model that performs strongest on the validation data set is not always the best We have noticed that an ensemble model performs better in reality.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
20
Copyright © 2007, SAS Institute Inc. All rights reserved.
A New York Times Science supplement study analyzed “feline high-rise syndrome” During the spring cats have a tendency to fall from high places. Analysis based on data collected from animal medical centers Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
1
From an analysis of the data they saw: −Below 7 stories the survival rate was 92% −Above 7 stories the rate jumped to 96% –Above 10 stories 100% survived.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Their conclusion: • Cats survival increases the higher the fall
Their explanation: • Until terminal velocity a cat extends it legs increasing possibility of injury. • At terminal velocity it can relax and stretch its legs and distribute impact. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
2
The Nature Conservancy works around the world to protect ecologically important lands and waters for nature and people.
Copyright © 2007, SAS Institute Inc. All rights reserved.
The Nature Conservancy TNC has over 3,000 employees across offices in 50 states and 30 countries. The Nature Conservancy has protected more than 117 million acres of land and 5,000 miles of river around the world. TNC relies on fundraising from its membership base. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
3
The Nature Conservancy Membership grew steadily throughout the 1990's with double digit growth in revenue. This rapid growth came to an abrupt halt in the early 2000's due to several factors. • Weak economy • Increased competition • Negative Press Copyright © 2007, SAS Institute Inc. All rights reserved.
The Challenge To maintain its reputation and continue its mission The Nature Conservancy needed to increase it revenues through increased efficiencies.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
4
The Challenge TNC were not effectively utilizing a potentially valuable asset • large database containing demographic and historical giving data.
Lacked the ability to turn the data into actionable insights.
Copyright © 2007, SAS Institute Inc. All rights reserved.
The Solution The creation of an in-house analytics team. Encourage organization wide adoption of analytics in determining strategy. Today all levels of the organization have information to make better business decisions. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
5
How TNC Raises Money Philanthropy – Donors who are capable of giving gifts of 100k or more. • Pool of about 30,000 potential givers − Personal contact
Membership – Solicits donors for smaller gifts • Audience of 2.5 million/ Average gift ~$60 − Direct mail − Telemarketing − Web/ Email − 30 million solicitations per year Copyright © 2007, SAS Institute Inc. All rights reserved.
Sample Decile Chart for Membership Appeal Model Size of Resp. Rev. Per Rank Group Rate Piece 1 2 3 4 5 6 7 8 9 10
96,715 96,740 96,294 96,778 97,917 95,817 104,412 89,794 95,245 97,424
6.9% 5.4% 4.4% 3.8% 3.6% 2.9% 2.4% 1.8% 1.5% 1.3%
$7.46 $2.67 $1.70 $1.24 $0.98 $0.75 $0.55 $0.41 $0.30 $0.20
Profit $668,961 $206,185 $111,441 $68,075 $42,595 $19,859 $594 ($11,857) ($22,643) ($32,891)
Cost Per $ Raised $0.07 $0.20 $0.32 $0.43 $0.55 $0.72 $0.99 $1.32 $1.79 $2.67
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
6
The Results Our approach has led to significant increases over the “business as usual” approach. Improvements in direct marketing campaigns have ranged from 5% to 75%
Select Model Business as Usual Model % Lift
Average Gift Resp. Rate Rev. Per Piece $57.52 4.7% $2.70 $47.61 3.3% $1.55 21% 42% 74%
Copyright © 2007, SAS Institute Inc. All rights reserved.
Success at The Nature Conservancy Today TNC raises over $400 million annually from its membership base. Forbes calculated fundraising efficiency at 88%. “Four-star rating” from Charity Navigator. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
7
How we got there? Making the case for the creation of the analytics team Choosing the right software Developing modeling algorithms Copyright © 2007, SAS Institute Inc. All rights reserved.
Making the case Bringing some outsourced dataprocessing tasks in-house lead to cost savings. Direct mail/ Telemarketing vendors often charge a lot for fairly routine tasks. This alone was about enough to cover the additional expense of the team. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
8
Choosing the software We needed a solution that could: • connect directly to the membership database • allow for easy data manipulation for file segmentation • provide excellent data-mining/ predictive modeling capability Copyright © 2007, SAS Institute Inc. All rights reserved.
The role of the analyst Some vendors implied that their software could build excellent models with the push of a button. Stories of Artificial Intelligence may imply computers are capable of doing almost anything. It was a decade ago that “Deep Blue” beat Kasparov in chess. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
9
The role of the analyst Advancements in software will help us to make better models Technology cannot replace the role of the analyst.
Copyright © 2007, SAS Institute Inc. All rights reserved.
The role of the analyst A Chinese game “Go” is one example where humans are still supreme.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
10
The role of the analyst In games there are defined rules and a defined playing surface Data mining is messier • New techniques are developed • Rules change • There is no grid Copyright © 2007, SAS Institute Inc. All rights reserved.
The role of the analyst Business Knowledge
Statistics
The overlap of these 3 skills is important IT/ Data Manipulation
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
11
Knowing your subject
Copyright © 2007, SAS Institute Inc. All rights reserved.
Knowing your subject Newspaper columnist George Will asserted: • “…[T]he states with the lowest per pupil spending… are among the states with the top SAT scores.” • “The public education lobby's crumbling last line of defense is the miseducation of the public.”
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
12
Knowing your subject
Copyright © 2007, SAS Institute Inc. All rights reserved.
From 1952 to 1976: • If the American League won the World Series, a Republican would win the presidential election. • If the National League won, a Democrat would win.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
13
From 1936 until 2004 the following held true: • A Washington Redskins win the week of the election meant a win for the incumbent party. • A Redskins loss would result in a victory for the challenging party.
Copyright © 2007, SAS Institute Inc. All rights reserved.
“As the old saw has it, garbage in, garbage out. The difficulty comes when you do not know what garbage looks like.”
*The Economist August 18, 2007 p.69
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
14
Knowing your subject CAS offer data appends and state: • “With over 500 demographics, psychographics, and lifestyle elements available for append, you can understand…”
How can you minimize the chance of selecting the wrong variables? Copyright © 2007, SAS Institute Inc. All rights reserved.
Knowing your subject It is not easy to tell which correlations are by chance Look at predictive variable across multiple data sets Create randomized input variables Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
15
Data Mining in Direct Marketing Finding patterns in large historical data sets that can be usefully applied to predict future customer behavior Observation Performance Present Date Date Future
Past
Observation Period
Performance Period
Development
Implementation
Copyright © 2007, SAS Institute Inc. All rights reserved.
Data Mining in Direct Marketing An example
Prev. Mail Date
Past
Observation Period
End of Current Responses Mail Date Future
Performance Period
Development
Implementation
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
16
Leaks from the future… Leaks are not always obvious. Certain data may be captured based on a triggering event. At TNC we capture appended phone numbers for donors at a certain giving level. Copyright © 2007, SAS Institute Inc. All rights reserved.
Leaks from the future… A recently published book tells readers to not “worry too much about how the data got there”. You should always know how the data “got there”. Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
17
Understanding the data Effective analytics can not be achieved without an understanding of where the data comes from. In the “high-rise syndrome” study were the correct conclusions drawn?
Copyright © 2007, SAS Institute Inc. All rights reserved.
Understanding the data The data came from a clinic People will bring in hurt animals Will an owner always take a pet to a clinic who died? Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
18
Choosing the best model
Copyright © 2007, SAS Institute Inc. All rights reserved.
Choosing the best model
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
19
Choosing the best model The model that performs strongest on the validation data set is not always the best We have noticed that an ensemble model performs better in reality.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
Copyright © 2007, SAS Institute Inc. All rights reserved.
20
Related Documents
Print Blackwell John
November 2019 15
Elizabeth Blackwell
November 2019 37
Morton Blackwell Survey
November 2019 26
