Cio Executive Board - Overcoming The Insight Deficit.pdf
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Cio Executive Board - Overcoming The Insight Deficit.pdf as PDF for free.
More details
- Words: 33,419
- Pages: 144
Information Technology Practice
Cio Executive Board
Overcoming the Insight Deficit Big Judgment in an Era of Big Data
© 2011 The Corporate Executive Board Company. All Rights Reserved.
Information Technology Practice
Cio Executive Board Executive Directors Shvetank Shah Warren Thune
www.executiveboard.com
Managing Directors Jaime Capellá Brian Foster Andrew Horne David Kingston Jay Shankavaram Senior Directors Vimarsh Bakaya Miles Gibson Bill Lee Chris Mixter Rumki Saha Alex Stille KD Weitzel Project Manager Matthew Charlet Shalini Das
Directors Aron Kuehnemann Tim Macintyre Dorota Pietruszewska Kristin Sherwood Associate Directors Brent Cassell Christiane Groth Karolina Laskowska
Production Designer Carolyn Lamond Contributing Designer Supriya Dhasmana Editor Nidhi Vikram Choudhury
Consultants Kelly Chambers Marina Murray Senior Analyst Pallavi Goel Shilpa Pental Analyst Raf Gelders
Copies and Copyright
Legal Caveat
As always, members are welcome to an unlimited number of copies of the materials contained within this handout. Furthermore, members may copy any graphic herein for their own internal purpose. The Corporate Executive Board Company requests only that members retain the copyright mark on all pages produced. Please contact your Member Support Center at +1-866-913-8101 for any help we may provide.
The Corporate Executive Board Company has worked to ensure the accuracy of the information it provides to its members. This report relies upon data obtained from many sources, however, and The Corporate Executive Board Company cannot guarantee the accuracy of the information or its analysis in all cases. Furthermore, The Corporate Executive Board Company is not engaged in rendering legal, accounting, or other professional services. Its reports should not be construed as professional advice on any particular set of facts or circumstances. Members requiring such services are advised to consult an appropriate professional. Neither The Corporate Executive Board Company nor its programs are responsible for any claims or losses that may arise from a) any errors or omissions in their reports, whether caused by The Corporate Executive Board Company or its sources, or b) reliance upon any recommendation made by The Corporate Executive Board Company.
The pages herein are the property of The Corporate Executive Board Company. Beyond the membership, no copyrighted materials of The Corporate Executive Board Company may be reproduced without prior approval.
CIO0361911SYN
Content Publishing Solutions
TABLE OF CONTENTS Executive Summary • iv An Information Management Strategy on a Page • vi Assess Insight IQ in Your Organization • vii Occasion for the Research • 1 Enable Information Usability • 25 Capability-Driven Tool Portfolio (Blue Cross Blue Shield of North Carolina) • 33 Flexible and Transparent Analytics (GlaxoSmithKline) • 47 CXO KPI Identification (PepsiCo) • 61 Support Big Judgment, Not Just Big Data • 77 Information Training Portfolio (Tiffany & Co.) • 87 Observe and Question to Identify Opportunities • 101 “Prosumer” Sounding Board (Ford) • 105 Anthropologically Driven Needs Identification (Alpha Company1) • 109 Appendix • 116
1
Pseudonym.
CIO0361911SYN
iv
Executive Summary Our ability to store and analyze information has grown exponentially across the past decade, but the buzz about “big data” misses the reality that as data volumes explode, it becomes harder, not easier, for knowledge workers to make sense of it. Less than 40% of knowledge workers have the skills and judgment to exploit information for competitive advantage. Paradoxically, as organizations amass more data and spend more on analytics, they face a growing insight deficit. The insight deficit is important to IT, as information management will likely be IT’s main contribution to enabling growth. In business areas that drive growth—innovation, marketing, sales, and customer service—up to 80% of IT enablement opportunities relate to business intelligence, collaboration, or the customer interface. Yet, the insight deficit means that efforts to improve information management may lead to worse, not better, decision making. Insight IQ: A Unique View of How Organizations Exploit Information for Decision Making To give IT leaders practical guidance on overcoming the insight deficit, we built an Insight IQ index to measure individuals’ ability to find and analyze information to make better decisions. Insight IQ has three elements: information attainability, information usefulness, and knowledge worker capability. Using the model, we surveyed almost 5,000 knowledge workers at hundreds of organizations globally. We used the survey to quantify the value of overcoming the insight deficit and pinpoint the actions needed to realize this value. (See page vii to learn how your organization can take the Insight IQ diagnostic). The benefits of overcoming the deficit are significant. Functions with the highest Insight IQ perform, on average, 24% better than their peers across a wide range of metrics. All three components of Insight IQ are vital to achieve this goal—if any are missing, the chance of failure is 65% or more.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Core information governance activities such as information architecture, business stewardship of data, and an analytics center of excellence are important drivers of Insight IQ. Most organizations know this and already have initiatives in place. What is more surprising is the impact of three capabilities directly related to the knowledge worker: information usability, knowledge worker skills, and the identification of emerging knowledge worker information needs. These three capabilities are often overlooked, yet all directly impact Insight IQ. 1. Enable Information Usability Only 50% of knowledge workers find information from corporate sources to be in a usable format. The problem will only get worse as the number of information sources, uses, and users continue to increase. In this diverse environment, trying to improve usability by offering self-service usually disappoints as few self-service capabilities are flexible enough to meet knowledge worker needs. To overcome this challenge, the best companies offer a choice of analytic tools and deploy information filtering and visualization capabilities. They develop a deeper understanding of how, when, and why information will be used by specific user segments, and they vary quality standards accordingly. Offer Greater Tools Choice—Greater choice in analytic tools drives Insight IQ. The greatest increase comes from moving from one standard tool for the enterprise to tools aligned to specific business units or functions. Rationalization programs for analytic tools often over-standardize, leaving employees without necessary capabilities and opening the door to unofficial tools. In response, progressive organizations, such as Blue Cross Blue Shield of North Carolina offer knowledge workers a portfolio of tools aligned to a list of analytic capabilities. Visualize and Filter—Many knowledge workers complain about the time required to aggregate information from supposedly user-ready reports. Consequently, leading organizations invest in capabilities such as information visualization, aggregation, and filtering before they invest in more sophisticated analytics.
Executive Summary (Continued) Make Information Quality Transparent and Targeted—Only one-third of knowledge workers trust information from other functions in the organization. To overcome this distrust, progressive organizations such as GlaxoSmithKline clarify the information’s source, how it has been manipulated, its current quality level, and caveats about how it can be used. Others, such as PepsiCo, recognize that certain groups, such as C-level executives, have distinct patterns of information usage that need to be targeted for different-in-kind support.
Provide Tools for Collaborative Decision Making—Making biases explicit and listening to alternative points of view are characteristics of informed skepticism. To support these behaviors, IT can provide workflow tools that connect stakeholders involved in decision making, capture “wisdom of the crowds” input, and add contextual data. In addition, business leaders can help promote informed skepticism by introducing formal decision-making models and by embedding the right behaviors in performance expectations.
2. Support Big Judgment, Not Just Big Data
3. Observe and Question to Identify Opportunities
Exploiting information requires broader and deeper analytic skills across the organization. Most employees are now knowledge workers and spend on average 36% of their time collecting and analyzing information. However, only 38% of employees are “informed skeptics” who combine analytic skills with judgment to make sound decisions. These critical thinkers are comfortable with analytics but maintain a healthy skepticism about its limitations.
Most IT organizations are familiar with process automation projects where business needs are known and stable. In contrast, information needs are context dependent, dynamic, and may be unarticulated or even unknown. Solving this challenge requires anthropological skills that are in short supply in IT.
Develop an Analytics Training Curriculum—Two-thirds of knowledge workers have access to analytics training, but only 25% find the training effective. Training that focuses on the tools but does not address information sources or analytical techniques is ineffective. Leading organizations, such as Tiffany & Co., adopt a multipronged approach that supports knowledge workers across the information lifecycle and focuses on conducting analysis. This type of training has no natural organizational home, so IT and business leaders should collaborate to determine ownership. Hire Analysts Who Can Coach—Knowledge workers should be supported by business intelligence experts or analysts who act as force multipliers. While coaching skills are what make these analysts most effective at driving Insight IQ, organizations typically look for quantitative and business skills when hiring analysts. Leading organizations have changed their recruiting approaches to actively test for coaching skills.
Identify and Learn from Early Adopters—Traditional requirements gathering fails when assessing information needs as the needs are fast changing, diverse, and difficult to articulate. Instead, progressive organizations, such as Ford, surface information needs by segmenting knowledge workers by their information usage patterns and their stance toward technology adoption. They also observe early adopters as an early-warning mechanism for emerging needs. Observe and Question—Knowledge workers often struggle to articulate their information needs, and these needs differ across individuals, organizational cultures, and collaborative practices. To proactively identify pain points in end-user workflows, leading IT groups adopt an anthropological approach to identify opportunity, observe knowledge workers in diverse user settings and conduct open-ended interviews.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
v
vi
An Information Management Strategy on a Page Statement of Strategy Shift the focus of IT investment and skills toward information management with the goal of providing knowledge workers with attainable and useful information and boosting their capability to exploit that information for competitive advantage.
Top Initiatives 1. Launch an information management and analytics center of excellence. 2. Redesign IT's opportunity identification process to make it proactive and informed by observation of distinct knowledge worker segments. 3. Identify analytic capabilities used by knowledge workers, and offer a portfolio of tools to meet those needs. 4. Develop and hire usability and interface design skills in IT. 5. Coach knowledge workers to boost their analytic capability and foster informed skepticism. 6. Harmonize and integrate a small subset of information subjects where there is greatest enterprise need.
Key Information Management Metrics
Current (2011)
Percentage of IT budget devoted to information and analytics projects
23%
40–50%
Percentage of budget spent on knowledge worker capability
< 5%
> 10%
> 80%
< 40%
1–3
8–12
38%
59%
51
65
Percentage of information subjects targeted for harmonization and integration Target number of analytic tools Percentage of knowledge workers who are informed skeptics Average Insight IQ score
Underlying Assumptions and Beliefs 1. The number of opportunities to drive growth through information management will equal or outstrip the opportunities for process automation. 2. Many of our knowledge workers lack the skills and judgment to use information effectively for decision making. 3. Not all information needs to be harmonized or integrated at enterprise level. Similarly, some information needs higher levels of quality than others. 4. Our business partners will take the lead in information stewardship. 5. Knowledge worker reliance on external information sources and on unstructured information will continue to rise. Obtain a customizable template for a strategy on a page at www.cio.executiveboard.com. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Goal (2014)
Use the Insight IQ Diagnostic to benchmark the maturity of your information management and prioritize areas for investment.
Assess Insight IQ in Your Organization Overview of the Insight IQ Diagnostic
■■
■■
■■
■■
Diagnostic Benefits
The diagnostic measures Insight IQ using online surveys for knowledge workers and an IT leader. Organizations identify at least 30 knowledge workers per business function, and at least three business functions participate in the diagnostic.
1. Assess Readiness—Identify which business functions and teams in your organization are ready for additional investment in information management and analytics and be forewarned about areas where adoption may be low.
2. Set Priorities—Prioritize the technology investments, governance processes, and skills that have the greatest impact on Insight IQ.
Each survey takes approximately 20 to 25 minutes to complete. The diagnostic is included in your existing CIO Executive Board membership.
3. Communicate—Demonstrate the business leadership, communication, and change management efforts necessary to close your organization’s insight deficit. 4. Benchmark—Compare your organization’s Insight IQ against industry peers.
Key Deliverables Custom Diagnostic Report and Improvement Recommendations
Detailed results benchmarked against 5,000 knowledge workers worldwide
Guidance on practical improvement steps, including tactics
Executive Advisor® walkthrough of results and key recommendations
For more information, contact your account manager or e-mail [email protected]. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
vii
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
viii
Occasion for the Research
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
1
“Big data” is quickly becoming a reality as information volumes grow by 60% annually, and 36% of all work time is devoted to information collection and analysis.
■■
■■
MORE INFORMATION, MORE INFORMATION WORK Estimated Rise in Global Data Volumes, 2010–2015 Indexed to 100 1,050 1050
A 60% annual rise in information volume means that average storage per end user will increase from 2 GB to 20 GB in five years. Knowledge worker time spent collecting and analyzing information is similar across all functions, for example: –– Finance, 40% –– General Management, 35%
Overcoming the Insight Deficit
60%
R
CAG
660 410
100 2010
160
2011
260
2012
2013
2014
Time Spent Collecting and Analyzing Information
2015
Drivers of Democratized Decision Making
Percentage of Total Knowledge Worker Work Time
–– HR, 33%
1
Decisions are made closer to the market (e.g., product design, channel mix).
2
Decisions are more dynamic and varied (e.g., demand forecasts, discounts).
3
Knowledge workers have access to more information and better tools (e.g., customer segmentation and value analysis).
–– Marketing, 36% –– Production and Operations, 33% –– Sales, 27% ■■
Knowledge workers dedicate more time to collecting and analyzing information as decision making is democratized.
64% All Other Work
36% Collecting and Analyzing Information
n = 4,941 knowledge workers. Source: “All Too Much” The Economist, 27 February 2010; Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
2
To drive growth, IT must support less familiar areas of the business where the majority of opportunities are in information management, not process automation.
FROM PROCESS TO INFORMATION TO DRIVE GROWTH Breakdown of IT Enablement Opportunities by Business Process Percentage of Total Opportunities None
9% 9%
15%
■■
Information Management (Collaboration, Business Intelligence, Customer Interface)
18%
27%
28%
IT enablement opportunities are divided between:
Process Automation
–– Information Management—Includes collaboration, business intelligence, and the use of technology at the customer interface, and
30% 63%
28%
63% 60%
–– Process Automation— Processes enabled by enterprise systems such as ERP, PLM, HRIS, and CRM.
54%
46%
28% 18%
Finance and HR
“Information management means we encounter functions with little prior experience working with IT. But to contribute to growth, we have to understand these areas.” CIO Consumer Goods Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
■■
■■
Production and Supply Chain
Extensive experience working with IT Primarily drive efficiency and quality
Customer Service ■■ ■■
Marketing and Sales
13% Product/Service Innovation …but increased support for growth moves IT into much less familiar areas.
Little experience working with IT Primarily drive growth
Continued focus on process automation, will mean IT misses opportunities to drive growth… Note: Totals may not equal 100% due to rounding. Source: “Process Classification FrameworkSM,” APQC, September 2009, http://www.apqc.org/knowledge-base/documents/apqc-process-classification-framework-pcf-cross-industryexcel-version-51-0.
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
3
As 62% of knowledge workers lack the ability to apply “big judgment” to information they use for decision making, increased information availability may do more harm than good.
Overcoming the Insight Deficit
Without Big Judgment, Big Data Magnifies Risk not Opportunity Have Processes and Skills to Use Information Effectively for Decision Making Percentage of Knowledge Workers
Risks from Big Data When Big Judgment Is Lacking
38% Yes
62% No
■■
Exaggerated or overlooked opportunities
■■
Underestimated risks
■■
Disputed decisions
■■
Misleading performance measures
■■
Wasted knowledge worker time
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
“The biggest talent challenge we face is finding people with the skills to create insight from data.” VP, HR Global Food Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
4
Effective information management reframed around the knowledge worker drives improved decision making.
■■
■■
MEASURING THE INSIGHT DEFICIT The Three Components of Insight IQ
Insight IQ is a quantifiable outcome that can be measured at the level of the individual knowledge worker. Linking information management initiatives with higher Insight IQ will refocus information management on the important outcome.
Insight IQ entails the ability to find and analyze relevant information to drive actions and decisions effectively.
The right information is available and easy to find.
Information Attainability
Information Usefulness
Information is of a known quality and in a usable format.
Insight IQ
Knowledge Worker Capability
Knowledge workers have the analytic ability and predisposition to generate insight from information.
Source: Corporate Executive Board, IT Practice Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
5
The CIO Executive Board would like to extend its sincerest appreciation to the following organizations for their participation in this research.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
With Sincere Appreciation Partial List of Participating Organizations
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
6
The CIO Executive Board’s database spans a variety of organization sizes and industries and includes responses from almost 5,000 knowledge workers globally.
■■
■■
Survey administration involved in-depth collaboration with 22 member organizations. Additional surveying of knowledge workers at more than 1,000 organizations worldwide was conducted to supplement the dataset.
The Only Survey of Its Kind Respondents by Industry
Respondents by Function
Sample Distribution
Sample Distribution
2% Energy and Utilities 3% Telecommunication 4% Other 4% Aerospace/Defense, Automotive, and Transport 4% Government 5% Professional Services 5% Manufacturing 9% Chemicals, Pharma, and Health Care 9% Banking and Financial Services 10% Retail
2% Legal 3% Corporate Strategy 3% HR 18% Finance
6% Other
27% High Tech
6% General Management
18% IT
7% R&D and Engineering
18% Insurance
15% Sales, Marketing, and Retail
10% Customer Service 12% Production/ Operations/ Procurement
Respondents by Geography Sample Distribution 1% Other 2% Latin America 14% APAC
65% North America
18% EMEA CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
7
Overcoming the Insight Deficit
A Model of Insight IQ Insight IQ entails the ability to find and analyze relevant information, to drive actions and decisions effectively.
1 Attainability—Information is easy to find. ■■
■■
■■
The information that I need to do my job is available. I know which sources to go to for the information I need. I can find subject matter experts and other sources of tacit knowledge.
2 Usefulness—Information is useful. ■■
■■
Information Attainability
Information Usefulness
■■
■■
■■
Insight IQ
Information is useful and does not require modification or manipulation. I understand how information is defined. There is a single version of the truth for specific information. Information is directly relevant to business decisions and processes. Information is timely, up-to-date, precise, and accurate.
Knowledge Worker Capability
3 Knowledge Worker Capability—Knowledge workers have the analytic ability and predisposition to generate insight from information.
■■
■■
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
I rely on the results of analysis to support business decisions, processes, and deliverables. I am comfortable with advanced analytics.
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
8
ASSESSING THE MOST POWERFUL DRIVERS OF Insight IQ 2 Determine Drivers of Insight IQ
1 Deploy Surveys Knowledge Worker Survey (n = 4,941.) Knowledge workers from all major functions and business areas are surveyed on information usage.
IT Leader Survey (n = 22.) IT representative provides details on information management platforms, strategy, governance, and architecture.
3 Translate Findings into Practical Guidance ■■
Potential Drivers of Insight IQ
■■
1. Knowledge Worker Attitudes and Incentives 2. Training and Support 3. Staff Knowledge and Competencies 4. Information Quality 5. Information Usage Patterns 6. Demographics 7. Information Management Structures and Processes 8. Investment and Resource Level 9. Information Strategy and Governance 10. Technical Environment
Distill the practical implications borne out by the data. Provide real-world illustration of key insights via practitioner tools and tactics.
Illustrative Driver: Availability of Training on Conducting Analysis Average Insight IQ Score, Indexed ∆ = 29% (Maximum Impact)
129
100
Sample Questions Which of the following best describes the analytic tools available to you?
No Tools
Information Usefulness
Information Attainability
Multiple Tools
Insight IQ
How effective is the training provided by your organization on conducting analysis?
Very Ineffective
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
No Training on Conducting Analysis Available
Knowledge Worker Capability
Very Effective
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Yes, Training Available on Analysis
Existing Research
Occasion for the Research
9
Some knowledge workers have up to three times higher Insight IQ than their peers.
Overcoming the Insight Deficit
Finding 1: Insight IQ Varies Substantially Across Knowledge Workers Knowledge Worker Insight IQ1 Range Database Percentile Values of Insight IQ
Some knowledge workers demonstrate almost three times higher Insight IQ than their peers.
77 65
51
38 28
10th Percentile
25th Percentile
50th Percentile
75th Percentile
90th Percentile
n = 4,941 knowledge workers. 1
Insight IQ is measured as an index constructed from the three model components. Index scores ranges from 0 to 100, with higher scores reflecting higher levels of Insight IQ.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
10
There is wide variation in Insight IQ across knowledge workers within the same organization.
Variation Within and across Insight IQ1 Range by Organization
90th Percentile ■■
■■
Although not all knowledge workers need to reach even the average level of Insight IQ, small improvements can have a significant cumulative effect given the ubiquity of knowledge workers within most organizations.
Average 10th Percentile
100 90 80
Variation within organizations demonstrates that company-level factors do not explain all differences in Insight IQ; there are important factors at the functional or knowledge worker level as well.
70 Spread of Average Insight IQ Across Organizations
60 50 40 30 20 10 0 Representative Sample of Participating Organizations
n = 22 organizations. 1
Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
11
Business functions with a high Insight IQ demonstrate 24% better performance than functions with a low Insight IQ.
■■
■■
Performance results were measured at the functional level to minimize the impact of exogenous factors and self-reporting bias. A performance index of 12 questions across the four categories of self-reported data was created to compare top- and bottomquartile functional teams.
Overcoming the Insight Deficit
Finding 2: OVERCOMING THE INSIGHT DEFICIT Has a Measurable Payoff Indexed Functional Performance by Level of Insight IQ
Performance of Functions with the Highest Insight IQ
1.24x
Twenty-four percent higher functional performance Performance of Functions with the Lowest Insight IQ
x
Components of Functional Performance Index Relative to similar functions at industry peers: ■■ ■■ ■■
■■
Functional effectiveness Functional efficiency Function-specific performance (e.g., market share growth, employee engagement) Impact of analytics on effectiveness, efficiency, and performance
n = 4,941 knowledge workers. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
12
The CIO Executive Board controlled for demographic factors that impact Insight IQ during all analysis.
Finding 3: Three Demographic Factors Impact Insight IQ Potential Demographic Drivers What Matters ■■
Seniority
■■
Openness to new technologies
■■
Importance of analytics to job role
■■
Age
■■
Highest degree earned
■■
College major
■■
Organizational tenure
■■
Company revenue
■■
Level of regulatory burden
■■
Information intensity
■■
Industry
■■
IT organizational structure
■■
Geographic span of business
What Doesn’t Matter
The CIO Executive Board applied statistical controls to void the impact of demographic factors that drive Insight IQ.
Knowledge Worker–Level Factors
Organization-Level Factors
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
13
Overcoming the Insight Deficit
Finding 4: All Components of Insight IQ Matter Poor performance in any of the three components of Insight IQ significantly diminishes the probability of strong business performance. Type 1: Knowledge Workers Cannot Easily Access Information
Information Attainability
Information Usefulness
Information Attainability
Type 3: Knowledge Workers Cannot Translate Information into Action
Information Usefulness
Information Attainability
Type 4: High Insight IQ
Information Usefulness
Information Attainability
Information Usefulness
Insight IQ
Insight IQ
Insight IQ
Insight IQ
Knowledge Worker Capability
Knowledge Worker Capability
Knowledge Worker Capability
Knowledge Worker Capability
Probability of strong business performance1 if information attainability is low: 18%
1
Type 2: Knowledge Workers Are Wary of Analytics
Probability of strong business performance if knowledge worker capabilities are low: 27%
Probability of strong business performance if information usefulness is low: 35%
Probability of strong business performance: 80%
Values are calculated as the probability of knowledge workers demonstrating top quartile performance in indexed business performance via logistic regressions.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
14
The likelihood of realizing full business value from an information management initiative is only 5% to 35% if the organization does not invest in all three components.
■■
■■
NO SHORTCUTS TO DEFICIT REDUCTION Likelihood of Realizing Full Value from Information Management and Analytics Percentage of Likelihood Under a Typical Deployment Pattern 80%
Yet, many organizations focus on system investments and governance at the expense of knowledge worker engagement.
User Engagement Typically unfunded
Each component considered individually is necessary, but not sufficient, to achieving full business value.
Data Governance Total Annual FTE costs1: $3 M
27% Without effective information governance, the likelihood of realizing full value is only 5%.
System Investments Annual project spend1: $18 M 5% 0% A data desert—no information is available.
1
Staff openness to analytics boosts the likelihood of realizing full value from 27% to 80%.
Technology without governance— information is attainable but quality and relevance are low.
Enough information to be dangerous— information is useful but staff don’t know how to analyze it.
Full value realization— there is complete absorption of analytics for decision making.
Estimates for a company with revenue of $8 B.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
15
We analyzed the dataset to isolate the most important drivers of Insight IQ and translate the findings into practical guidance.
Overcoming the Insight Deficit
FINDING 5: TOP DRIVERS OF INSIGHT IQ
Potential Drivers of Insight IQ Tested 1. Knowledge Worker Attitudes and Incentives 2. Training and Support 3. Staff Knowledge and Competencies 4. Information Quality 5. Information Usage Patterns 6. Demographics 7. Information Management Structures and Processes 8. Investment and Resource Level 9. Information Strategy and Governance 10. Technical Environment
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Information Usefulness
Information Attainability Insight IQ
Knowledge Worker Capability
Overview of findings
Roadmap for the discussion
Existing Research
16
Partial List of tested drivers I. Information Management Structures and Processes 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.
Analytic Capabilities and Practices Shared Analytics Teams Centralized Companywide Central Analytics Group in IT, Reporting Directly to CIO Central Analytics Group in IT, Within Information Management Delivery Channels for Providing Information/Analytics Degree of Integration of Management of Analytics and Information Degree of Knowledge Worker Involvement with Other Functions/Groups Existence of Formal Analysis Processes/Frameworks Existence of Master Data Management Tool Existence of Metadata Structure (Standards and Models) Integration of External Sources of Information into the Data Warehouse Information/Analytics Requirements Incorporated into Other Projects Early IT Organization Structure (e.g., Centralized or Decentralized) Knowledge Workers Have Enough Time to Conduct Analysis Master Data Management Process Includes Error Detection and Correction Mechanisms of Delivery for Each Type of Information— Pushed or Pulled Prevalence of Analytics in Group Versus Individual Decisions Process for Sharing Information Responsibilities for IM Group (Including Who Owns What) Responsibilities of Central Analytics Group Use of Aggregation, Visualization, and Analysis Use of Agile Software Development Use of External Providers for Analytics
II. Training and Support
24. 25. 26. 27.
Analytic Mind-Set of Respondent’s Manager BU/Functional Performance Metrics Reported Regularly Formality of Decision-Making Process Leadership Attitudes/Behaviors Toward Fact-Based Decision Making 28. Overall Frequency of Using Analysis 29. Use of KPIs/Metrics to Track Organizational Performance 30. Performance Reviews Based on Measurable Goals 31. Performance Reviews Evaluate Analytic Ability 32. Time Spent on Analysis 33. Existence of Processes/Capabilities to Overcome Biases
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Shading Denotes Top 15% Most Impactful Drivers
III. Knowledge Worker Attitudes and Incentives
V. Quality of Information (Continued)
34. User Confidence in Information Quality (Cleanliness and Accuracy) 35. What Level of Confidence Do You Have in the Quality (Cleanliness and Accuracy) of Information from Each of the Following Functions? 36. Knowledge Workers Are Rewarded and Promoted Based on Outcomes of Their Analysis and Decisions. 37. Knowledge Workers’ Perceived Ease of Use of Information Contained in Central Repository 38. Knowledge Worker Preference for Push Versus Pull for Types of Information 39. Knowledge Worker Trust in Information Contained in Central Repository 40. Knowledge Worker Trust in Information from Other Groups/ Functions (and Reason) 41. Level of Blame for Relying on Inaccurate Prediction 42. Perception That Being Analytic Is Advantageous to Career Development. 43. Perception That Analytics Is Hard Due to Lack of Time. 44. Perceived Importance of Information Aggregation Capabilities 45. Perceived Importance of Information Filtering Capabilities 46. Perceived Importance of Information Visualization Capabilities 47. Visibility of Internal Opportunities That Require Analytic Skills
60. New Technologies Prescreened for Enterprise Data Model Alignment 61. Quality Standards Vary by Function 62. Quality Standards Vary by Information Type 63. Quality Standards Vary by Seniority 64. Satisfaction Level for Each Type of Information 65. Timeliness Standards Vary by Information Type 66. Use of Common Definitions and Vocabulary in Information Presentations
VI. Information Usage Patterns
IV. Investment and Resource Level 48. Approach Taken to Getting Approval for Investments in IM/ Analytics 49. Cost of/Investment in Analytic Capabilities (by Function) 50. Level of IT Investment in Capabilities to Support Group Decision Making 51. Number of IT Staff Responsible for Information Management 52. Technology Costs for Managing Information
67. Capabilities in Place to Obtain Cross-System/Functional Information 68. Degree of Integration of Cross-Functional Information 69. Frequency of Using Each Type of Information/Analysis 70. IT Support for Different Types and Sources of Information 71. Nature of Information/Analytics Use (e.g., Customer Analysis, BPO) 72. Need of Information from Different Functions 73. Number of Sources Required for Each Type of Analysis 74. Purposes of Information/Analytics Use (e.g., Decisions, Reporting, Pattern Spotting) 75. Relative Use of Different Types of Info. (e.g., Structured/ Unstructured, Internal/External) 76. Satisfaction Level for Mobile Access to Key Information Types 77. Time Spent on Information Visualization, Filtering, Aggregation, and Analysis 78. Types of Analyses Being Done (e.g., Predictive, Prescriptive) 79. Types of Information Being Used 80. User Feedback on Information Sources and Analytics Tools Collected 81. Users Information Needs Assessed via Surveys 82. Users Information Needs Assessed by Tracking Information Requests
V. Quality of Information
53. Existence of Automation of Information Quality Governance and Standards 54. Existence of Process for Updating and Retiring Information (Info. Lifecycle Management) 55. Existence of Standard Definitions for Information 56. Data Transformations Are Transparent to Users. 57. Frequency of Reviewing/Updating Analytic Models and Assumptions 58. Functional Information with Standard Definitions 59. Information Quality Regularly Reviewed/Audited
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
17
Partial List of tested drivers (Continued)
Overcoming the Insight Deficit
18
Shading Denotes Top 15% Most Impactful Drivers
VII. Staff Knowledge and Competencies
VIII. Information Strategy and Governance
IX. Technical Environment
83. Analyst Skills Valued 84. Analytics Staff Hired and Assessed for Training and Coaching Abilities 85. Assessment of Analyst Speed, Quality, and Responsiveness 86. Availability of Support in Finding or Analyzing information 87. Availability of Training on Finding or Analyzing Information 88. Consumption of Training on Finding or Analyzing Information 89. Existence of Training on Use of IM and Analytic Systems 90. Effectiveness of Training on Finding or Analyzing Information 91. High-Quality Training on Finding, Understanding, and Analyzing Info. 92. Inclusion of Analytics in Job Descriptions 93. Knowledge Worker Analytic Skills and Level 94. Knowledge Worker Awareness of Security Policy Related to Information Sharing 95. Knowledge Worker Degree of Proficiency in Using Analytic Tools 96. Knowledge Worker IT Skills 97. Knowledge Worker Knowledge of What Information Exists in the Central Repository 98. Knowledge Worker Perception of Ease of Use of Sharing Analysis/Info. Using Analytic Tools 99. Knowledge Worker Relies on Judgment When Disagreeing With the Results of Analysis 100. Knowledge Worker Understanding of Analysis Behind Strategic Decisions 101. Knowledge Worker Understanding of How Information is Useful in Other Functions/Groups 102. Knowledge Worker Understanding of Responsibilities of Other Functions/Groups (i.e., What They Do) 103. Presence of Analytics Coach (Formal or Informal) 104. Processes for Understanding Tool Usage and Workflows 105. Skills and Knowledge of Staff Responsible for Managing Information 106. Use of Hypotheses to Drive Analysis
1 07. 108. 109. 110. 111. 112. 113. 114. 115. 116. 117. 118. 119. 120. 121.
134. Analytic Tools Bought Off the Shelf or Developed In House 135. Analytics Solutions Deployed Using a Test and Learn Approach 136. Automated Decisions Implemented 137. Customizability of Analytic Tools 138. Enterprise Search Implemented 139. Existence of and Types of Collaboration Tools/Capabilities 140. Existence of Automated Information Capturing Capabilities 141. Existence of Central Repository 142. Existence of a Data Warehouse 143. Existence of In-Memory Technology Capabilities 144. Existence of Non IT-Supported Information Repositories 145. Degree of Integration of Operational Systems 146. Information Sources (e.g., Smart Networks, Sensors) 147. Knowledge Workers Have Access to Analytic Tools Other Than Excel 148. Knowledge Worker Can Choose from Multiple Analytic Tools Available 149. Legacy Systems 150. Number of Analytic Tools Being Used/Provided 151. Number of Features of Analytic Tools 152. Number of Information Repositories 153. Number of Operational Systems (by Function) 154. Tool Interface Design, Testing, and Enhancements 155. Use of Cloud Applications to Generate, Store, or Analyze Information
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
122. 123. 124. 125. 1 26. 127. 128. 1 29. 130. 131. 132. 133.
Occasion for the research
Approach Taken on Analytic Tool Selection and Distribution Approach Taken on Analytic Tool Selection and Distribution Clear Goals or Strategies for Social Media in Place Components of Analytic Strategic Plan Coverage of Analytics Strategic Plan Decision Stakeholders Degree of Standardization of Key Business Processes Degree of Sharing Information with Customers Degree of Sharing Information with Suppliers Documentation of Business Processes Existence of Analytics Strategic Plan Existence of Data Steward Role Existence of Dedicated Information Management Team Existence of Strict Security Policies Around Information Information Quality Standards and Policies Are Defined and Enforced Information Quality Standards and Policies Enforced by Info. Management Group Information Quality Standards and Policies Enforced by Shared Services Group Information Security and Access Policies Do Not Inhibit Attainability of Information Inclusion of Analytics in Broader Corporate/IT/Other Function Strategic Plan Information Architecture in Place or Planned Keeping Track of New Information Systems and Analytic Capabilities Organization Characteristic (e.g., Consensus-Based or Hierarchical) Organization Strategic Business Goals/Objectives Overall Ownership of IM Group Perceived Clarity of Roles and Responsibilities for IM Group Policies for Adding Information to a Data Warehouse Robustness of Enterprise Data Model
The Insight IQ Diagnostic
Overview of findings
X. Demographics 156. Function Responsible for Developing Analytics Strategic Plan 157. Knowledge Worker Career Background (Experience, Number of Functions Worked In) 158. Knowledge Worker Function 159. Knowledge Worker Seniority 160. Organization Industry
Roadmap for the discussion
Existing Research
1
CIO0361911SYN
CIO Executive Board IT Practice www.cio.executiveboard.com
Occasion for the research The Insight IQ Diagnostic
Overview of findings
24.7% 24.6%
15.0% 14.8%
Roadmap for the discussion
10. Surveying KWs to Determine Information Needs
2. Support Big Judgment, Not Just Big Data
3. Collecting KW Feedback on Information Sources and Analytics Tools
16.4%
18. Hiring Analytic Staff for Their Training and Coaching Abilities
13.7%
16. Considering KW’s Analytics Capability When Evaluating Performance
17.4%
11. Basing Performance Reviews on Measurable Goals
18.6%
8. Regularly Reporting Business/ Functional Performance Metrics
20.0%
7. Providing KWs with Access to Analytic SMEs
6. KWs Are “Informed Skeptics”
1. Enable Information Usability
5. Formalizing Decision-Making Process
25.2%
4. Training KWs on How to Find and Conduct Analysis
14.9% 14.7%
20. KWs Choosing from Multiple Analytic Tools
16.0%
19. Enabling Enterprise Search
17.2% 16.4% 16.2% 15.5%
17. Providing Information Visualization and Aggregation Tools
The Usual Suspects
14. Providing Visibility into Data Transformations
2. KWs Having Confidence in Information Quality
15. Defining and Enforcing Quality Standards
13. Having Central Analytics Group or COE
11. Having a Mature Data Model
9. Having an Information Architecture
1. KWs Having Enough Time to Conduct Analysis
The Top Drivers of Insight IQ1 3. Observe and Question to Identify Opportunities
30.3% 24.8%
16.8%
n = 4,941.
KW = Knowledge worker.
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Existing Research
Occasion for the Research 19
To help drive insight IQ, IT organizations will require new roles or skill sets in four areas.
Overcoming the Insight Deficit
20
FINDING 6: INSIGHT IQ HAS IMPLICATIONS FOR IT ROLES AND SKILLS IT Roles and Skills Required in a High Insight IQ Organization
1. Enable Information Usability
2. S upport Big Judgment, Not Just Big Data
3. O bserve and Question to Identify Opportunities
Information Architects
Information Insight Enablers
Corporate Anthropologist
Develops and maintains information architecture that enables knowledge worker productivity and decision making
Supports knowledge workers with insights, business intelligence, and management reports for effective decision making
Uses observational and experience mapping techniques to identify unmet needs for information and analytic capabilities and other technology functionality
User Experience Specialists INFORMATION TECHNOLOGY PRACTICE
CIO EXECUTIVE BOARD ™
The IT Talent Implications of the Future of Corporate IT A Guide to Filling Emerging IT Skills Gaps
Designs and configures user-centric interfaces and information visualization techniques based on a deep understanding of end-user requirements and behaviors
For full details about our analysis of emerging IT skills and roles, see The Talent Implications of the Future of Corporate IT study. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Business leaders outside IT have a vital role to play in creating a high Insight IQ organization.
Finding 7: Information Ownership is a Critical Business Role Guidance to Business Leaders for Driving High Insight IQ
■■
To highlight the criticality of business leader involvement in information management, Corporate Executive Board sent a summary of Overcoming the Insight Deficit to all members in all functions globally.
1. Take Ownership of Your Information Business leaders must determine what data is necessary, what analyses they want to run, how data is shared across organizational boundaries, and how they derive insight from data. EXECUTIVE GUIDANCE FOR 2011
Overcoming the Insight Deficit Big Judgment in an Era of Big Data
2. Put Information at the Heart of the Business Plan Tightly integrating information management plans with business strategic plans drives higher Insight IQ. This means deconstructing vague strategy into more stable business capabilities. 3. Create Incentives for Cross-Company Data Standardization Efforts Executives should create a strong governance process that cuts across the company, including a corporate “tax” to overcome the first-mover disadvantage. 4. Improve the Quality of Your Information Business leaders should own the maintenance of authoritative data sources, approve new and modified definitions, establish confidentiality levels, and define data-quality processes. 5. Centralize Management, Not Information Foundational analytic investments, such as a dedicated analytics team, training design, and information quality norms, benefit from centralization, create the oversight to sustain decentralized information sources.
To access copies of this report for business executives, visit http://www.executiveboard.com/executive-guidance/2011/Q3/index.html.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
21
Overcoming the Insight Deficit
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
22
Information management is not a new topic; most organizations have foundational capabilities in place.
Building on a Solid Foundation Existing Capabilities and Relevant Resources Architecture Roadmapping
■■
While necessary, these capabilities are not sufficient to boost an organization’s Insight IQ or to move IT’s capabilities from process automation to information management.
Target Architecture Roadmaps (Wells Fargo)
Business Architecture (KBC Bank)
Information Strategy and Stewardship Task-Specialized Data Stewardship (RBC Financial) Information Quality Data Quality Audit Team (Hewlett-Packard)
Information Health Scorecard (International Monetary Fund)
Process Breakpoint Identification (Intel)
Report Generation Business Metrics Selection Tool (Corporate Executive Board) Systems Integration and Rationalization End-to-End Simplification Business Case (Charles Schwab)
To access the Board’s existing resources, Visit the Information Management topic center at www.cio.executiveboard.com. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Applications Portfolio Rationalization (CNA Financial)
User Support Segmented Information User Support (Hallmark)
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
23
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
24
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Enable Information Usability
25
One-half of all knowledge workers find that information from corporate sources is not in a usable format.
■■
Overcoming the Insight Deficit
THE USABILITY DEFICIT Information from Corporate Sources Is in a Usable Format Percentage of Knowledge Workers
Improve information usability by letting knowledge workers choose the tools they find easiest to use, prioritizing information visualization and offering greater transparency into quality levels.
50% No
50% Yes
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
26
Knowledge workers rely on a wide range of information sources, both internal and external.
■■
The proliferation of information sources creates challenges for information aggregation, presentation, and quality.
INFORMATION OMNIVORES Mean Number of Corporate Information Sources Used by Knowledge Workers at Least Weekly
Corporate Operational Systems
4.7
Corporate Unstructured
4.2
Use External Sources of Information in Day-to-Day Work Percentage of Knowledge Workers
24% No
76% Yes
n = 4,941 knowledge workers. CIO Executive Board IT Practice www.cio.executiveboard.com
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
27
The proliferation of information users, uses, and sources drives new usability problems.
28
INFORMATION EVERY WHICH WAY Usability Challenges
More Information Users
More Information Uses
Executives
Customer value analysis
Line managers
Risk management
Analysts
Mobile presence
Frontline staff
Executive dashboards
Customers and suppliers
Automated decisions
1
Inadequate Tools—No single analytic tool can support all needs and skills levels.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Internal
External
2
Inflexible Reports and Cubes— Predefined reports cannot answer all knowledge worker questions while cubes are slow and offer limited dimensions.
3
Centralized Ad Hoc Reporting Inefficiencies—Relying on a central group to run custom queries creates lags in analysis time and can be an inefficient use of resources.
Flexible, Transparent Information Access
More Information Sources
CXO KPI Identification
4
Information Quality Issues— Quality varies significantly by information source in ways that are not always transparent to the user.
The IT Skills
Appendix
■■
To avoid a free for all, the choice should be shaped by a deep understanding of the business questions knowledge workers need to answer.
USABILITY = Bounded FLEXIBILITY AND TRANSPARENCY
Capability-Driven Tool Portfolio
Goal: High Uptake of Self-Service Analysis
High
Flexible and Transparent Analytics
Value Associated with the Approach
Usability can be improved by providing knowledge workers with greater flexibility in tools choice, information access, and use.
CXO KPI Identification Data Cubes
Ad Hoc EDW Queries by Specialist
Data Free for All
Dashboards Standard Reports Low One Size Fits All
Choice Shaped by Business Need Approach to Usability
1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
29
If knowledge workers don’t find the analytic tools provided by IT useful, they have an array of other choices.
■■
Users have choice (official or not) Use Analytic Tools That Are Not Supported by IT
Reasons for Using Analytic Tools That Are Not Supported by IT
Percentage of Knowledge Workers
Percentage of Knowledge Workers
The most common reason for using an unofficial tool is lack of an official alternative, the second most common reason is ease of use.
IT Does Not Provide an Alternative Tool
–– This is a 1.5 times more frequent reason than the need for a more powerful tool.
51%
32% Yes The Unofficial Tool Is Easier to Use
45%
68% No
The Unofficial Tool Is More Sophisticated
29%
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
30
Overcoming the Insight Deficit
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Greater choice in selecting analytic tools to use is a powerful driver of Insight IQ.
■■
Choice has Value Maximum Impact on Insight IQ1
The greatest increase comes from moving from one standard tool for the enterprise to standard tools aligned to a specific business unit or function.
∆ = 6.0%
Only Analytic Tool Available to Knowledge Workers Is Excel
∆ = 16.0% Knowledge Workers Must Use a Single Tool That Is Standardized Across the Organization
13.7%
7.7%
Knowledge Workers Must Use a Single Tool That Has Been Selected for Their Business Unit or Function
Knowledge Workers Can Choose from Multiple Analytic Tools Available
(8.3%) ∆ = 13.1%
(21.4%)
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
31
Greater tool choice doesn’t mean chaos but rather providing a portfolio of tools that meets the full range of capabilities needed by different knowledge worker communities.
■■
■■
■■
Rationalization programs often try to over-standardize the analytic toolset, leaving knowledge workers without needed capabilities and opening the door to unofficial tool adoption.
Overcoming the Insight Deficit
Not too Few, Not Too Many Three Approaches to Standards for Analytic Tools
Increases knowledge worker productivity
“Capability-Driven” Align the Tool Portfolio by Capability
Reduces redundancy while acknowledging diverse needs, preferences, and skills Provides principled criteria for future tool selection
ROI of the Tool Portfolio
Number of Tools in the Portfolio
Progressive organizations rightsize their portfolio of analytic tools by identifying and aligning with capabilities. IT leaders should avoid applying the same criteria to analytic tools that they use for other applications rationalization efforts, as the optimal number of analytic tools is likely to be higher than for other categories of application.
“Dictatorship”
“Anarchy”
Rationalize to One Tool
Ungoverned Tools
No single tool provides all capabilities. Rationalization expense erodes savings. Having only one tool may encourage use of unofficial tools.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
Proliferation of tools leads to uncontrolled licensing costs. Different tools are used by different teams for the same purpose. ROI dips as tools have limited usage or unexploited functionality.
CXO KPI Identification
The IT Skills
Appendix
32
Capability-Driven Tool Portfolio
Overview A methodology for selecting a set of analytic tools that collectively provide 10 key capabilities to knowledge workers Executive Teaching Focus on helping knowledge workers use the right tool for the right purpose rather than aiming to minimize the total number of tools. Identify the analytic capabilities most needed by knowledge workers, and use the capabilities to assess current and future analytic tools to identify gaps and redundancy in the tool portfolio. Company Snapshot Blue Cross Blue Shield of North Carolina Industry: Health Insurance 2010 Employees: 4,600
BlueCross and BlueShield of North Carolina (BCBSNC) provides health care insurance products and related services to about 3.7 million members in North Carolina. BCBSNC is a licensee of the Blue Cross and Blue Shield Association and was founded in 1933. The company’s health plans include Blue Care (HMO) and Blue Options (PPO), as well as consumer-directed plans. BCBSNC also provides dental, life, disability, long-term care, and Medicare supplemental insurance, as well as prescription drug coverage.
Source: http://www.hoovers.com. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
33
In an internal survey of tools used for analytic functions, BCBS NC learned that its portfolio of analytic tools is misaligned and duplicative and used for suboptimal purposes.
Overcoming the Insight Deficit
A Plentiful but Ill-Fitting Portfolio Blue Cross Blue Shield of North Carolina Analytic Tool Environment 2009 One Tool Fits All Teams use one tool (often Excel or Access) for all needs, missing the chance to use tools that better meet their needs. Everyone Gets Their Own Tool Multiple tools are used for the same purpose by different teams, resulting in wasteful duplication. A Screwdriver to Hammer in Nails Tools are used for purposes for which they are not optimized. Tools with Unfulfilled Potential Tools have with limited usage or unexploited functionality.
“We found that many teams were using the one tool they knew for as many functions as possible, while better tools went unused.” Todd Norris Director of IM Delivery BCBS NC CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
34
Define analytic capabilities needed by knowledge workers across tools, not functionality within tools.
■■
■■
■■
■■
BCBS NC defines 10 capabilities supported by analytic tools. The company surveys knowledge workers to determine which tools meet which capabilities. The capabilities are reviewed annually and adjusted according to evolving knowledge worker needs. See page 37 for an overview of the survey process and page 38 to 39 for a list of the survey questions.
Ten Analytic Capabilities Capability
Description
1. Data Collection/ Conditioning/Validation
Gathering, integrating, consolidating, modifying, and validating data
2. Static/Printable Reporting
Reports are specifically designed to be consumed as a printout. These reports have no “interactivity” and are typically scheduled to run on a regular basis or have a few short prompts for the user to enter.
3. Self-Service Reporting
Reports/queries are given to users to run on demand. They can be interactive with prompts and drill downs.
4. Ad Hoc Report Development
Reports are developed as part of exploration and are typically for a one time use. These may turn into self-service or printable reports.
5. Advanced Analytics/ Data Mining
Analysis typically done by a small number of analysts who look for new business insights by examining relationships of information and information trends, this includes data and text mining.
6. Data Visualization
Not limited to charts, this includes any functionality providing faster consumption of information through “pictures” or presentations other than data grids.
7. Dashboards/Scorecards
Dashboards—On-screen visualizations that provide a quick status of key performance indicators specific to a role Scorecards—Use colors, indicators, and numbers to compare actual performance of key metrics against a target or budget. These are slightly more detailed than dashboards and less visual.
8. Geospatial Analysis
This is an array information across a geography to show relationships between data and locations.
9. Operational Reporting
This is not used to analyze performance, but used for operational reporting. Examples include sending an invoice to a customer or a letter to a member.
10. Publishing and Delivery
Report distribution including e-mail, print and mail, or using a business intelligence (BI) tool to distribute through a built-in or custom portal for providers, members, or internal staff.
Nonspatial analysis and mobile BI will be added as emerging capabilities during the next refresh. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
35
Maintain a portfolio of tools that supports all 10 capabilities.
A Portfolio of ENTERPRISE and Specialty Tools
■■
BCBS NC identifies capability gaps and helps teams use the tools that best provide the capabilities they need.
Publishing and Delivery
Operational Reporting
Geospatial Analysis
Dashboards/Scorecards
Data Visualization
Advanced Analytics/ Data Mining
Ad Hoc Report Development
–– Specialty Tools support one capability in one particular function; another six to nine of these tools may be needed.
Data Collection/ Conditioning/Validation
–– Enterprise Tools support multiple capabilities used for the same purposes across the company; between three and five of these tools should support most capabilities.
Self-Service Reporting
Tools are designated in one of two categories:
Static/Printable Reporting
■■
Overcoming the Insight Deficit
Enterprise Tools (3 to 5) BObj Web Intelligence SAS Core Crystal Reports
Speciality Tools (6 to 9) “While we will always retire tools where the ROI isn’t there, we are more concerned about helping people use the right tool for the right purpose and get away from using one tool for everything.”
Amisys MACESS Teradata SQL Assistant MCSource EDIWatch
Todd Norris Director, IM Delivery BCBS NC
Capability gaps
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
36
The process of capturing analytic capabilities by surveying knowledge workers requires executive sponsorship and clear communication.
IMPLEMENTATION GUIDE: CAPABILITY SURVEY ROADMAP Keys to Successfully Surveying Knowledge Workers on Their Capability Requirements
Only key analytic areas identified by this group are surveyed.
1. Obtain sponsorship and direction from VP–level information governance group.
Representatives are told that each survey should take less than an hour per analytic area to complete.
2. Identify user representatives in each division to administer the survey.
3. Hold an orientation session to make sure knowledge workers understand the intent of each question. “The process of surveying was just as important as the questions we asked. The survey was not intended to capture perfect information but to give informed guidance for next steps.” Susan Helm-Murtagh VP, Information Management and Analytic Services BCBS NC
4. Follow up surveys with an interview with each group to validate results and ask questions. Three Weeks for Entire Process
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
37
Overcoming the Insight Deficit
38
IMPLEMENTATION GUIDE: Capabilities Survey I. Tools and Technology 1. Indicate the tools used in your area, by capability.
2. N umber of people proficient with each tool and proficiency level (1 = general understanding of the tool; 5 = expert level)
X X
X
2 1
3
4
5
Other 2
Other 1
Publishing and Delivery
Feedback on BI Technical Skills:
X
X
Other 1 X Other 1 X Any other feedback on the tools used in your area Any other feedback on the tools used in your area
1 2
Tool 3
Tool B Tool C
Tool Tool 1 Tool 2 Operational Reporting
Geospatial Analysis (mapping)
X
Visualization
X
X
Dashboards/ Scorecards
X
Advanced Analytics/ Mining
X
Self Service
X
Ad hoc Reporting
Tool A
Static/Printable Reporting
Tool
Data Collection/ Conditioning
BI Capabilities
X
X
X
3. C omment on tools that are particularly critical to your area or feel free to mention tools you feel, if improved, could save you time or provide greater information insights. 4. C omment on activities you do in your area that you feel should not be your responsibility or you feel your team is not adequately trained to support.
II. Effort 5. Number of people who do analytical or BI development work.
6. Percentage of time spent by high-level activity
Number of full-time equivalent employees in your area
Collecting and assembling data for analysis
20%
Conditioning, filtering, or validating data for analysis
20%
Developing new reports, report solutions, or analysis
20%
How many FTE resources would you need to fully satisfy the requests of your team?
9 9
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
IMPLEMENTATION GUIDE: Capabilities Survey 7. O f the time your area spends collecting and assembling data, list the percentage spent on each activity. Activity
8. O f the time spent conditioning and filtering information, how do you spend that time? %
Activity
%
Pulling information from various sources
50%
Developing/applying filtering and rules unique to your needs
90%
Manipulating information in spreadsheets/MS Access/SAS
50%
Developing/applying filtering rules than are/could be used by others
10%
100%
Total
Other 1 Total
100%
9. L ist the main sources of information you use or plan to use in the near future.
10. How does your report solution development time breakout?
Source
Type of Information
Activity
Source 1
Claims, membership, provider info
Time spent developing a specific report for a specific question
20%
Source 2
Call center information
Time spent creating a solution to answer many questions
80%
Source 3
Call center forecasting, staffing models
Total
100%
%
Critical (1–5)
Supported (X)
Support Hours
III. Applications and Solutions
Solution 1
5
X
20
Solution 2
4
X
2
Solution 3
5
Solution Name
11. In the grid to the left enter the following information: ■■ Solution Name: type in a solution name. ■■
Critical: 1 = nice to have; 5 = mission critical in your area
■■
Supported: put an X if the solution is supported by your area
■■
Support Hours: monthly hours spent supporting the solution
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
39
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
40
■■
To avoid a free for all, the choice should be shaped by a deep understanding of the business questions knowledge workers need to answer.
USABILITY = Bounded FLEXIBILITY AND TRANSPARENCY
Capability-Driven Tool Portfolio
Goal: High Uptake of Self-Service Analysis
High
Flexible and Transparent Analytics
Value Associated with the Approach
Usability can be improved by providing knowledge workers greater flexibility in tools choice, information access, and use.
CXO KPI Identification Data Cubes
Ad Hoc EDW Queries by Specialist
Data Free for All
Dashboards Standard Reports Low One Size Fits All
Choice Shaped by Business Need Approach to Usability
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
41
Less than one-half of all knowledge workers find standard reports and dashboards useful.
■■
Overcoming the Insight Deficit
REPORTING THE END OF THE REPORT Reports and Dashboards That Are Pushed1 Are Valuable Percentage of Knowledge Workers
Standard reports rarely contain the right information depicted in the right way to be to useful knowledge workers as they make decisions. 51% No
49% Yes
n = 4,941 knowledge workers. 1
Information supplied in standard dashboards and reports at a predefined frequency, often distributed electronically through e-mail.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
42
The proliferation of information sources increases the value of information visualization, aggregation, and filtering.
■■
Putting it All Together Maximum Impact on Insight IQ1 of Information Presentation and Analysis Capabilities
14.9%
These capabilities can be defined as follows:
13.6%
–– Visualization: the graphical depiction of information
11.6%
∆ = 2.1x
–– Aggregation: the combination of information from multiple sources –– Filtering: the selective omission of information to improve relevancy and legibility
5.5%
Information Visualization Capabilities
Information Aggregation Capabilities
Information Filtering Capabilities
Advanced Analytics Capabilities
n = 4,941 knowledge workers. 1
CIO Executive Board IT Practice www.cio.executiveboard.com
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
43
It is nearly impossible to uphold all dimensions of quality in all information sources.
■■
■■
Organizations with less than perfect information quality have two options: hide information until it is fixed or use information selectively in its current state. See pages 74–75 for full definitions and tips for measuring each of the nine dimensions of information quality.
Overcoming the Insight Deficit
44
Can All Information Be Perfect All the Time? Common Dimensions of Information Quality
1. Usability—Information is presented in a usable format.
2. Precision—Information is sufficiently granular.
3. Currency—Lag time before updated information is accessible.
4. Accuracy—Information accurately reflects the real world.
5. N on-Duplication—Information is not duplicated.
6. T imeliness—Information is available when needed.
7. Validity—Information conforms to business rules.
8. Consistency—The information conforms to its definition.
9. C ompleteness—Information fields are complete.
Information quality has many dimensions; few information sources comply with all of them.
Option 2: Help Knowledge Workers Use Information in Its Current State
Option 1: Avoid Using Information Until All Quality Problems Are Fixed “Perfect” quality requires costly multiyear effort.
Extracts value from information faster Helps prioritize most important quality fixes
External information quality may be unfixable.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Despite the high impact on Insight IQ, only about one-third of knowledge workers trust information from other functions in the organization.
■■
■■
■■
■■
Trust No One Maximum Impact on Insight IQ1 of Knowledge Worker Confidence in Information Quality
Have Confidence in Information Provided by Other Functions Percentage of Knowledge Workers
25.2%
This lack of confidence translates into perceived low information quality and usability.
32% Have Confidence in Information from Other Functions
The most trusted function is Finance, followed closely by Legal, with 47.5% and 43% knowledge workers willing to trust information from these functions, respectively.
16.5%
Marketing and R&D functions were the least trusted functions, with only about 25% knowledge workers willing to trust information from these functions.
Average for All Top 10% Drivers
68% Don’t Have Confidence in Information from Other Functions
Knowledge Worker Confidence in Information Quality (Cleanliness and Accuracy)
See page 73 for additional details on how each function assesses the quality of its own and other functions’ data. n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
45
Lack of transparency in how information is collected, stored, or analyzed is the number-one reason why knowledge workers lack confidence in the quality of available information.
■■
Knowledge workers need clarity on where information came from and how it has been transformed.
Overcoming the Insight Deficit
Trust From Transparency Reasons for Lack of Confidence in Information Quality Provided by Other Functions Percentage of Knowledge Workers
Lack of Transparency in How Information Is Collected, Stored, or Analyzed
60.0%
Multiple Versions of the Same Information Exist
Lack of transparency is a more significant reason for lack of confidence than the underlying information quality problems.
42.0%
Lack of Common Vocabulary/ Standard Definitions
23.8%
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
46
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Flexible and Transparent Analytics Overview An approach to information management and architecture that provides the flexibility and transparency knowledge workers need to conduct the analysis they require within a universe of information related to key business performance questions Executive Teaching A lack of understanding of information sources, definitions, quality, and relationships makes it hard for knowledge workers to assemble the information they need to address key business performance questions. The diversity of knowledge worker information needs means that it is not possible to support them with predefined reports and cubes and central support for ad hoc queries is slow and resource-intensive. Instead IT should promote analysis freedom and confidence by offering a flexible architecture and transparency into underlying information sources. Component Teachings
1. Business Performance Question Mapping—Identify the universe of information knowledge workers require by mapping the business performance questions they need to address.
2. Flexible Data Analysis Architecture—Enable flexible, on-the-fly analysis capabilities through in-memory architecture organized around key business questions. 3. Contextualized Information Access—Provide knowledge workers with access to information organized by their top business questions. 4. Data Quality Transparency—Provide transparency into data quality and source to help knowledge workers decide when the data can and can’t be used. Company Snapshot GlaxoSmithKline plc Industry: Pharmaceuticals 2010 Sales: US$43.9 B Employees: 98,485
GlaxoSmithKline (GSK) is one of the world’s leading pharmaceutical firms. GSK produces medicines that treat major disease areas such as asthma, anti-virals, infections, mental health, diabetes, cardiovascular, and digestive conditions. In addition, the company is a leader in vaccines and is developing new treatments for cancer. GSK also markets other consumer products, many of which are among the market leaders.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
47
A knowledge worker survey revealed high dissatisfaction with the amount of time wasted trying to assemble data provided in disparate reports and dashboards.
■■
■■
GSK’s North American Sales and Marketing IT department set out to understand the lifecycle of information delivery to support business processes. The results revealed a need for better understanding of key business questions to improve the speed and flexibility of analysis capabilities in the sales team.
Overcoming the Insight Deficit
Assembling Data is Very Time Consuming Knowledge Worker Data Satisfaction Survey Results Illustrative Results Shown
Knowledge Worker Pain Points
Director Area Sales
Director Sales Support
Product Director
Field Sales Specialist
Field Sales Manager
1. Unanswered Questions 2. Missing Data 3. Can’t Access Data 4. Tools Aren’t Useful 5. Slow Response to Data Needs 6. Data Inconsistency and Inaccuracy 7. Need More Historical Data 8. Waste Time Assembling Data
No Expressed Problem in This Area
The most serious pain point was the amount of time knowledge workers spent assembling data before they could conduct analysis.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
I Experience This Problem, However It Does Not Have a Significant Impact on My Work I Experience This Problem, and It Has a Significant Negative Impact on My Work
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
48
Reducing wasted time assembling data demands improvements to data mapping, flexible access, and metadata transparency.
Why Knowledge Workers Waste Time Assembling Data
Assembling Data Is Very Time Consuming
Challenge
Root Causes
“Knowledge workers wanted to assemble and manipulate data to explore cause and effect, likelihood of recurrence, and predictive analysis, but in our old model that meant intervention by IT. We needed to find ways to let them do this themselves.”
Asking knowledge workers what data they need misses how the data should be assembled.
1 Solution Components
2
Business Performance Question Mapping
Knowledge workers don’t know which combination of data will answer their business questions.
Predefined reports and cubes answer a narrow range of business questions.
3
Flexible Data Analysis Architecture
Knowledge workers don’t understand the data sources’ quality, definitions, and business rules.
4 Contextualized Information Access
Metadata Transparency for Power Users
Rob Jester Director, Enterprise Business Architecture, Sales and Marketing GSK CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
49
To identify the universe of information knowledge workers require, start by isolating the top business performance questions they need to answer.
■■
■■
Asking knowledge workers what information they want leads to a siloed approach that produces information that requires further aggregation. Instead, the Information Architecture group examines the business questions knowledge workers need answers for; from there, they can identify the combination of information subjects and sources that are necessary to answer these questions.
Overcoming the Insight Deficit
First, Ask the Right Questions
1. Identify Top Business Performance Questions
Business surveys and focus groups lead to identification of top 400 business questions. Example question: What prescription details help to predict likelihood to fulfill prescriptions?
IT works with business stakeholders to group the 400 questions into 20 categories.
2. Group Closely Related Questions
Example question groups: prescription fulfillment
3. Map Questions to Data Subject Areas Qu.1 Qu.2 Qu.3 Qu.4 Qu.5 Qu.6 RX
Information architects combine business questions and data knowledge to develop a detailed Business Subject Area Data Map that documents the relationships between the questions and the data subject areas. Example subject areas: prescriber, prescriptions
Prescriber Claims Patient Product
“To understand how knowledge workers needed to assemble data, we first had to know the questions they were asking.” Rob Jester Director, Enterprise Business Architecture, Sales and Marketing GSK
Information architects identify related data sources.
4. Identify Related Data Sources
Example data sources: patient database, sales database, EDW, prescriber database CRM ERP
EDW HR
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
50
Help knowledge workers answer business performance questions by equipping them to assemble, analyze, and visualize information from multiple sources as they need it.
Flexible Analysis Architecture Data Analysis Tool Architecture Illustrative
Presentation Layer
3 Users can access and segment data through high-quality visualizations to get fast answers to their top business questions.
Flexible Analytical Layer
2 Relevant data is moved into the in-memory database of the analysis tool to enable fast processing across all possible data dimensions.
In-Memory Architecture
Data Sources
1 Data sources are selected based on the business questions documented by the data map.
Prescriber DB
Rx Sales DB
CRM
Enterprise Data Warehouse
Note: GSK uses QlikView as its reporting tool. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
51
Provide an online data map to help knowledge workers understand information availability and interrelationships.
■■
■■
■■
The data map provides an overview of available information and shows the connections between information.
SHOW HOW IT ALL FITS TOGETHER Clickable Data Map Aligned to Business Subject Areas Illustrative
Business Subject Areas: Sales and Marketing
Claims
Providing information access organized by business subject areas makes it easy for knowledge workers to identify the most relevant information for their question or role.
Patient
Payer
Rx
Detailed data concept models, fact sheets, and information flow charts lie under each business subject area, available for heavy analytics users in the organization.
Prescriber
Formulary
Business
Order
Contracts Marketing Events
Call Activities (Sales)
Visualizing Data Relationships—A product manager can visualize how claims data links back to Rx data through patient data, to help understand how to conduct analysis. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Flexible, Transparent Information Access
Non-Retail
Clicking on a business subject area provides additional information about the data quality, sources, and relationships.
CXO KPI Identification
The IT Skills
Appendix
52
Provide full transparency into information sources, flows, and quality to increase usefulness and knowledge worker confidence in the information.
■■
■■
FULL TRANSPARENCY MAKES DATA MORE USABLE Centralized Metadata Portal Illustrative
Knowledge workers can view this information through the data map.
GSK Connect Welcome to Data Point Home
Metadata and additional information models and training is available for IT and power users in the business.
Data Glossary
Subject Areas
Standard Reports
Data University
Prescription (Rx) Sales Data Subject Area Description: Metrics (business facts) related to both non-retail sales of drug products and retail prescription fills. Retail prescription data are collected at the national and subnational level.
Quality Transparency—A product manager can review the Rx Data Fact Sheet to learn about the accuracy of the data and become aware of any caveats in how it should be used for analysis.
A
Data Fact Sheet—Provides description of data types, sources, granularity, and quality levels Information Flow—Visually depicts flow of information from original source to reporting tools
?
Data Concept Model—Visually depicts logical data model for national prescriptions Data Owner—Click Here to contact.
Users who need more detail or clarification can easily contact data owners.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
Users can access training materials and information about registering for upcoming training.
The IT Skills
Appendix
Enable Information Usability
53
By providing the tools and contextual information needed for self-service, GSK reduces the time knowledge workers spend assembling information.
Overcoming the Insight Deficit
Faster Assembly, Less Help Required
Early Benefits Claims
Patient
Payer
Rx
IT and the rest of the business share an enterprise-wide view of customer data needs.
Example: Time to Complete Complex Analysis on Rx and Claims Data 30 Days
Simplified access to information 87% 4 Days
Reduction in time spent generating reports Traditional Approach
New Approach
The new approach eliminated the need to create 65 reports when analyzing patterns in the relationship between prescriptions and claims.
“Sales and Marketing have always had strong demand for information on a short turnaround. Now we are equipping them to get the information they need faster and with much less effort.” Rob Jester Director, Enterprise Business Architecture, Sales and Marketing GSK CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
54
Data Fact Sheets provide power users with metadata details to support their understanding of the data sources and quality levels.
Implementation Guide: Data Fact Sheet
Data Fields on the Data Fact Sheet ■■
Data Area Name
■■
Integration Requirements
■■
Data Area Type
■■
Dimension Requirements/Gaps
■■
Data Subarea
■■
Data Anomalies
■■
Data Description
■■
Data Security Requirements
■■
Data Objective
■■
Target Delivery Requirements
■■
Data Source/Vendor
■■
Current Usage
■■
Data Granularity
■■
Disaster Recovery
■■
Facts/Metrics
■■
Data Quality Requirements
■■
Source Delivery Mechanics
■■
Date Entered
■■
Source Delivery Frequency
■■
Data Architect
■■
Refresh Strategy
■■
Reference Documentation
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
55
Analytical power users can attend a Data University to deepen their understanding of data sources, relationships, and usability.
■■
■■
Overcoming the Insight Deficit
Implementation Guide: Data University Course Schedule Day One
Course
9:00 a.m.–9:10 a.m.
Welcome, Introductions, and Course Overview
GSK’s training group facilitates the training program, with data architects speaking throughout the sessions to provide subject matter expertise.
9:10 a.m.–9:30 a.m.
Data Use, Disclosure, and Confidentiality
9:30 a.m.–9:50 a.m.
Customer Dimensions
9:50 a.m.–10:05 a.m.
Break
10:05 a.m.–10:25 a.m.
Alignment Dimensions
10:25 a.m.–10:45 a.m.
Product Dimensions
Sessions focus on understanding the sources of data and relationships between information, enabling knowledge workers to independently apply judgment on data usage.
10:45 a.m.–11:05 a.m.
Static Views/Goals Data
11:05 a.m.–11:15 a.m.
Introduction: Retail
11:15 a.m.–11:45 a.m.
Retail, Dynamic Claims Lifecycle, and Consumer Rx data
11:45 a.m.–12:30 p.m.
Lunch
12:30 p.m.–1:50 p.m.
Retail, Dynamic Claims Lifecycle, and Consumer Rx data (Continued)
1:50 p.m.–2:15 p.m.
State Data Restrictions
2:15 p.m.–2:40 p.m.
Monthly Retail (IRx, PRx)
2:40 p.m.–3:10 p.m.
Weekly Rx
3:10 p.m.–3:25 p.m.
Break
3:25 p.m.–4:00 p.m.
Prescriber Track/Passport
4:00 p.m.–4:20 p.m.
CRx (Consumer Rx)
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
56
Implementation Guide: Data University Course Schedule (Continued) Day Two
Course
9:00 a.m.–9:05 a.m.
Welcome/Introduction
9:05 a.m.–9:35 a.m.
Targeting
9:35 a.m.–10:30 a.m.
Formulary/Payer Dimensions
10:30 a.m.–11:00 a.m.
Break
11:00 a.m.–11:25 a.m.
Call Activity
11:25 a.m.–11:30 a.m.
Non-Retail Introduction
11:30 a.m.–12:30 p.m.
IMS Health: Non-Retail Data
12:30 p.m.–1:15 p.m.
Lunch
1:15 p.m.–1:35 p.m.
Non-Retail Data
1:35 p.m.–1:55 p.m.
Specialty Sales Request System (SSRS)
1:55 p.m.–2:20 p.m.
Non-Retail Reports
2:20 p.m.–2:25 p.m.
National Data Introduction
2:25 p.m.–3:20 p.m.
IMS Health: National Audits Data
3:20 p.m.–3:30 p.m.
Break
3:30 p.m.–4:00 p.m.
IMS Health: National Audits Data (Continued)
4:00 p.m.–4:20 p.m.
National Audits Data from IMS Health
4:20 p.m.–4:50 p.m.
Dashboard Reports
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
57
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
58
■■
To avoid a free for all, the choice should be shaped by a deep understanding of the business questions knowledge workers need to answer.
USABILITY = Bounded FLEXIBILITY AND TRANSPARENCY
Capability-Driven Tool Portfolio
Goal: High Uptake of Self-Service Analysis
High
Flexible and Transparent Analytics
Value Associated with the Approach
Usability can be improved by providing knowledge workers greater flexibility in tools choice, information access, and use.
CXO KPI Identification Data Cubes
Ad Hoc EDW Queries by Specialist
Data Free for All
Dashboards Standard Reports Low One Size Fits All
Choice Shaped by Business Need Approach to Usability
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
59
The traditional approach to defining key performance indicators (KPIs) fails to meet CXO and operational needs simultaneously.
■■
■■
■■
Rolling up metrics from operations requires extensive systems and information harmonization and may still miss external or new information that is not used for operations.
Overcoming the Insight Deficit
Rigid Roll-Ups, Cumbersome Cascades Methods for Defining KPIs
Traditional Approach 1 Roll Up from Standardized Operational Metrics
Traditional Approach 2 Cascade Down from KPIs Aligned to Strategy
CXO: Reframe Strategic Questions Operational: Map Workflow Decision Points
Operational Metrics Misses external information required for strategy but not operations
Misses nonstrategic metrics required to manage operations
Identifies 18 to 20 CXO KPIs (estimated 50% overlap with operational metrics)
Takes too long and requires excessive harmonization of operations and/or systems
Requires ongoing modification of operational metrics as strategies change
Creates flexibility for CXO KPIs and stability for operational metrics
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Alternative Approach Differentiate CXO KPIs and Operational Metrics
CXO KPIs
Cascading down from strategy may be too narrow, as some operational metrics are needed that do not directly drive strategic outcomes. Progressive organizations recognize that neither approach is ideal and instead pursue differentiated approaches to CXO KPIs and operational metrics.
60
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
CXO KPi Identification
Overview A methodology to segment CXO and operational information needs to target the appropriate level of information depth, quality, and refresh rates for each EXECUTIVE TEACHING CXOs’ information needs differ from operational management needs. Executives require a broader perspective, need speed more than information quality, and have frequently changing needs; operational information needs are narrower, deeper, and less dynamic. Consequently, a single KPI framework and a single approach to information management will not meet the needs of both. Company Snapshot PepsiCo, Inc. Industry: Annual Revenue: Employees:
Food and Beverages US$60 B 294,000
PepsiCo offers the world’s largest portfolio of billion-dollar food and beverage brands, including 19 different product lines that generate more than $1 billion in annual retail sales each. PepsiCo’s main businesses—Quaker, Tropicana, Gatorade, Frito-Lay, and Pepsi Cola—also make hundreds of other enjoyable and wholesome foods and beverages that are respected household names throughout the world. With net revenues of approximately $60 billion, PepsiCo’s people are united by the company’s unique commitment to sustainable growth by investing in a healthier future for people and the planet, which the company believes also means a more successful future for PepsiCo.
Source: http://www.hoovers.com. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
61
As PepsiCo migrates from country-level to Europelevel management, its information and reporting needs change.
■■
Overcoming the Insight Deficit
Consolidated Information for a Consolidated Business PepsiCo Migration from Country-Level to European Sector Organization
The current reporting environment differs by country, making it difficult for Europe-level management to get the information they need.
Sector-Level Information Needs and Challenges
Objective
Italy
Spain
France
Russia
Identify the information needed to manage the sector. Sample Sector-Level Management Questions ■■
European Sector and Global Functions ■■ ■■ ■■ ■■
Procurement Finance Commercial Treasury
■■ ■■ ■■
■■
Planning R&D1 Food Technology1 HR1
■■ ■■
Country-Level Reporting Challenges ■■
■■
“Business and markets have changed, and following the recession historical information has lost its value, so we should forget about reporting and start thinking about information.” Austin Wildmore Enterprise Architect PepsiCo
■■
1
5,000+ applications and thousands of country-level reports Global chart of accounts but local definitions Up to eight weeks to aggregate metrics across Europe
Global functions.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Where do we get the best returns on trade promotion spend? Which distributors are most profitable for us? How can we consolidate transport costs across Europe?
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
62
The information needs of CXOs and operational managers are fundamentally different.
■■
■■
Executives require highlevel strategic information, and they tend to focus only on one or two objectives, which change frequently.
Diverse and Dynamic versus Focused and Stable
CXOs Role Characteristics
Operational managers require consistent, reliable, and deep drill downs and their metrics are relatively stable.
■■ ■■
■■
■■
Information Needs
CIO0361911SYN
■■
Sequential interest is focused on one or two priorities at any given time.
■■
Cross-cutting priorities affect multiple functions or BUs.
■■
Priorities change with strategy.
■■
Manage operational performance. Parallel interest have a portfolio of priorities. Focused responsibilities lie within one function or business unit. Priorities are stable.
Broad Perspective—Information from Diverse Internal and External Sources
Information from Limited Sources
Information Needs Change Frequently
Information Needs Are Stable
Where Necessary, Speed to Insight Over Quality and Depth
Quality and Depth Are Critical
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction
Business Unit and Functional Managers
Manage strategy, not operations.
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
63
IT should frontload its involvement in reporting by helping to reframe strategic questions to make the business outcomes measurable.
■■
■■
PepsiCo’s enterprise architects partner with IT business relationship managers and transformation directors in Finance to provide guidance during the annual strategic planning cycle. Their goal is to define make strategic objectives measurable.
Overcoming the Insight Deficit
Reframe Strategic Questions to Identify CXO KPIs Process for Identifying CXO KPIs Illustrative 1. Frontload Participation in Strategic Discussions ■■
■■
Enterprise architect supports information collection for annual refresh of three-year strategic plan. Proactive outreach anticipates information requests.
2. Reframe Strategic Questions to Make the Business Outcomes Measurable Two reframing questions ■■
■■
What business outcome are we trying to achieve? What can we measure to ensure we are making progress on that business outcome?
Original Strategic Goal Grow our business in Russia. What Business Outcome Are We Trying to Achieve? We want to become the market leader by exploiting production, distribution, and marketing synergies between our soda, snack, and juice businesses. What Can We Measure to Ensure We Are Making Progress on That Business Outcome? ■■
■■
3. Identify and Collect Information Needed ■■
“We are not trying to define strategy, but unless we can help make the strategy measurable we can’t provide the information the company’s leadership needs.”
4. Stop Measuring Objectives That Are Removed During the Strategy Refresh
■■
■■
Percentage of total sales consolidated with a single distributor in each region Percentage of total trade promotion spend directed to a single distributor in each region Percentage of revenue generated through a shared sales force Percentage of juice products carried by our soda distribution fleet Market share by region
Austin Wildmore Enterprise Architect PepsiCo CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
64
The CXO KPIs require an approach to information management that prioritizes speed and flexibility over quality and depth.
■■
Use the KPIs as a guide to avoid overinvesting in enterprise-level information standards or reports.
CXO KPIs Require a Different Approach to Information Management Implications of CXO KPIs for Information Management
Broad Perspective
1. Prioritize Sector-Level Information Standards of Enabling CXO KPIs Most information does not need to be harmonized across countries or functions. The CXO KPIs indicate where harmonization is needed. 2. Integrate Internal and External Information CXO KPIs often combine internal and external information.
Frequently Changing Needs
3. Retire Old Reports Reports fall out of use as strategies evolve so review and retire regularly. 4. Avoid Overinvesting to Meet Short-Term Needs CXO priorities change, so avoid overinvestment in gathering information that may only be needed for a year or so.
Speed and Brevity
5. Manual Effort Is Okay The need for quick information often requires the data to be pulled together manually. For short-term needs, automation may not be justified. 6. Drill-Downs Only by Exception XOs rarely need drill-downs—save time and effort by providing drillC downs on request rather than enabling them proactively.
“A lot of important information is now external. You need to approach the data with a specific question; you can’t try to aggregate it all.”
7. Accept Gaps in System Architecture Incomplete information is sufficient if it avoids delays. Set information management priorities around filling in the gaps.
Austin Wildmore Enterprise Architect PepsiCo CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
65
Highlight information gaps to drive to consensus on priorities for systems harmonization or integration.
■■
To establish priorities for change, PepsiCo enterprise architects use a model of core business components overlaid with system information to demonstrate to business partners the sources of gaps in CXO information needs.
66
Highlight (Don’t Hide) the Gaps Analysis of Sources of Information Gaps Illustrative
Russian Distributor Consolidation Report
Component Business Model—Russia To accurately measure sales through a common distributor and the reuse of distribution assets, we need to integrate operational systems for soda and snacks, and fill functionality gaps in our commercial and supply chain systems. Soda
Snack Foods
Juice
Dairy
Finance
Supply Chain
Commercial
Operations
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Vendor A
Vendor A
Vendor B
Vendor B
Vendor A
Vendor A
Vendor C
Vendor B
Gap
Vendor C
Gap
Unintegrated System
Flexible, Transparent Information Access
Vendor D
Unintegrated System
CXO KPI Identification
The IT Skills
Appendix
Identify operational metrics by identifying the information needed at key decision points in business processes.
MAP OPERATIONAL WORKFLOW TO FIND DECISION POINTS Decision Points in the Develop and Manage Sales Plans Process Illustrative
■■
■■
While CXO KPIs are identified by reframing strategic questions, PepsiCo takes a different approach to defining operational metrics used by managers below CXO level. These operational metrics are linked to decision points in business processes.
Process
Workflow
Decision Points
Information Needs
Manage Customer Sales
Manage Customers and Accounts
Perform Pre-Sales Activities
Manage Sales Orders
Perform Sales Calls
■■ ■■
Record Outcome of Sale
Decide on Discount Level
Decide Which Prospects to Call
■■
Close the Sale
Expected value Gap to quarterly goal Product availability
■■
■■
Average discount offered in quarter Previous prices paid by prospect
Source: “Process Classification FrameworkSM,” APQC, September 2009, http://www.apqc.org/knowledge-base/documents/apqc-process-classification-framework-pcf-cross-industryexcel-version-51-0. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
67
By focusing narrowly on CXO information needs, PepsiCo International significantly cut the time needed to gather new information and the number of legacy reports.
Overcoming the Insight Deficit
Better Decisions, Faster Time to Respond to Requests for New Reports
Reduction in Report Volume Illustrative
8 Weeks 1. Immediate elimination of 1,000+ legacy reports 2. Ongoing discipline to refresh or retire reports
< 1 Week
2009
2011(E)
2009
2010
2011
2012
“By asking the right questions, we can now track progress against our goals in Russia despite imperfect information. In the past we were swamped with metrics, few of which were useful. Now we only get those that matter.” Simon Black European Commercial Transformation Director PepsiCo
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
68
■■
The Information Architect role is critical to IT’s ability to tie business questions to the data knowledge workers need, to enable flexible analytical capabilities.
Information Architect
We estimate that demand for information architects will increase by 250% by 2015, the largest increase of any role in corporate IT.
■■
New-to-World Skills
Key Responsibilities ■■
■■
■■
■■ ■■
Conducts meetings with senior IT leadership and key business stakeholders across the organization to understand information needs
■■
■■
Analyzes the objectives of information consumers
Information analysis (structured and unstructured) Information integration
Other Important Skills
Visualizes and documents the inter-relations between information needs, technology, and design Devises metadata schemas for classifying content according to business needs Involved in prototyping and usability testing
■■
Information architecture development
■■
Information policy formation
■■
Information modelling
■■
Taxonomy creation
■■
Stakeholder management
Works with business stakeholders and IT project teams to define and promote enterprise information standards
Possible Backgrounds and Hiring Sources Most likely to have a background with a consultant or vendor specializing in information management or information-centric products Sourcing Difficulty
Seniority
Middle Management
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
69
A user-experience specialist collaborates with service managers and end users to understand and improve user experience and workflow for new and existing applications.
User-Experience Specialist Key Responsibilities
■■
■■
The role designs and configures user-centric interfaces for in-house and cloud applications.
New-to-World Skills
■■
Analyzes business and functional requirements
■■
Information visualization
■■
Creates user-centered design
■■
User-behavior analysis
■■
Improves the user experience
■■
Usability design
Visualizes and presents information in a userfriendly manner to end users
Other Important Skills
■■ ■■
This enables end users to access, visualize, and navigate information and analytics with ease. The skills required in this role match the primary skill sets of business analysts, data analysts, and software developers.
■■
Agile development
■■
Business domain analysis
■■
Functional requirements analysis
■■
Information architecture development
■■
Requirements management
■■
Usability design
Possible Backgrounds and Hiring Sources Likely to have a specialist background in behavioral science, graphic design, or product design; more likely to be found in a consultant or other specialized role than in a corporation Sourcing Difficulty
Seniority
Individual Contributor
For full details of this role, see the user-experience guru job descriptions on the CIO Executive Board website. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
70
Key Takeaways: Enable Information Usability
Make Usability a Core IT Competency: IT requires new skills and processes to improve the usability of information and analytic tools. Drive knowledge worker productivity by letting them choose the tools they find easiest to use, by offering greater transparency into quality levels, and by increasing their ability to navigate and analyze information from diverse internal and external sources. At Organizations with a High Insight IQ...
At Other Organizations...
1. Knowledge workers choose from a portfolio of tools that cover key capabilities.
One toolset is provided for all purposes.
2. Information presentation aggregation and filtering are top priorities.
Investment in advanced analytics is prioritized over presentation.
3. Knowledge workers can see all the information available to them and the relationships between that information.
There is no single list of all available information.
4. Information quality levels are transparent to knowledge workers.
Knowledge workers cannot ascertain the quality of information available to them.
5. KPIs are defined by reframing strategic questions.
KPIs are rolled up from operational metrics or cascaded down from strategy.
6. Speed to insight is prioritized when enabling CXO KPIs.
No KPIs are enabled until the underlying information is complete.
7. IT organizations invest in information architects and user experience specialists.
IT staff lack information architecture or usability skills.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Enable Information Usability
71
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
72
Knowledge workers are universally more confident about the quality of information from their own function than they are about information from other functions.
APPENDIX: Keeping Things Close to Home Average Knowledge Worker Confidence in Information Quality from Their Own Function and in Information from Other Functions 68%
High Confidence in Information from Own Function
62%
High Confidence in Information from Other Functions
56%
54% 48%
45%
44% 38%
35%
34% 27%
Legal
27%
Finance
34%
26%
R&D and Engineering
HR
Customer Service
Sales
Marketing
The IT Skills
Appendix
n = 4,941 knowledge workers.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
Enable Information Usability
73
Overcoming the Insight Deficit
Appendix: Definitions of information Quality Dimensions
Definition
Metrics (Illustrative)
Monitors (Illustrative)
Ease of Measurement
Measurement Comments
Timeliness
The relative ability of data to support a given process within the timetable required to perform the process
Days between product launch and support document publish date
Manual quarterly audit of lag time between new product launches and publication of supporting documentation
Medium
Like currency, this depends on data flow information to understand whether performance goals are met.
Validity
The degree of conformance of data to its domain values and business rules
Percentage of product codes in use corresponding to valid product codes
Automated cross-check of product codes in customer database against master product database to test code validity
Medium
It is usually fairly straightforward to create automated monitors to measure this characteristic.
Consistency
The conformance of the meaning of data with its data definition (also called Definition Conformance)
Match rate between internal and external product nomenclature
Automated monthly audit of match rate between all internal and external product names, performed by macro-comparing Excel fields for exact match
Medium
Not widely used; this often reveals a root cause of accuracy problems.
Completeness
The characteristic of having values for the data fields
Percentage of complete customer entries with complete e-mail information
Automated scan of customer database to measure percentage with existing, valid e-mail address entries
High
This is easy to measure and is recommended as a starting point to illustrate why monitoring matters.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
74
Appendix: Definitions of information Quality Dimensions (Continued) Definition
Metrics (Illustrative)
Monitors (Illustrative)
Ease of Measurement
Measurement Comments
Usability
The degree to which the information presentation is directly and efficiently usable for its purpose
Customer satisfaction with format and clarity of inventory data
Survey of business data customers to determine satisfaction with level of usability of data
Low
Measurement is usually done through some sort of customer satisfaction survey.
Precision
The characteristic of having the right degree of granularity in the data values
Percentage of deliveries scheduled as an hourly level of precision
Automated scan of delivery schedules to determine percentage with precision of one or two hours
Medium
This characteristic is not widely used within Intel, and problems usually surface in usability tests.
Currency
The lag time between when information is updated and when it is knowable in a redundant database
Lag time between inventory scan-in and updates to supply chain planning system
Automated measurement of time stamp differences between inventory scan-in and data refresh
Medium
Measuring this requires information about the flow of information—server, ETL logs, etc.
Accuracy
The degree to which the data accurately reflects the real-world object or event being described
Percentage of active employee records with accurate home address information
Manual audit conducted (by calling or e-mailing a sample of employees) to assess accuracy of home address on file
Low
This is the hardest characteristic to measure automatically; usually requires manual audits.
Nonduplication
The degree to which a one-to-one correlation exists between records in a dataset and real-world object or events
Percentage of products listed more than one time in product database
Automated cross-check of product databases to determine duplicate entries
High
This characteristic is very easy to measure and can uncover basic problems in business processes.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
75
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
76
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
77
Unquestioning faith in analysis can be as damaging as overreliance on intuition.
Overcoming the Insight Deficit
Why Balancing Analysis With Intuition Matters
Risks from Overreliance on Intuition ■■
High
■■
Reliance on Intuition
■■
Unable to account for complex environments Highly subject to personal biases Hard to replicate, scale, or use to foster buy-in
Risks from Overreliance on Analysis ■■
■■
■■
May hinge on incorrect assumptions or inaccurate data Unable to adapt to discontinuous change Fails to account for situational context by looking at the data too narrowly
Low Low
High Reliance on Analysis
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
78
Knowledge workers fall into three categories based on their relative reliance on analysis and intuition.
■■
■■
Informed Skeptics like to make biases and assumptions explicit and often question their own thinking; they listen to alternative points of view without defaulting to consensus-based decisions. The most common profile is the Unquestioning Empiricist, who tends to rely excessively on analysis, so in many organizations the greatest risk comes from too much analysis, not too little.
Visceral Decision Maker 19% of the sample Average Insight IQ1 = 45
High
■■ ■■ ■■
Seldom trusts analysis Has weak analytic skills Makes decisions unilaterally ■■
Reliance on Intuition
■■
The Value of Big Judgment
■■ ■■
Informed Skeptic 38% of sample Average Insight IQ1 = 60 Applies judgment to analysis Has strong analytic skills Listens to others and is comfortable with dissent
Visceral Decision Makers are more rare but represent almost one-fifth of knowledge workers.
■■
■■
■■ ■■
The profiles are determined using cluster analysis to identify common groups of analytic skills and behaviors.
Unquestioning Empiricist 43% of the sample Average Insight IQ1 = 48
Trusts analysis over judgment Supplements analytics with precedent Values consensus
Low Low
High Reliance on Analysis
n = 4,941 knowledge workers. 1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
79
Informed Skeptics are most common among the more senior ranks.
Overcoming the Insight Deficit
Learned Skepticism Knowledge Worker Profile Distribution Percentage of Knowledge Workers
■■
■■
While knowledge worker profiles vary substantially by seniority, there is little difference by organizational tenure, indicating that institutional knowledge is not a distinguishing factor.
Unquestioning Empiricists Visceral Decision Makers
22% 37%
43%
47%
Informed Skeptics
9%
Similarly, education and extent of cross-functional experience do not matter. 14% 17% 21%
69% 50% 41%
32%
Individual Contributors
Midlevel Managers
Senior Managers
Executive Leadership
As knowledge workers gain experience, they increasingly rely on judgment to supplement data when making decisions. n = 4,941 knowledge workers. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction
Informed Skepticism
Who Does What
Appendix
80
Unquestioning Empiricists represent at least one-quarter of knowledge workers in every function.
No Function Is Immune from Overreliance on Data Knowledge Worker Profile Distribution Percentage of Knowledge Workers
47%
57%
60%
Visceral Decision Makers are most heavily represented in Sales, Communications, and Customer Service.
30%
43%
30%
25%
19%
50%
51%
31%
12%
14% 18%
23%
16%
14%
18%
ns un ic G
en er al
M
m C om
58%
an ag em C en or t po ra te St ra te gy
47%
at io
39%
IT
37%
l
37%
Le ga
36%
ce
&D R
ar M
35%
na n
34%
29%
22%
g
48%
50%
39%
14%
18%
ke tin
41%
Fi
Unquestioning Empiricists most heavily reside in Marketing, R&D, and HR.
R
■■
25% 36%
H
■■
Informed Skeptics represent the majority in General Management and Corporate Strategy.
Sa Pr od le s uc t Pr ion oc / ur Op em er en s/ C us t to m er Se rv ic e
■■
Informed Skeptics
Visceral Decision Makers
Unquestioning Empiricists
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
81
Informed skepticism can be fostered through a combination of decision-making models, analytical skills training and coaching, and tools for collaborative decision making.
■■
■■
Some areas have little to do with IT, while others clearly require technology. Training and coaching represent “gray areas” of responsibility where CIOs should work with their business counterparts to determine ownership.
Overcoming the Insight Deficit
WHO DOES WHAT TO BUILD INFORMED SKEPTICS?
Activities and Responsibilities for Fostering Informed Skepticism
Clear Role for IT
Organizational “Gray Area”
Business Leadership Role (Unlikely to Involve IT)
Build Tools for Collaborative Decision Making
Provide Analytical Skills Training and Coaching
Define Decision-Making Models and Behaviors
Examples
Examples ■■
Build workflow tools that connect stakeholders involved in collaborative decision making.
■■
■■
Shift from up-front training on new tools to ongoing development of analytic skills. Hire and incentivize analytics staff on their effectiveness as coaches, not just analysts.
Examples ■■
■■
Introduce decision-making techniques that promote critical thinking and make biases and assumptions explicit. Embed analysis and collaborative decision making in competency models and performance criteria.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
82
Introduction
Informed Skepticism
Who Does What
Appendix
■■
Collaborative tools can enable more effective decision making.
IT Role: Tools for Collaborative Decision Making
Collaborative decision making, listening to others, and accepting dissent are characteristics of informed skepticism.
Essential Features of Collaborative Decision-Making Tools Wisdom of Crowds—Find and connect people across the organization to get broad input into ad hoc decisions.
Annotations and Note-Taking—Add context to data by annotating reports and integrate all comments in a note-taking platform.
Collaborative Workflow—Improve the quality of decisions by enabling group comments and conversations to reach consensus.
Integrated Workflow—Integrate information and collaborative processes into a single platform.
Information Freedom—Allow for a free flow of relevant and contextual information by moving data out of siloed e-mail platforms.
Decision Transparency—Develop a corporate decision repository to record discussions, information, and analyses around past decisions.
Sample Collaborative Decision-Making Tools
Do use collaborative decisionmaking tools to gather a wide range of opinions from experts across the organization.
EMC uses a collaboration platform from Jive to enable an internal idea-sharing and development community.
The UK Department for Work and Pensions uses collaborative tools from Spigit to create a virtual stockmarket where employees “invest” in ideas they believe have value.
DON’T use collaborative tools when speed is of paramount concern and one person’s expertise far exceeds that of others. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Macquarie University uses Yellowfin to improve information accessibility and research data visualization.
Source: MC Corporation, EMC|ONE: A Journey in Social Media, 2008; Dinham, Peter, “Yellowfin, Macquarie Uni Win Global BI Award” ITWire, 9 December 2011, http://www.itwire.com/ it-industry-news/market/51672-yellowfin-macquarie-uni-win-global-bi-award; Applications Executive Council, “Redefined Governance for Innovation and Improved Project Prioritization” Business 2.0, vol. 1, Corporate Executive Board, Washington DC, 2011, p. 83; Starner, Tom, “Profiting by Playing Games” Human Resource Executive Online, 27 October 2011, http://www.hreonline.com/HRE/story.jsp?storyId=533342586.
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
83
A knowledge worker’s employment history and educational background have less impact on their Insight IQ than providing access to training and analysts.
Overcoming the Insight Deficit
84
Shared Role: Analytical Skills Training and Coaching Maximum Impact on Insight IQ1 of Knowledge Worker Training, Support, and Background
24.7%
Factors that can be influenced in the near term have far greater impact than long-term factors such as educational background or employment history.
18.6%
2.4% 0.3% Training Available
Analyst Available to Provide Help
Has Held a Previous Role in Another Function
Science or Business Degree Program
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Effective information and analytics training is a strong driver of Insight IQ, but only 25% of knowledge workers experience effective training.
Only 25% Find Information and Analytics Training Effective Attendance and Effectiveness of Information and Analytics Training Percentage of Knowledge Workers
Total Knowledge Workers
100%
No Training Available
36%
Did Not Attend Training
15%
Attended Training but Didn’t Find It Effective
Attended Training and Found It Effective
Sixty-four percent of knowledge workers have access to training on information and analytics...
24%
25%
...but only 25% attend and find it effective.
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
85
Shift IT’s focus from up-front training on new tools to ongoing support for conducting analysis.
Overcoming the Insight Deficit
86
TRAIN TO ANALYZE Maximum Impact on Insight IQ1 of Different Types of Information and Analytics Training
Training on how to conduct analysis has more than twice the impact on Insight IQ of any other training subject.
28.7%
13.2%
13.7%
Tool Functionality
Definitions and Terminology
Conducting Analysis
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Information Training Portfolio
Overview A training portfolio that provides ongoing support to knowledge workers for conducting analysis and using analytics tools, reports, and dashboards Executive Teaching Training on new tools, without ongoing support and reinforcement, is ineffective as knowledge workers struggle to apply the training to their jobs. Training that focuses on the functionality of the tools and does not cover information sources or analytical techniques is also ineffective. Adopt a multipronged approach to design training offerings that support knowledge workers across the tool lifecycle and focus on conducting analysis. Company Snapshot Tiffany & Co. Industry: Jewelry Stores 2010 Revenue: $3.1 B 2010 Employees: 9,200
Tiffany & Co., founded in 1837, engages in the design, manufacture, and retail of fine jewelry. The company also offers brand merchandise, including timepieces and clocks, sterling silver merchandise, crystal, custom engraved stationery, and fashion accessories. The company sells its goods exclusively through about 230 Tiffany & Co. stores and boutiques worldwide, its website, business-to-business accounts, and catalogs.
Source: http://finance.yahoo.com; http://www.hoovers.com. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
87
IT struggles to meet rapidly increasing demand from knowledge workers for deeper support on using information and conducting analysis.
■■
■■
Tools Training Falls Short
IT Challenges
Most existing training programs are triggered by the launch of a new tool. To meet this increased demand for analyzing information, IT ends up providing ad hoc training and support.
Overcoming the Insight Deficit
Cost of Failure
Up-front, one-size-fits-all training doesn’t recognize varied knowledge worker needs and learning styles.
■■
■■
Solution
Limited training penetration due to unmet varied needs Lower tool adoption
Knowledge workers fail to capture value because they don’t understand the data or know how to conduct analysis.
■■
■■
Under-utilized or misused analytics capabilities Unexpected support costs
1. Make training an ongoing process with a portfolio of training programs mapped across a tool’s lifecycle. 2. Deliver customized training programs using a variety of channels that map to diverse knowledge worker needs and preferences. 3. Provide training on understanding information and conducting analysis, not just using tools.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
88
Provide a portfolio of training offerings that supports knowledge workers across the tool lifecycle.
■■
■■
Effective analytics training is designed as an ongoing process, not just an event tied to the launch of a new tool. Offering a range of training options and channels ensures that training maps to diverse knowledge worker needs and preferences.
INFORMATION TRAINING PORTFOLIO Illustrative Training Focus Areas Build Awareness of New Tool
Type of Training BI Day
■■
On-site training days on BI/ analytics tools, new subject matter, reports, dashboards, query techniques
Video Training
■■
Online training modules for all staff globally Includes training on data subjects as well as a series of training on new data marts
■■
Classroom Training
■■
■■
Train on New Tool Functionality
Provide Support for Analysis and Information
Provide Support for Tools
Weekly program delivered to business unit data stewards, culminating in a project Primarily for new analysts or refreshers
Analytic Team Portal
■■
For communication to users and troubleshooting
Ongoing Analyst Support
■■
A BI specialist for two months after launch of new tool or information Data stewards for intradepartmental training and support
■■
Introduce New Analysis and Information
Ongoing training is provided across the tool lifecycle, with greater focus on supporting knowledge workers in using information and analytic techniques.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
89
Focus training on how to conduct analysis and use information, not just tool functionality.
■■
■■
■■
■■
Train on Analysis, Not Just Tools Business Intelligence Day Agenda
List of Online Video Training Topics
Illustrative
Illustrative
Tiffany & Co. uses BI Days to educate groups of knowledge workers about new information, reports, dashboards, tools, and query techniques available to them.
Agenda
Business subject matter experts and vendors are also invited to lead these sessions; BI Days are publicized internally by e-mail and office message boards.
11:00 a.m. How to Analyze Reports and Dashboards Standard offerings and ad hoc analysis
In addition, the information management group creates online videos to offer knowledge workers easy, on-demand access to training on tools and information. Training videos are typically 15 to 20 minutes long and range from general, beginner level to specific, advanced level.
Overcoming the Insight Deficit
Learning About Our Information—Six Video Modules
9:00 a.m. BI: The Journey to Here
■■
10:00 a.m. How to Analyze and Use the New Customer Information Hands-on training demos and tips
Enterprise data, data marts, general information
■■
Sales data
■■
General Ledger data
■■
Merchandising data
………
Noon Lunch Learning to Use Our Analytic Tools— Seven Video Modules
1:00 p.m. Overview of Enterprise BI and Data Warehouse Includes BI strategy
(Specific to Information Types)
2:00 p.m. Using Queries to Help You Make Decisions Tips and tricks 3:00 p.m. BI Technologies and New Tools
■■
Analytics tools
■■
Reporting tools
■■
Dashboard tools
…….
Facilitators: Information management group
Audience: 800 ad hoc knowledge workers, other viewers of analytics, reports and dashboards
Attendees: 100 business unit management staff Location: Large venue, agenda tailored to regional needs
Location: Online
Training Focused on Conducting Analysis or Using Information CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
90
Hire and incentivize analytics staff on their effectiveness at coaching, not just analyzing.
HIRE QUANTS WHO CAN COACH Maximum Impact on Insight IQ1 of Hiring Criteria for Analyst Skills
Skills That Organizations Rate as Most Important When Hiring Analysts Percentage of Organizations
■■
■■
Although organizations rate business knowledge skills as nine times more important when hiring analysts than coaching skills, business knowledge skills provide no opportunity for impact on Insight IQ.
Business Knowledge Skills
14.8%
6.8%
See page 98 for interview questions to use to test for coaching skills when recruiting new employees and page 99 for an example of a training program to develop coaching skills.
Quantitative Skills
0.0% Coaching Skills
Technical Skills
45%
36%
Quantitative Skills
Business Knowledge Skills
Technical Skills
Coaching Skills
(11.1%)
9%
5%
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
91
Develop specialized information analysis and insight skills to bolster IT’s ability to support knowledge workers.
Role Profile: Information Insight Enablers Profile of the Information Insight Enabler Role Key Responsibilities
■■
■■
A subset of the skills required in this role are the same as those often found in data analysts; consequently, data analyst is the most common background for the information insight enabler role. For full details of this role, see the information insight enabler job description at www.cio.executiveboard. com/Members/ ResearchAndTools/Abstract. aspx?cid=100236298.
Overcoming the Insight Deficit
■■
New-to-World Skills
Understands the decision-making process and the workflows of business unit heads and service managers
■■
Identifies knowledge workers’ information needs
■■
Represents information in a user-friendly manner
■■
■■
■■
■■
Identifies trends and patterns; generates insight for business units and senior leadership Develops framework and process to analyze unstructured information Performs market and customer research and analysis and creates dashboards and scorecards Teaches knowledge workers about their information and analytic tools
■■ ■■ ■■ ■■
Market and competitor analysis Information insight generation Information visualization Unstructured information analysis
Other Important Skills ■■
Business domain analysis
■■
Coaching and Training Skills
■■
Data warehousing
■■
Functional requirements analysis
■■
Global teaming and remote collaboration
■■
Information taxonomy creation
■■
Information modeling
■■
Usability design
Possible Backgrounds and Hiring Sources Likely to have a background in market or financial research or in analytics or statistics “Right now, we have lots of people who can build reports. In the future, I will need people who help business partners find the right insight.”
Sourcing Difficulty
Seniority
Individual Contributor
CIO US Construction Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
92
Informed skepticism can be fostered through the introduction of decision models and behaviors that encourage inference and judgment.
■■
■■
Business Leadership Role: Decision-Making Models and Behaviors Maximum Impact on Insight IQ1 of Decision Models and Performance Criteria
These are areas where IT has little influence and therefore requires business leadership.
24.6%
Performance metrics are transparent and in regular use and are supplemented by formal measures of analytic ability.
17.4%
16.4% 15.0%
Decision Processes Are Formalized
Functional Performance Metrics Are Regularly Reported
Performance Review Is Based on Measurable Goals
Performance Criteria Explicitly Include Ability to Conduct Analysis
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board IT Practice, Insight IQ Diagnostic 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
93
An insight-driven culture is best realized when business and decisionmaking processes are formalized.
■■
■■
Not all decision-making models will promote informed skeptics (e.g., models such as Six Sigma and lean that focus on reducing variation and ambiguity are less likely to lead to informed skepticism). See Appendix on page 97 for more details about applying critical thinking skills to decision making.
Overcoming the Insight Deficit
Sample DECISION-MAKING Models Essential Features of Decision-Making Models That Promote Informed Skepticism
Formalized Methodology—Mitigate biases and foster scalability by introducing a systematic decision-making process.
Nonlinear Methodology—Adapt to change by incrementally following decision steps that can be tested and revised at short intervals.
Critical Thinking—Improve decision making by combining judgment and common sense with normative decision-making processes.
Incorporate the Unknown—Actively engage with uncertainty by separating the known from the unknown in the decision-making process.
Explicit Biases and Assumptions—Surface and adjust for biases and document assumptions so their validity can be tested.
Sample Decision-Making Models Design Thinking Integrates intuition, reason, and imagination into strategic decision making
The Mintzberg Model Incorporates feedback loops to design an iterative and dynamic decision-making model
Collaborative Decision Making Connects disparate teams of experts across the organization to collectively analyze problems
The Cynefin Framework Reflects the complexity of strategic decisions by acknowledging ambiguity
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
94
Informed skepticism can be fostered by setting the right expectations through performance criteria.
■■
Competency models and performance criteria should also describe the analytical, skeptical, and collaborative behaviors needed for informed skepticism.
Sample Performance Criteria Leadership Criteria
Proficiency Criteria
■■
■■
■■
■■
Uses analytics to identify trends and works to understand the driving factors
■■
Uses analytics in developing plans to achieve target business results
■■
Shows good judgment in interpreting the results of analytics
■■
Creates an analytic culture and drives usage by building analytics into day-to-day work Holds team accountable for using analytic tools and encourages full but skeptical usage Encourages a collaborative culture and is open to dissent
Collaborates with others in analysis and decision making
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
95
Overcoming the Insight Deficit
96
Key Takeaways: Support Big Judgment, Not JUST Big Data
Foster Informed Skeptics: If knowledge workers cannot apply judgment to analysis, greater information access may do more harm than good. Therefore, IT and business leaders must both promote a skeptical attitude toward information and analysis through decision models, performance criteria, tools, and training. At Organizations with a High Insight IQ...
At Other Organizations...
1. Most knowledge workers are Informed Skeptics.
Most knowledge workers are Visceral Decision Makers or Unquestioning Empiricists.
2. Decision making is supported by collaborative tools.
There are no tools for collaborative decision making.
3. Analytical skills are developed.
Analytical skills are hired.
4. Training is an ongoing process, so a portfolio of training offerings is available at all times.
Training is an event, tied to the launch of a tool.
5. Training focuses on using information and conducting analysis.
Training focuses on using the tool.
6. Ability to coach is a key job requirement for analysts.
Analysts are hired and evaluated solely for their analytical skills and domain knowledge.
7. Decision models and performance criteria promote informed skepticism.
Decision models and performance criteria are lacking or promote unquestioning reliance on data.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Foster critical thinking skills to increase informed skepticism.
Appendix: Critical Thinking Skills
Nine Gates for Critical Thinking1 Prepared for First Data Corporation
1
1
Clarity Could you illustrate what you mean? Could you give examples?
2
Accuracy How could you find out if that is true?
3
Precision Could you be more specific? Could you give more details?
4
Relevance How does the data relate to the problem?
5
Depth What are some of the difficulties that may arise that we may need to deal with?
6
Breadth Do we need to look at this problem from another perspective?
7
Logical Does the data make sense together?
8
Significance What is the most important problem to consider?
9
Fairness Do you have biases that will affect the outcome?
Copyright Complete Intelligence LLC.
Source: Complete Intelligence LLC. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
97
Identify candidates with coaching skills by asking them to give examples of the following behaviors and competencies.
Overcoming the Insight Deficit
Appendix: Identifying Coaching Skills During Interviews Please describe a time when you... 1. Assessed Development Progress: Reviewed where an employee stood against their development goals
8. Gave Feedback on Personality Strengths: Gave an employee feedback during a performance review about their personality strengths
2. Helped to Draft an Individual Development Plan: Helped an employee create a plan to meet their development goals
9. Gave Advice from Your Own Experience: Gave an employee advice based on your experience
3. Ensured That an Employee Had the Necessary Skills/Knowledge: Ensured that an employee had the required skills and knowledge before they needed it 4. Ensured That a Project Provided Learning: Ensured that each project or assignment was a learning experience
10. Helped an Employee Apply New Skills/ Knowledge: Helped an employee apply new skills or knowledge in their work 11. Helped an Employee Find Training: Help an employee find the training they need 12. Passed Along a Development Opportunity: Passed along relevant development opportunities to an employee
5. Explained Performance Evaluation Standards: Helped an employee understand their performance review standards 6. Gave Feedback on Performance Strengths: Gave an employee feedback during a performance review about their performance strengths 7. Gave Feedback on Performance Weaknesses: Gave an employee feedback during a performance review about their performance weaknesses
13. Passed Along Job Openings: Passed along information about internal job openings to an employee 14. Provided an Employee with Experiences That Develop: Helped an employee obtain the experiences at work that helped them develop over time 15. Taught a New Skill or Procedure: Taught an employee a new skill, concept, or process
Source: CLC Learning and Development research; CIO Executive Board research.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
98
Introduction
Informed Skepticism
Who Does What
Appendix
Use this process to help employees learn how to be effective coaches.
Appendix: A Formula for Coaching Success BCBSMA’s Learning Process for Coaching Align Coach Profile with Leadership Style
Reflect on Ideal Coach Attributes
■■
■■
■■
Which coach from my own experiences do I admire most? What did this coach do and say that was effective, and how did he or she interact with me? What kind of goals did this coach help me to achieve?
■■
■■
■■
What are my natural strengths as a leader based on my DISC1 Leadership Assessment? How does my idea of an ideal coach align with my leadership style? How can I reevaluate my definition of an ideal coach to better align with my leadership strengths?
Factor Team Dynamics and “Coachability” ■■
■■
■■
■■
How is my coaching and behavioral style similar and different from my direct reports? How can I adjust or tailor my style according to the behavioral styles and needs of my direct reports? What are potential areas of tension of my coaching style and the differing behavioral styles and needs of my direct reports? What might my direct reports need more or less from me?
1
CIO0361911SYN
■■
■■
■■
+
Focus on the “Right” Coaching Style CIO Executive Board IT Practice www.cio.executiveboard.com
Learn Coaching Fundamentals
How should my coaching activities differ based on performance-, development-, or career-based objectives? How can I better delegate to make projects a coaching opportunity? What role does my coaching style have in teambased problem solving?
Address Common Coaching Pitfalls
■■
■■
■■
What are the common challenges of coaching? How can I manage the emotions involved in delivering difficult feedback? What are strategies to address and resolve conflict within my team?
Focus on the Most Impactful Coaching Activities
=
Increased Coaching Effectiveness
DISC refers to dominance, influence, steadiness, and conscientiousness.
Learning and Development Roundtable, “Awareness-to-Action Coaching Toolkit” Profiles of Coaching Programs and Tactics, Corporate Executive Board, Washington DC, 2007, p.29
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
99
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
100
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
101
Collecting knowledge worker feedback on information sources and analytic tools has a 24.8% maximum impact on Insight IQ.
Overcoming the Insight Deficit
102
IT PAYS TO LISTEN Maximum Impact on Insight IQ1 of Knowledge Worker Feedback
24.8% 24.8%
16.5% 16.5%
Average for All Top 10% Drivers
Knowledge Worker Feedback on Information Sources and Analytics Tool Is Collected
n = 4,941 knowledge workers.
1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
IT needs to adopt a proactive collaborative and anthropological approach to identify opportunities to provide information and analytic tools.
Approaches to Identifying Opportunities to Provide Information and Analytic Tools
High
Progressive organizations engage knowledge workers earlier and more frequently throughout the project lifecycle using observation and experimentation.
1
Anthropologically Driven Need Identification
Value Associated with the Approach
■■
From LISTENING to OBSERVING
“Prosumer” Sounding Board
Use anthropological observation and interviewing techniques to discover information needs.
Observe progressive users to identify pain points and emerging solutions.
Agile Development Information Usability Surveys Requirements Gathering Low Feedback Collection
Proactive Observation Approach to Observation
1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
Observe and Question to Identify Opportunities
103
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
104
“Prosumer” Sounding Board
Overview A methodology to observe knowledge workers (both early adopters and mainstream users) to surface unarticulated and emerging information needs EXECUTIVE TEACHING Segment knowledge workers based on their information usage patterns and their stance toward technology adoption, then customize engagements to learn from each segment. Observe early adopters as an early-warning mechanism for emerging needs but balance this with input from the mainstream. Company Snapshot Ford Motor Company Industry: Auto Manufacturing 2010 Sales: 2010 Employees:
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
US$129.0 B 164,000
Ford Motor Company, a global automotive industry leader based in Dearborn, MI, manufactures and distributes automobiles across six continents. The company provides financial services throughout Ford Motor Credit Company.
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
Observe and Question to Identify Opportunities
105
Ford’s Digital Worker Office gathers user feedback on a regular basis to ensure continued relevance of collaboration services and communications.
■■
For additional background on the Digital Worker Office see the profile of Ford’s Digital Productivity Package in the IT Leader’s Guide to Collaboration at www.cio.executiveboard. com/Members/ ResearchAndTools/Abstract. aspx?cid=100159974.
Overcoming the Insight Deficit
DEVELOP User LISTENING POSTS to track evolving needs
Target Services Across Users
Key Challenges
■■
Solutions
■■
Engage Progressive Users
User needs vary within roles, making it difficult to scalably engage with the user base while ensuring services match with needs.
Allow users to self-segment based on preferred engagement and tech savviness to understand preferences for collaboration services, communications, and training.
■■
■■
IT misses the perspectives of the most tech-savvy users as they often bypass IT to deploy point solutions.
Ford creates a sounding board of progressive users to understand pain points and pressure-test IT plans.
“We want to make IT an enabler of user productivity, rather than play a ‘control and command’ role.” Scott Blow Manager, Digital Worker Office Ford Motor Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
106
IT allows users to select personas based on their needs and customizes engagement accordingly.
CREATE PERSONAS TO CUSTOMIZE ENGAGEMENT Personas
■■
IT creates personas based on user attitude toward technology and workflow needs to learn about challenges and target services and communications.
Engagement Strategy
1 Prosumers Willing to assume the risk of using new, uncertified, cutting-edge technologies with a light-touch support model
IT calls on prosumers to pilot new tools and suggest collaboration improvements.
Typical Demand Pattern—Employee-owned IT browsers
Early warning of emerging needs...
2 Senior Leaders ■■
■■
Users can select more than one of the five personas on the Digital Worker Office site. IT directs users to its site in training sessions and on the corporate intranet.
Company officers and others who require highend, auditable, discoverable collaboration services
IT dedicates a communications liaison and gathers recommendations on required collaboration tools and services.
Typical Demand Pattern—Support for off-site access to all systems
3 Information Creators Contribute information for decision making and need sophisticated knowledge management tools
IT knowledge management experts provide assistance with specialized needs such as domain search.
Typical Demand Pattern—Administrative responsibilities for team websites
DO provide users with incentives—such as early access to new tools—for providing feedback to IT.
DON’T invest equally in gathering input across all user segments; instead, favor scalable methods such as community sites over focus groups for users whose needs are easier to diagnose. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
4 Mobile Workers Knowledge workers who work primarily from outside the office
...is balanced with insight into mainstream demand.
IT connects users with each other to share problem-solving tips and device preferences.
Typical Demand Pattern—High-end access to information from outside the office
5 Standard Collaborators Consumers, reviewers, and processors of information
IT provides information about collaboration capability improvements.
Typical Demand Pattern—Occasional participation in meetings
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
Observe and Question to Identify Opportunities
107
Create a sounding board of progressive users to understand barriers to end-user productivity and identify innovative solutions.
Overcoming the Insight Deficit
108
CO-Opt CRITICS TO Identify Gaps Progressive User Sounding Board
Selection Criteria ■■
■■
The Digital Worker Office at Ford engages a subset of the prosumer segment to surface productivity pain points and drive innovation. To prevent technology evangelism, the facilitator guides users to diagnose the root cause of the underlying problem they are trying to solve with technology.
DO engage tech-savvy IT critics to identify capability gaps and push innovation.
DON’T overengineer conversations with users. Establish an informal dialogue to surface concerns about IT and understand new technologies users are experimenting with. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
■■
Independently solved technology constraints (e.g., obtaining access to enterprise data or e-mail on personal mobile device)
■■
Comfortable with voicing concerns or dissatisfaction with IT
■■
Understands the economics of niche versus enterprise solutions
■■
Demonstrates the drive to use new tools IT implemented
■■
Has a track record of recommending new technologies to IT
Progressive User Sounding Board Description The sounding board is a global, cross-functional group of 12 users that meets monthly. Role ■■ Articulate pain points. ■■ Provide early feedback on IT deployment plans. ■■ Participate in collaboration pilots.
Sample Discussion IT Facilitator
Sounding Board Participant
“Where do you see opportunities for improving productivity?”
“We need Device B to access enterprise e-mail while traveling.”
“Will Device A work just as well?”
“Device B has better functionality and interface.”
“What specific aspects are better?”
“Device B allows me to carry one device for corporate and personal use of e-mail and web.”
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
ANTHROPOLOGICALLY DRIVEN NEEDs IDENTIFICATION
1
OVERView The use of anthropological techniques to discover hidden information needs and opportunities for productivity improvement, by observing knowledge workers in their own environment and conducting open-ended interviews Executive Teaching Traditional approaches to requirements gathering fail when assessing knowledge workers’ information needs because the needs are fast-changing, diverse, and difficult to articulate. Alpha Company adopts an anthropological approach to opportunity identification. By observing knowledge workers in diverse settings and conducting open-ended interviews, Alpha Company gains a deeper understanding of knowledge worker needs and proactively identifies pain points in end-user workflows. Component Teachings ■■
■■
■■
Component 1: Direct Observation—Use anthropological observation techniques to surface unarticulated and context-dependent knowledge worker needs that are missed by more focused requirements-gathering tactics. Component 2: Interviews and Discourse Analysis—Conduct open-ended interviews with various stakeholders and analyze end-user stories beyond the immediate language to discover patterns of user experience. Component 3: Skills Implications for IT—Collectively, these observational and anthropological approaches require IT to develop a new set of anthropological skills.
Company Snapshot Alpha Company 2010 Sales: > US$10 B
1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Alpha Company is a leading technology provider.
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Observe and Question to Identify Opportunities
109
While knowledge workers often lack the ability to articulate their information needs, these needs also differ across individuals, organizational cultures, and collaborative practices.
Overcoming the Insight Deficit
CAN THE KNOWLEDGE WORKER SPEAK?
Needs Identification Scenarios
1
Identify Opportunities for Productivity Improvement
Surface Unarticulated Knowledge Workers’ Information Needs
Challenges
Information needs are context dependent and vary across individuals, organizational cultures, and collaborative practices.
Knowledge workers lack the ability or inclination to communicate their workflow or information needs.
Illustrative Examples
Sales Team Collaboration—Alpha Company found that the value of a sales team collaboration tool varied significantly across the year as collaboration declined in Q4.
Customer Relationship History—By observing sales teams in two regions, Alpha Company found that information on relationship history was far more valuable in one region than the other, due to cultural factors.
Mobile Sales Report Writing—Alpha Company found that trip reports written on the road were less detailed, took more time, and had more errors than those written in the office.
Information Source Diversity—Alpha Company discovered that sales teams with a certain profile of manager needed access to a broader variety of information sources when travelling.
“Anthropological research does not start from an ideal state to be achieved. Instead, it provides input to process redesign based on actual conditions on the ground.” Corporate Anthropologist Alpha Company 1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
110
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Traditional processcentric workflow mapping techniques fail to surface knowledge workers’ actual needs and behaviors.
■■
FROM WORKFLOW MAPPING TO ANTHROPOLOGY
Traditional Workflow Mapping Approach ■■
Most requirements-gathering techniques assume that there is only one right way to do something, and that this is the best solution for all knowledge workers in all contexts.
■■
■■
1
Alpha Company’s Anthropological Approach
Assumes there is only one best known method of working
■■
Uses standardized process discipline with pre-set rules for mapping workflows
■■
Solutions for knowledge worker needs aligned to stated needs and a predefined business case
■■
Assumes there may be several contextspecific and equally valid ways of working Uses observational techniques to identify roadblocks in day-to-day workflows and de-familiarization to mitigate biases Solutions based on unarticulated or unstated knowledge worker needs
When to Use an Anthropological Approach Workflows are intuitively executed and the result of many ad hoc workarounds.
Knowledge worker needs and workflows are context specific and difficult to articulate. “While end-user surveys show you what knowledge workers say they do, anthropological research tells you what knowledge workers actually do.”
The goal is to enable greater productivity and insight by understanding how knowledge workers work.
Corporate Anthropologist Alpha Company 1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Observe and Question to Identify Opportunities
111
Use anthropological observation and interviewing techniques to surface unarticulated and context-dependent knowledge worker needs.
■■
■■
1
Overview of Common Anthropological Techniques
Scenario: Help sales reps frame needs in terms of their underlying challenges in accessing information, rather than the technologies they need.
While end-user surveys follow a predetermined trajectory with little room for maneuvering, openended interviews follow a conversational model that allows for immediate feedback.
DO engage knowledge workers with an open mind, without a desired business goal, process, or workflow in mind.
DON’T assume that one process is superior or that knowledge workers use the same workflow in all situations. 1
CIO0361911SYN
112
OBSERVE AND ASK ASSUMPTION-FREE QUESTIONS
Gain insights into knowledge worker requirements through direct observation in different user settings, organizational contexts, and across points of time.
CIO Executive Board IT Practice www.cio.executiveboard.com
Overcoming the Insight Deficit
Anthropological Observation Techniques
Open-Ended Interviewing Techniques
1. Contextualized Observation Observe sales teams in their natural environments, such as customer visits or in sales meetings. Look across internal divisions and at organizational and cultural contexts.
1. A cknowledge Expertise Neither pretend to know more about the subject matter than the sales team does, nor buy in to easy answers, but always ask follow-up questions.
2. Observation Across Space and Time Observe sales teams across multiple spaces and points of time, looking for differences that may relate to location or time of day.
2. Reiterate and Qualify Repeat what is heard to clarify meaning, but also ask whether these are the right questions to ask.
3. S elf-Reporting Ask sales reps to use written or audio-visual diaries to record their workflow and log stage gates that cause bottlenecks or stalls.
3. Read the Silence Look for ideas and actions that are missing from the sales reps’ narrative, and identify variability and consistency across knowledge worker experiences.
4. Auto-Reporting Record the use of collaboration and analytic tools, using real-time monitoring to measure response times and gain deeper insights into sales reps workflows.
4. Ignore Nothing In the context of unarticulated needs, anything may turn out to be significant.
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Screen for attributes such as intellectual curiosity and critical thinking to develop a set of core anthropological skills within IT.
■■
■■
While trained anthropologists are best qualified to conduct anthropological research, IT organizations should first identify employees with intellectual curiosity and critical thinking skills. Use behavioral interviewing techniques to locate and identify intellectual curiosity and observational and problem solving skills during the interview process.
CAPTURE THE CURIOUS Overview of Anthropological Skills
Technical Anthropological Skills
Skills Implications of Anthropological Observation General Skills
Specific Skills
■■
Critical thinking and intellectual curiosity
■■
Critical thinking and intellectual curiosity
■■
Social agility and sensitivity
■■
Empathy and open-mindedness
■■
Contextual understanding
■■
Contextual understanding
■■
Primary research and fieldwork methodologies
■■
Discourse analysis and intertextuality
■■
Identifying patterns of behavior
■■
■■
■■
Understanding the evolutionary and material context of end-user needs
CIO0361911SYN
■■
Storyboarding ■■
Where to Find These Skills
CIO Executive Board IT Practice www.cio.executiveboard.com
Skill Implications of Open-Ended Interviews
Interviewing skills with high levels of reflexivity Gathering, integrating, synthesizing, and analyzing data Storyboarding
Where to Find These Skills
■■
Anthropology Graduates
■■
Anthropology Graduates
■■
Human Factors Engineers
■■
Market Researchers
■■
User Interface Designers
■■
Investigative Journalists
■■
Interaction Designers
■■
Information Scientists
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Observe and Question to Identify Opportunities
113
IT organizations can embed anthropological skills in several existing or emerging roles.
■■
THE ANTHROPOLOGIST INSIDE US
Criteria for Locating Anthropological Skills
The ongoing focus on information management and user-behavior analysis will see the emergence of IT roles that require capturing knowledge worker requirements for effective decision making.
■■
Has domain knowledge from alignment with business units
■■
Has time outside specific projects
■■
Has seniority, credibility, and business engagement skills
■■
Links back to project, service management, or architecture teams
Alpha Company’s Approach Dedicated Corporate Anthropologist Role ■■
■■
■■
For full descriptions of the responsibilities and skill requirements for these roles, please see the CIO Executive Board’s IT Job Descriptions Database. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
Alternative Approaches Embed Anthropological Skills in Existing Roles
For large organizations For organizations with people-intensive processes (e.g., Sales, Prof. Services, R&D) When project warrants dedicated resources
Introduction
Approaches to Opportunity Identification
Embed Anthropological Skills in New-to-World Roles
Business Analyst
Collaboration and Social Media Evangelist
Business Liaison
Information Insight Enabler
Service Manager
User-Experience Designer
Solution Architect
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
114
KEY TAKEAWAYS: Observe and Question to Identify Opportunities
Use Observations of Knowledge Workers in Addition to Observations from Knowledge Workers: Adopt an anthropological approach to identify opportunities and deliver analytic capabilities successfully. Identify and learn from early adopters, observe segments of knowledge workers across the organization and in diverse user settings, and conduct open-ended interviews to identify bottlenecks and pain points.
At Organizations with a High Insight IQ...
At Other Organizations...
1. IT is proactive in observing knowledge worker information needs.
IT listens to knowledge workers only when gathering requirements.
2. IT segments knowledge workers by their attitude toward technology and information usage patterns.
IT segments knowledge workers solely by role or seniority.
3. IT learns from early adopters about new analytic functionality.
IT restricts early adoption of new functionality.
4. IT observes knowledge workers in diverse user settings, locations, and points in time.
IT gathers requirements for information and analytic tools primarily through business process mapping.
5. IT conducts open-ended interviews to capture the voice of the knowledge worker.
IT communicates with knowledge workers through end-user surveys.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Observe and Question to Identify Opportunities
115
Overcoming the Insight Deficit
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
116
CIOs face a range of options for central or local management of information and for management within IT or outside.
Information Management—How to Organize, Where to Locate?
Centralized (Dedicated) Managing information centrally avoids duplication, promotes integration, and provides a single view of the enterprise.
Diffused (Integrated with Other Activities) Information should be managed close to the people who use it to allow for speed and flexibility. Key Questions
Business Ownership Only information producers and users understand its meaning and value and only they can improve quality.
Business/ Centralized
IT Ownership IT manages the information sources and can offer a neutral perspective.
IT/ Centralized
Business/ Diffused
1. What is the right degree of centralization for information management and analytics? 2. How should responsibility be divided between business areas and IT?
IT/ Diffused
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
Appendix
117
Foundational analytic and information management activities benefit from centralization and create sufficiently strong oversight to sustain decentralized information sources.
Overcoming the Insight Deficit
CENTRALIZE Management, NOT INFORMATION Maximum Impact on Insight IQ1 of Centralized Models for Information Management 13.0%
10.5%
10.4%
6.9%
6.7%
0.0% Centralized Analytics Team
Centralized Knowledge Worker Training
Centralized Information Quality
Centralized User Support for Analytic Tools
Centralized Information Architecture
Organizations with a high insight IQ centralize information management and support activities…
Single Unstructured Information Repository
0.0% Single Data Warehouse
…which enables them to keep the information itself decentralized.
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
118
There is no one ideal model for an analytics center of excellence as the optimal location depends on a number of organizational factors.
Where Should An Analytics COE Reside? Factors Determining Organizational Location of Analytics Center of Excellence (COE)
1. Location of Analytics Expertise
IT is the main source of analytics expertise.
Other functions/business units have analytics expertise.
2. Organizational Diversity
IT is the largest central function.
Other central functions are well resourced or influential.
3. Nature of Analytics Use
Information is used to produce the product.
Information is used to price or sell the product.
4. Information Management Focus
Information management is focused on integration and quality.
Information management is focused on knowledge worker capability.
I. Analytics Coe Within It
II. Paired IT and Business COEs
III. Analytics COE Outside IT
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
Appendix
119
While many activities migrate between IT and a COE outside IT, information stewardship remains primarily a business responsibility in all models while source system management always remains in IT.
Overcoming the Insight Deficit
DIVISION OF LABOR BETWEEN IT AND THE REST OF THE BUSINESS
I. A nalytics COE Within IT
Activity
II. P aired IT and Business COEs
III. Analytics COE Outside IT
Source Systems Management Information Architecture Data Warehousing Strategy and Prioritization External Information Acquisition Reporting Knowledge Worker Training Analytics Information Stewardship
Examples
IT–Owned Activities Owned Elsewhere in the Corporate Center
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
120
Introduction
Information Management COEs
Skills Implications
Information management and analytics activities are centralized within IT.
I. Analytics COE WITHIN IT Information Management and Analytics Organization Structure
A. E ach major business unit is represented on the Data Governance Council; the council was set up as a business council, and IT’s participation on the council is purely as one of the represented business units.
CIO
A Data Governance Council
B. EDW and BI user support team addresses questions related to both business intelligence tools and content.
Enterprise Business Intelligence (EBI)1
EDW Content and Data Governance
C. B I solutions team develops reports and tools for users; as there is a strong focus on user self-service, the team typically builds templates for users to adapt rather than providing ongoing maintenance and support.
EDW Content and Architecture
Business Applications
EBI Asia Support and Development
Business Intelligence C
Asia Business Intelligence
BI Solutions
D
D. A pplication-specific and factory-related reporting is run outside the EBI group to cater to special requirements of these groups.
Factory Systems
PMO and Data Modeling
BI Strategy and Architecture
EDW and BI User Support
Advanced Analytics
Asia EDW Content
B
Real-Time and ApplicationSpecific Reporting
D
Real-Time and ApplicationSpecific Reporting
Reports to IT 1
Quality and engineering applications support teams are not shown on this chart but they also report into the executive director, Enterprise Business Intelligence.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
Appendix
121
Information management and analytics responsibilities are divided with a business COE acting as an interface between IT and the rest of business.
Information Management and Analytics Organization Structure
Analytics Resource Center (ARC)
B. ARC is supported by business subject matter experts and change agents from the lines of business. C. D ata Analysis COE reports to IT and derives data requirements and data delivery priorities from ARC’s prioritization and governance processes.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Data Analysis COE
BI Delivery and Adoption Team
Program Executive/ Group Analytics Lead
Data/Information Analysis Lead ARC Prioritization and Data Governance Processes
Analytics Advisory Board Program Management Office
B
B
Change Agents
Business SMEs
–– Support analysts across the organization in nonactuarial analytics, and –– Lead data governance efforts across the organization.
122
II. PAIRED IT AND BUSINESS COEs
A. K ey mandates for the Analytics Resource Center (ARC): –– Represent business partners in the development of BI and provide testing and training,
Overcoming the Insight Deficit
A
Analytics
Data Analysis
Business Intelligence
Data Governance Strategy
Reporting
Policy Committee
Data Delivery
Data Integration
C
Data Analytics Data Analysis
Process Team
Support
Aligned by BUs
Technical Lead
Metadata/ Governance
Aligned by Subject Areas
Business-Led Function That Reports to the Actuarial Group Introduction
Information Management COEs
Reports to IT
Skills Implications
Information management and analytics activities are centralized under a corporate function, shifting most of traditional information management activities out of IT.
■■
■■
■■
III. Analytics COE Outside IT Information Management and Analytics Organization Structure
Executive VP and Strategy Officer Enterprise Information Committee
At point A, touchpoints with IT are maintained through representation in committees and through regular meetings with the IT staff and CIO. At point B, a few technical information management– related roles such as data integration remain outside of the COE; information architecture responsibilities are represented on both sides. BCBSNC’s COE is being built iteratively, first by including the transactional activities (e.g., data sourcing, reporting, training) and some critical pure analytics functions (e.g., group analytics, care management).
Information Management and Analytic Services Analytics Center of Excellence
Analytic Services and Solutions
Analytic Delivery
Analytic and Information Management Advisory Committee
Continuous Improvement
A B
Informatics Solutions
Analytic Services
Solutions Portfolio Analyst Support Technology Architect
Informatics Solutions Analysts
Informatics Analysts
Portfolio and Process Analyst(s)
B
Information Strategy and Architecture Data Integration and Production Support
Informatics Solutions Developers
Corporate Function That Reports to the Strategy Group CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction
Information Management COEs
Skills Implications
CIO0361911SYN
Appendix
123
An analytics COE within IT requires new skills such as information visualization, analytics service management, and knowledge worker training and coaching.
Overcoming the Insight Deficit
Skill IMPLICATIONS OF Each COE MODEL I. A nalytics COE Within IT
Skills
II. P aired IT and Business COEs
III. Analytics COE Outside IT
Database Design and Mgmt. Info. Systems Design Info. Architecture Info. Integration Business Analysis Info. Visualization Analytic Service Mgmt. Strategy and Portfolio Mgmt. Info. Policy Formulation Analysis/Insight Generation Training and Coaching
IT Skills That Rise in Importance Business Skills That Rise in Importance
For a description of the key information management and analytics skills, please see page 125. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
124
Implementation Guide: SKILLS Descriptions
Analytics Service Management
The skills required manage analytics as an end-to-end service from service strategy to service delivery and improvement
Database Design
The skills required to produce a detailed data model for a database
Database Management
The skills required to create, access, and maintain a database
Information System Design
The skills required to analyze and design information systems, concentrating on entities and their attributes and interrelationships; also includes data modeling for individual databases and the corporate data model, as well as coordinating the definition of data across multiple distinct databases
Information Architecture and Development
The skills required to define how information and information systems support the objectives of the business
Information Integration
The ability to integrate information and content from multiple sources and channels
Business Analysis
The skills required to identify business needs and determine solutions to business problems
Information Visualization
The ability to depict information in intuitive and insightful ways
Information Policy Formulation
The skills required to establish guidelines for information dissemination and usage
Information Analysis and Insight Generation
The ability to understand the meaning of structured and unstructured business information and generate insight from it
Note: For detailed analysis on the emerging skill gaps and role changes for IT, please ask the CIO Executive Board staff or your account manager for a copy of our recent study on The IT Talent Implications of the Future of Corporate IT. You can also download this study from www.cio.executiveboard.com. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction
Information Management COEs
Skills Implications
CIO0361911SYN
Appendix
125
List of Questions Asked in the Insight IQ Diagnostic
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
126
I. Information Management Structures and Processes 1. Has your organization introduced new or enhanced information management or analytics capabilities in the past 18 months? 2. How are individuals/teams dedicated to analytics structured within the organization? –– Centralized companywide –– Partially centralized by business unit –– Partially centralized by function –– Decentralized—most teams/subfunctions have their own analytic experts 3. If your organization has a central analytics group or Center of Excellence, where does it reside? –– IT –– Business—Shared Services –– Business—office of CEO or President –– Marketing –– Finance –– Other 4. Which of the following capabilities are in place to improve decision making within your organization? –– Automated decisions –– Collective intelligence tools –– Decision making team structure based around diversity and independence –– Formal decision frameworks or guidelines –– Other 5. Is information primarily pushed to you or pulled by you? 6. Do rules exist around which information can be shared across the organization? 7. Has your organization undertaken the following in the past 12 months. –– Adding new information sources to the data warehouse –– Enhancing existing data source feeds –– Updating or modifying business rules used in standard reports –– None 8. Which best describes your organization’s approach to managing information and analytics? –– Information and analytics are managed completely
separately. –– Information and analytics are managed separately, but the owners of each are aware of the other’s processes and accommodate them as appropriate. –– Processes for managing information and analytics are partially integrated. –– Processes for managing information and analytics are fully integrated throughout the lifecycle. 9. Which best describes the structure of your organization’s information management group? –– Formal—within IT (with dedicated staff) –– Informal—within IT (i.e., a couple of individuals are responsible but is not their full-time job) –– Formal—within Shared Services –– Informal—within Shared Services –– Formal—part of broader Analytics Group –– Informal—part of broader Analytics Group –– Formal—within individual BUs/functions –– Informal—within individual BUs/functions –– Exists, but is integrated into day-to-day tasks of broader IT –– Doesn’t exist 10. Who owns each of the following information management responsibilities? –– Information architecture –– Information lifecycle management –– Information quality and standards –– Knowledge and content management –– Analytics tools –– Enterprise data warehouse 11. If you have an Information Management group, what is the perceived clarity of its roles and responsibilities? 12. Indicate the degree to which the following are centralized. –– Data warehouse –– Analytical tools –– Unstructured information repositories –– Knowledge worker training in information use and analytics –– User support for analytical tools
–– Information ownership –– Information architecture –– Information quality 13. How are individuals/teams dedicated to analytics structured within the organization? 14. If your organization has a central analytics group or Center of Excellence, what are its responsibilities? –– Coordinate analytics initiatives. –– Share analytics best practices and provide training. –– Provide analytical expertise/capabilities. –– Deploy analysts where needed. –– Develop analytics strategic plan for the organization/ functions. –– Provide guidance/rollout of analytical tools/technologies. –– N/A 15. If your organization has a central analytics group or Center of Excellence, where does it reside? –– IT—Reports directly to CIO –– IT—Applications –– IT—Infrastructure/Operations –– IT—Information Management –– IT—Other, please indicate –– Business—Shared Services –– Business—Office of CEO or President –– Marketing –– Finance –– Other, please indicate –– N/A 16. Does your organization use an external service provider for analytics (excluding cloud providers)? 17. If you do use an external service provider for analytics, which of the following is the external service provider responsible for? –– Information architecture –– Information lifecycle management –– Information quality and standards –– Knowledge and content management –– Analytic tools and capabilities –– Enterprise data warehouse
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
127
I. Information Management Structures and Processes (Continued)
––
20. Do you use cloud applications for analyzing information? 21. What capabilities are in place at your organization to ease the capture of unstructured information in shared systems? . –– Automated processes (e.g., automated tagging, document uploads) –– Rules and guidelines on what information to capture and tag –– Rules and guidelines on how to capture information and tag –– Dedicated team or individual responsible for capturing, tagging and transferring information into shared systems –– Other –– None 22. For which of the following does your organization have defined processes in place? –– Create/Acquire –– Store/Share –– Find/Use –– Archive/Retain –– Dispose –– Manage/Control –– None 23. How do you define metadata in your data warehouse? –– For all information types –– For information that is shared between functions –– For information that is shared between business units –– For information that is deemed critical to the business –– For information that is used most frequently –– For information that is stored in multiple systems or repositories –– For information where there is frequent confusion regarding the definition –– For unstructured information –– For no types of information 24. Which best summarizes your current level of metadata management maturity? –– Incomplete business and technical metadata are
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
128
II. Training and Support
18. Do you use cloud applications for generating information? 19. Do you use cloud applications for storing information?
Overcoming the Insight Deficit
–– –– ––
maintained in pockets of solutions or documents, typically at a project level. Consolidation of some business definitions or technical information occurs, but governance processes are immature. Business and IT accountabilities and processes are starting to be formed to manage repositories of metadata. Business and IT are utilizing common metadata across departments and projects to improve understanding. Formal processes and accountabilities are in place for managing official repositories of metadata utilized for guiding information use.
25. Do rules exist around types of information that could be shared between levels and/or different parts of the organization? 26. How would you assess your organization’s current level of proficiency in the following Enterprise Architecture disciplines? –– Business Architecture –– Information Architecture –– Applications Architecture –– Technical Architecture
1. Is there an analyst available to you to help you find information or conduct analysis? –– There is an analyst embedded directly in my team. –– There is a team of analysts in my business unit or function. –– There is a companywide central team of analysts. –– There are informal analytical experts on my team. –– There is no one available to me for help in finding information or conducting analysis. 2. Does your organization provide training on finding information or conducting analysis? 3. If you attended training provided by your organization on finding information or conducting analysis, how effective or ineffective was that training in the following areas? –– Finding information I need –– Information definitions and terminology –– Conducting analysis –– Communicating and displaying information –– Using the functionality of the tools provided –– Where/how to get help 4. How formalized is the process for making decisions in your organization? –– Non-existent: No decision making processes exist. –– Ad hoc: Individuals and/or teams have processes that are followed irregularly. –– Undocumented: Individuals and/or teams follow processes that are not standardized nor documented. –– Defined: Processes used across business units and/or functions are documented but not enforced. –– Managed: Processes are documented and enforced. –– Optimized: Processes are documented, enforced, and measured to drive improvement. 5. Which of the following are in place to improve decision making within your organization? –– Automated decisions –– Collective intelligence tools –– Decision making team structure based around diversity and independence –– Formal decision frameworks or guidelines –– Other –– None
III. Knowledge Worker Attitudes and Incentives 1. Are you evaluated in your performance review on your ability to conduct analysis effectively? 2. To what extent do you agree with the following: I enjoy conducting analysis? 3. To what extent do you agree with the following: If I conduct analysis and disagree with the findings I will rely on my own judgment instead? 4. To what extent do you agree with the following: I have enough time to conduct analysis that is valuable to me in making decisions in my daily work? 5. To what extent do you agree with the following: Expertise in analytics increases my ability to advance in my desired career path? 6. To what extent do you agree with the following: I feel very comfortable conducting analysis and am able to meet all my information and analytical needs without external help? 7. When making important decisions in your organization, how critical are each of the following considerations to you, your direct manager, and your organization’s leadership? –– Information and Analytics –– Intuition –– Precedent –– External Advice 8. How effective is your direct manager in using analytics in carrying out his/her job responsibilities? 9. To what extent do you agree with the following: My performance review is based largely on metrics and achievement of measurable goals and objectives? 10. To what extent do you agree with the following: Performance metrics and progress against my business unit/function’s goals and objectives are reported to me on a regular basis?
11. How significant a contributor to your personal analysis and decision making are each of the following types of information? –– Centrally provided information available to and used across the organization –– Centrally provided information available to and used across my business unit/function –– Information generated by and managed by my subfunction or team –– Information I personally obtain through external sources 12. Which characteristics do you value most when using information and analytics in your daily work? –– Information Accuracy –– Information Timeliness –– Information Consistency –– Information Relevance 13. How important are the following capabilities when you make decisions? –– Visualization—The graphical or visual presentation of information (e.g., through static or interactive graphs, charts, maps, images) to make it easier for the users to understand the information contents and attributes –– Analysis—The process of examining information with the purpose of drawing conclusions about that information –– Filtering—The process by which redundant or unwanted information from an information stream is removed and only information that meets specific criteria (e.g., time, value, user groups) is displayed or presented –– Aggregation—The process by which information is gathered and expressed in a summary form, typically combining two or more information attributes or occurrences
15. How important is information and/or analytics in enabling you to do the following? –– Make Better Decisions –– Make Faster Decisions –– Improve Your Job Performance 16. How frequently do you regret decisions you’ve made where information/analytics contributed to the decision? 17. To what extent do you agree with the following: All of the information I need to do my job is available to me? 18. To what extent do you agree with the following: In my daily work, I rely heavily on the results of analysis (charts, graphics, reports, dashboards, etc.) to support my business decisions, processes and deliverables? 19. To what extent do you agree with the following: The reports and dashboards that are pushed to me are very valuable and I read, analyze and use them extensively? 20. To what extent do you agree with the following: The reports and dashboards that I pulled are very valuable and I read, analyze and use them extensively? 21. If information you would like to have as an input to making a decision is not available in a standard report or dashboard, how willing are you to go look for the information?
14. How effective a contributor is information and/or analytics to your organization’s ability to do the following activities? –– Product or service development –– Improving product or service profitability –– Evaluating and prioritizing investment proposals –– Developing a corporate or business unit strategy –– New market identification and market strategy development –– Identifying and targeting customer segments –– Forecasting demand for products, services, or resources
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
129
IV. Investment and Resource Level
V. Quality of Information
1. What is your organization’s total budget for the following projects/initiatives/capabilities in 2011? Include all spend both inside and outside of IT. –– Business Intelligence –– Information Management –– Collaboration
1. To what extent do you agree with the following: “The information from corporate sources is in a usable format and does not require modification or manipulation?”
2. At which groups are your analytics investments and initiatives targeted? 3. How many staff are part of the information management group? 4. Rank the following four capabilities in terms of effort and investment. –– Visualization –– Analysis –– Filtering –– Aggregation 5. How are analytics projects and initiatives approved? –– Very formal business case review –– Somewhat formal business case review –– No formal business case review –– Piggyback on other IT/business initiatives –– Other 6. Which of the following investment approval processes do you use for analytics investments and initiatives? –– A business-enabling capital investment –– An IT infrastructure investment –– An R&D investment 7. How are information management projects and initiatives funded? –– As a business-enabling capital investment –– As an IT infrastructure investment –– As an R&D investment 8. How are information management projects and initiatives approved? –– Very formal business case review –– Somewhat formal business case review –– No formal business case review –– Piggyback on other IT/business initiatives –– Other, please indicate
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
2. To what extent do you agree with the following: “I have a good understanding of the definitions of the information from corporate sources?” 3. To what extent do you agree with the following statement: “There is a single version of the truth across information from all corporate sources?” 4. To what extent do you agree with the following: “The information from corporate sources is directly relevant to business decisions, processes, and deliverables in my daily work?” 5. To what extent do you agree with the following: “The information from corporate sources is always timely and up-to-date enough to be useful in my daily work?” 6. To what extent do you agree with the following: “The information from corporate sources is always precise and accurate enough to be useful in my daily work?” 7. What level of confidence do you have in the quality (cleanliness and accuracy) of information from each of the following? –– Human Resources –– Finance –– Legal and Compliance –– Marketing –– Sales –– Customer Service –– Supply Chain and Operations –– Research and Development 8. For functions where you have low or no confidence in the quality of their information, what are the reasons behind your lack of confidence? –– Lack of transparency in how information was collected –– Lack of transparency in how information was stored –– Lack of transparency in how information was analyzed –– Multiple versions of the same information exist –– Lack of trust in systems –– Lack of common vocabulary/standard definitions
Overcoming the Insight Deficit
130
–– There are no means to assess or verify information quality –– Other 9. For which of the following has your organization established standard definitions? –– Information that is shared between functions –– Information that is shared between business units –– Information that is deemed critical to the business –– Information that is used most frequently –– Information that is stored in multiple systems or repositories –– Information where there is frequent confusion regarding the definition 10. Is each of the following information types defined in a data dictionary? 11. What steps does IT take to remedy information quality issues (such as duplicates, errors, outliers, contradictions, inconsistencies, missing values) in relation to the information that is entered or maintained in the data warehouse? –– There is an information quality framework that establishes quality standards and guidelines for information to be added or kept in the data warehouse or data marts. –– The information is fixed in the source systems before entering it into the data warehouse or data marts. –– The information is fixed both in the source systems and during information movement. –– The information is fixed in the data warehouse or data mart with the fixes fed back to the source systems. –– The quality of information in our data warehouse is reviewed or audited on an ongoing basis. –– The quality of information in our data warehouse or data mart is reviewed or improved as and when business users raise quality concerns. –– The information quality process is integrated into the ETL process to maintain information quality thresholds. 12. To what degree are information quality governance and standards automated through technology? 13. Which of the following describes the information quality standards within your data warehouse or data marts? –– All information has the same standards for information quality.
V. Quality of Information (Continued) –– Information for certain functions (e.g., Finance, HR, supply chain) has higher standards for information quality. –– Information that is used more frequently has higher standards for information quality. –– Information that has been deemed to be more valuable to the business has higher standards for information quality. –– Information that will be used for certain types of analysis has higher standards for information quality. –– Information that will be used by more senior or executive staff has higher standards for information quality. –– Information from certain corporate systems has higher standards for information quality. 14. Which of the following describes the information timeliness standards within your data warehouse or data marts? . –– All information has the same standards for information quality. –– Information for certain functions (e.g., Finance, HR, supply chain) has higher standards for information quality. –– Information that is used more frequently has higher standards for information quality. –– Information that has been deemed to be more valuable to the business has higher standards for information quality. –– Information that will be used for certain types of analysis has higher standards for information quality. –– Information that will be used by more senior or executive staff has higher standards for information quality. –– Information from certain corporate systems has higher standards for information quality.
VI. Information Usage Patterns 16. Who in your organization is primarily responsible for enforcing information quality standards and policies? –– Business leaders –– Subject matter experts in the business –– IT –– Information Management group –– Analytics group –– Shared services group –– No one 17. On which of the following characteristics do you tag information in your data warehouse or data marts? –– Individual author of the information –– Function that created the information –– Date and time the information was created –– Cleanliness and timeliness standards for the information –– Relevant governance processes –– Frequency information is used –– Purpose of the information –– Types of analysis information is used for –– Types of decisions information is used to make –– Standards reports or dashboards information is used in –– Other –– None
1. How frequently do you use each of the following? –– Workforce Information –– Financial Information –– Risk Information –– Customer Information –– Product, Materials, and Suppliers Information –– Research, Product Development and Testing Information –– Supply Chain Information –– Demand Information –– Web Information –– Consumer and Marketing Information –– Customer Service Information –– Sales Operations Information –– Economic and Industry Information –– Financial Market and Trading Information –– Assets and Maintenance Information –– Business Process Efficiency, Service Levels, and Quality Information 2. Which of information types are most important to you in your daily work? 3. Is your team or function responsible for owning and/ or generating a significant proportion of the following information types? 4. How frequently do you consult each of the following in your day-to-day work? –– Operational Structured Information –– Analytical Information –– Unstructured Information –– Undocumented information from colleagues or experts –– External company-endorsed sources –– External non-company endorsed sources
15. Who in your organization is primarily responsible for defining information quality standards and policies? –– Business leaders –– Subject matter experts in the business –– IT –– Information Management group –– Analytics group –– Shared services group –– No one
5. How many different sources of information/analysis do you use on a weekly or more frequent basis? –– Corporate operational systems –– Corporate unstructured information systems and document repositories, excluding e-mail –– Reports, dashboards, data/information warehouses, or data marts On average, how much of your time at work do you spend finding information or conducting analysis?
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
131
VI. Information Usage Patterns (Continued) 6. On average, how much of your time at work do you spend finding information or conducting analysis? 7. Of the time you spend finding information and conducting analysis, what percentage is spent finding information? 8. How frequently do you undertake each of the following types? –– Descriptive Analytics—The process of using analytics to describe what happened in the business (using reports and dashboards) –– Predictive Analytics—The process of using analytics to predict what will happen in the business (using data mining, pattern recognition and alerts, Monte-Carlo simulation, forecasting, predictive modeling) 9. How much time do you spend obtaining information or conducting analysis that does not ultimately lead to a decision or productively inform your or your colleagues’ work? 10. To what extent do you agree with the following: When I need specific information, I know where to find it? 11. To what extent do you agree with the following: In instances where someone in my organization has conducted similar analysis before, I am able to find documentation, lessons learned, or notes on their process and experiences? 12. To what extent do you agree with the following: For projects/tasks where someone in my organization has worked on something similar before, I am able to find documentation, lessons learned, or notes on their process and experiences? 13. Which of the following formats/channels do you use to access each information type? –– PC-based web browser –– Mobile devices –– E-mail –– Online system –– Paper 14. If you access information from mobile devices, how do you rate their value and functionality relative to when accessing them on a standard desktop or laptop?
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
132
VII. Staff Knowledge and Competencies –– Significantly higher value due to speed/convenience of mobile device accessibility –– Somewhat higher value –– Same value –– Lower value, as some information is missing –– Significantly lower value, as key information is missing –– N/A—I do not have mobile access. 15. Do you analyze information obtained from any of the following sources? –– Smart Networks and Sensors –– Radio-Frequency Identification –– Global Positioning System –– Collaboration Tools –– External Social Media 16. Do you provide information to corporate systems for others’ consumption (e.g., inputting information into a CRM system or sharing information/knowledge within a collaboration system or document repository)? 17. Do you have the opportunity to provide feedback on information sources and business intelligence/analytics tools for improvement or functionality enhancements? 18. How does your organization determine end-users’ needs for information with regard to tools, capabilities, quality, etc.? –– Link information needs to strategic decisions. –– Derive information needs from strategic goals or targets/ KPIs. –– Define information needs based on workflow processes. –– Keep up to date of market/industry/functional trends. –– Directly ask knowledge workers via survey. –– Directly ask knowledge workers via focus groups. –– Track information requests from knowledge workers to understand trends in information needs. –– Take in feedback from knowledge workers in terms of their satisfaction with information (to inform future needs).
1. Is conducting analysis explicitly mentioned in your job description? 2. Which of the following skills do you most value in staff responsible for conducting and providing analysis to you and your team? –– Business knowledge and understanding –– Quantitative skills –– Technical skills –– Relationship and advisory skills –– Training and coaching skills 3. How frequently do you interact with staff from each of the following functions for work-related purposes? –– Human Resources –– Finance –– Legal and Compliance –– Marketing –– Sales –– Customer Service –– Supply Chain and Operations –– Research and Development 4. To what degree are your decisions made independently? –– I make the majority of decisions completely independently. –– I take my peers’ perspectives into account but ultimately make decisions independently. –– I make decisions in conjunction with peers to incorporate the group’s perspectives. –– Decisions are largely dictated by my manager or other leaders. 5. Which of the following best describes the methods you use to conduct analysis? –– I develop specific theories that I then test using analysis. –– I have some theories in mind that I test using analysis, but also manipulate the information in other ways to see if interesting trends or insights emerge. –– I manipulate the information without a clear idea of what I expect to find, and see if interesting trends or insights emerge.
VIII. Information Strategy and Governance 1. Does your organization have a data steward role? 2. If you responded “Yes, formal data steward role,” how many data stewards does your organization have? 3. Indicate if the following are available to knowledge workers through self service. –– Data definition –– Name of the data steward –– Source of the data –– Details of transformations performed on the data –– None 4. To what degree do information security and access policies inhibit your ability to obtain information that is valuable/ important to your analysis or decision making? 5. How mature are business processes within your function/ business unit? –– Non-existent: No business processes exist. –– Ad hoc: Individuals and/or teams have processes that are followed irregularly. –– Undocumented: Individuals and/or teams follow processes that are not standardized nor documented. –– Defined: Processes used across business units and/or functions are documented but not enforced. –– Managed: Processes are documented and enforced. –– Optimized: Processes are documented, enforced, and measured to drive improvement. 6. Does your organization have a strategic plan for analytics and information management? –– Yes, we have a standalone strategic plan for analytics and information management. –– Yes, analytics and information management are incorporated into a broader IT strategic plan. –– Yes, analytics and information management are incorporated into a broader business strategic plan. –– No, we do not have a strategic plan for analytics and information management.
7. Which components and initiatives are included in your strategic plan for analytics and information management? –– Formalizing the structure, roles, and responsibilities for analytics –– Implementing information architecture requirements –– Implementing information governance requirements –– Implementing information quality standards –– Implementing new analytical tools or capabilities –– Conducting training on using analytical tools –– Developing knowledge worker analytical skills –– Developing an analytical culture –– Measuring analytical performance 8. Are your analytics and information management initiatives aligned to business strategic goals? 9. Which groups are involved in developing the strategic plan for analytics and information management? –– IT –– Finance –– Marketing –– Corporate Planning –– Business Shared Services –– Business Unit –– Standalone Information Management Group –– Sales –– Supply Chain –– Other, please indicate –– N/A—No Strategic Plan 10. If IT is involved in developing the strategic plan for analytics and information management, what role do the following play? –– CIO –– Project Management Office –– Enterprise Architecture –– Infrastructure –– Applications –– Security –– Dedicated Information Management Team –– Other, please indicate: –– N/A—IT not involved in strategic plan
11. Rank order the business motivations for making analytics investments . –– Product or service development –– Improving product or service profitability –– Evaluating and prioritizing investment proposals –– Developing a corporate or business unit strategy –– New market identification and market strategy development –– Identifying and targeting customer segments –– Forecasting demand for products, services, or resources –– Other 12. Does your Enterprise Architecture group have information architects? 13. How many information architects does your EA group have? 14. How would you assess your organization’s current level of proficiency in the following disciplines? –– Data Strategy and Governance –– Data Stewardship –– Data Standardization –– Master Data Management –– Data Quality Management –– Reporting and Analytics –– Information Lifecycle Management 15. When IT provides a new report or analytics tool, does the project plan include formal steps to change business processes or workflow to make better use of it? 16. Are the implications of information and analytics requirements incorporated into other IT projects? –– Yes, before the project has been approved –– Yes, when building out requirements for the project –– Yes, after the project has been deployed 17. Which of the following best describes your organization’s approach to master data management? –– No formal initiative is currently in place or planned around master data management. –– Master data is defined and managed at the individual system level. –– Siloed, department or group-level master data management strategy has been defined and established.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
133
VIII. Information Strategy and Governance (Continued) –– A unified, enterprise-level master data management strategy has been defined and established. 18. Which of the following do you use as part of your master data management? –– Source identification –– Data collection –– Data transformation –– Normalization –– Rule administration –– Error detection and correction –– Data consolidation –– Data storage –– Data distribution –– Data governance –– None 19. Who owns or stewards master data subjects (e.g., customer, product)? –– IT –– Business unit –– A central or cross-functional master data management or governance group –– Other –– None 20. Does your organization have an enterprise data model for organizing information? 21. Which information types are included in the enterprise data model? 22. If your organization does use a data model, which of the following components are included in your data model? –– Data architecture –– Data modeling –– Data properties –– Data organization –– Data structure –– Data model theory –– N/A—no data model 23. If your organization does use a data model, is there a process for reviewing new applications, tools, and databases for alignment with the enterprise data model? –– Yes, before they are developed or purchased CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
134
IX. Technical Environment –– Yes, after they are developed or purchased –– No –– N/A—no data model 24. Does your organization have an information architecture in place? –– Yes—We have documented both the current and future state architectures for the enterprise and have a roadmap in place against which we are moving toward the future state. –– Yes—We have architected the current and future states for different business units or areas of the enterprise but have yet to aggregate into a unified whole. –– Yes—We have an initiative in place but are still in the early stages. –– No—We have a vision for what we want to do but no formal initiative. –– No—We don’t have an information architecture and don’t plan to create one. 25. Which of the following best describes your data governance processes? –– We have formal governance structures in place that are understood and adhered to across the enterprise. –– We have formal governance structures in place that are understood and adhered to, but only in parts of the organization. –– We have some formal and some informal governance processes in place; adherence is mixed. –– Governance processes are generally ad hoc and adherence is generally low across the organization. –– There are no formal and few informal governance processes in place. 26. Rate the following end user requirements that the Information Management group is working to solve in order of urgency). –– Information quality or accuracy –– Information timeliness –– Information consistency –– Information relevance –– Information accessibility or ability to search –– Usability of reporting and analytical tools –– Flexibility to use personally preferred tools, e.g. via cloud or mash-ups
1. Do you use any document repositories that are not provided and supported by corporate IT (e.g., Google documents, Dropbox, iDisk) for business purposes? 2. Do you use any business intelligence/analytics tools that are not provided and supported by corporate IT? 3. If you answered “Yes,” why do you use tools not provided and supported by corporate IT? –– IT does not provide an alternative tool. –– The non IT provided tool is easier to use. –– The non IT provided tool is more sophisticated. –– The rest of my team uses the non IT provided tool. –– Other 4. Which of the following best describes the analytical tools available to you? –– There is no tool available to me. –– I only have access to Excel. –– There is a single tool I have access to that is consistent across the entire company. –– There is a single tool I have access to that is consistent across my business unit/function. –– There are multiple tools with similar functionality I have access to and I am free to use the one I prefer. 5. Which of the following collective intelligence tools are currently being used at your organization? –– Discussion forms –– Blogs –– Wikis –– Prediction markets –– Ratings and online surveys –– Other, please indicate 6. Which of the following best describes your organization’s approach to selecting analytical tools? –– Tools are centrally selected and deployed across the organization. –– Tools are centrally selected and deployed across the organization, but we customize capabilities to meet the needs of different functions/business units. –– Tools are centrally selected, but different tools are deployed to different business units/functions based on their unique needs.
IX. Technical Environment (Continued) –– Business units/functions choose their own tools with guidance from IT and the information management and/ or analytics group. –– Business units/functions choose their own tools and IT/ Information Management are not/rarely involved in the decision process. –– Individual knowledge workers choose their own tools. 7. How many different analytical tools are in use? –– Across the company –– Maximum within a business unit –– Maximum within a function –– Per information type 8. How many different operational systems are in use? –– Across the company –– Maximum within a business unit –– Maximum within a function –– Per information type 9. How many different corporate information repositories are provided? –– Across the company –– Maximum within a business unit –– Maximum within a function –– Per information type 10. What is the total volume of storage in your organization (include primary, backup, and archive) in TB? 11. What is the annual percentage growth rate of total storage volume in your organization (include primary, backup, and archive)? 12. Which of the following emerging technologies and capabilities have you implemented? –– In-memory analytics –– Data mash-ups –– Cloud/SaaS BI or analytics –– Columnar databases –– Mobile BI –– Interactive visualization –– Enterprise search –– Semantic technologies –– Complex event processing –– Advanced analytics
13. Which of the emerging technologies and capabilities are in use (either broadly or narrowly)? 14. To what extent are analytics and information management solutions deployed using a test and learn approach? 15. Does your organization have an enterprise data warehouse? –– Yes—data from both structured corporate systems and unstructured information repositories are all stored in a single data warehouse. –– Yes—structured information from all corporate systems are stored in a single data warehouse, but Unstructured information is stored elsewhere. –– Yes—data from some corporate systems are stored in a single data warehouse, but there is a significant amount of data outside the data warehouse. –– No—data is stored in several data marts. 16. What is the total number of data warehouses within your organization?
contributing to its removal? Please check all that apply. –– No longer required to retain the information for regulatory and compliance purposes –– Information no longer has business value. –– Too costly to continue storing the information 20. Does IT integrate external sources of information into the data warehouse? –– Yes—we proactively search for external sources of information that could be valuable to the business and integrate it into the data warehouse. –– Yes—when the business recurrently uses an external source of information, we integrate it into the data warehouse. –– Partially—we make some external sources of information available internally, but it is kept separate from the data warehouse. –– No—we do not integrate external sources of information into our environment. 21. Do you have a master data management tool?
17. What is the total size of data warehouses within your organization in TB? 18. Which of the following best describes the information that is included in your organization’s data warehouse? –– All information generated by our organization is included in the data warehouse indefinitely. –– All information generated by our organization is included in the data warehouse, but it is retired and removed after a defined period of time. –– Only information deemed valuable to the business is included in the data warehouse. –– Only certain information is included based on technical feasibility. –– Only certain information is included based on business requests. –– N/A—not in data warehouse 19. If your organization removes information from the data warehouse after a period of time, what are the factors
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
135
X. Demographics 1. What is your age? 2. What is the highest level of education you obtained? 3. In which subject or discipline did you achieve an undergraduate- or graduate-level university degree? 4. What is your tenure at your current organization? 5. Which of the following most closely describes your current level in the organization? 6. Which of the following most closely describes your current function in the organization? –– Communications –– Corporate Strategy –– Customer Service/Contact Center –– Finance and Accounting –– Financial Services –– General Management Functions –– Human Resources –– Information Technology –– Legal and Compliance –– Marketing/Market Research –– Production/Operations/Procurement –– Professional Services –– Real Estate and Facilities –– Research, Development, and Engineering –– Retail –– Sales 7. Indicate any additional functions you have worked in previously. 8. In what country do you primarily work? 9. In which country is your organization headquartered? 10. Where do you spend the majority of your time working? 11. Which of the following best describes your attitude toward new technologies? 12. Which of the following best describes your organization’s industry? 13. How geographically dispersed is your organization? 14. What is the ownership structure of your organization?
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
15. How high is the regulatory burden faced by your organization? 16. What is the level of business integration at your organization? 17. What is the level of information intensity of each of the following at your organization? 18. What is the overall annual revenue (in millions of your currency) for your organization? 19. To what degree does your organization handle sensitive or confidential information? 20. How many employees does your organization employ? 21. In how many countries does your organization have operations? 22. Which of the following best describes your IT organization’s structure? 23. How many IT users does your organization support? 24. How significant a part of your job responsibilities is conducting analysis?
Overcoming the Insight Deficit
136
Cio Executive Board
Overcoming the Insight Deficit Big Judgment in an Era of Big Data
© 2011 The Corporate Executive Board Company. All Rights Reserved.
Information Technology Practice
Cio Executive Board Executive Directors Shvetank Shah Warren Thune
www.executiveboard.com
Managing Directors Jaime Capellá Brian Foster Andrew Horne David Kingston Jay Shankavaram Senior Directors Vimarsh Bakaya Miles Gibson Bill Lee Chris Mixter Rumki Saha Alex Stille KD Weitzel Project Manager Matthew Charlet Shalini Das
Directors Aron Kuehnemann Tim Macintyre Dorota Pietruszewska Kristin Sherwood Associate Directors Brent Cassell Christiane Groth Karolina Laskowska
Production Designer Carolyn Lamond Contributing Designer Supriya Dhasmana Editor Nidhi Vikram Choudhury
Consultants Kelly Chambers Marina Murray Senior Analyst Pallavi Goel Shilpa Pental Analyst Raf Gelders
Copies and Copyright
Legal Caveat
As always, members are welcome to an unlimited number of copies of the materials contained within this handout. Furthermore, members may copy any graphic herein for their own internal purpose. The Corporate Executive Board Company requests only that members retain the copyright mark on all pages produced. Please contact your Member Support Center at +1-866-913-8101 for any help we may provide.
The Corporate Executive Board Company has worked to ensure the accuracy of the information it provides to its members. This report relies upon data obtained from many sources, however, and The Corporate Executive Board Company cannot guarantee the accuracy of the information or its analysis in all cases. Furthermore, The Corporate Executive Board Company is not engaged in rendering legal, accounting, or other professional services. Its reports should not be construed as professional advice on any particular set of facts or circumstances. Members requiring such services are advised to consult an appropriate professional. Neither The Corporate Executive Board Company nor its programs are responsible for any claims or losses that may arise from a) any errors or omissions in their reports, whether caused by The Corporate Executive Board Company or its sources, or b) reliance upon any recommendation made by The Corporate Executive Board Company.
The pages herein are the property of The Corporate Executive Board Company. Beyond the membership, no copyrighted materials of The Corporate Executive Board Company may be reproduced without prior approval.
CIO0361911SYN
Content Publishing Solutions
TABLE OF CONTENTS Executive Summary • iv An Information Management Strategy on a Page • vi Assess Insight IQ in Your Organization • vii Occasion for the Research • 1 Enable Information Usability • 25 Capability-Driven Tool Portfolio (Blue Cross Blue Shield of North Carolina) • 33 Flexible and Transparent Analytics (GlaxoSmithKline) • 47 CXO KPI Identification (PepsiCo) • 61 Support Big Judgment, Not Just Big Data • 77 Information Training Portfolio (Tiffany & Co.) • 87 Observe and Question to Identify Opportunities • 101 “Prosumer” Sounding Board (Ford) • 105 Anthropologically Driven Needs Identification (Alpha Company1) • 109 Appendix • 116
1
Pseudonym.
CIO0361911SYN
iv
Executive Summary Our ability to store and analyze information has grown exponentially across the past decade, but the buzz about “big data” misses the reality that as data volumes explode, it becomes harder, not easier, for knowledge workers to make sense of it. Less than 40% of knowledge workers have the skills and judgment to exploit information for competitive advantage. Paradoxically, as organizations amass more data and spend more on analytics, they face a growing insight deficit. The insight deficit is important to IT, as information management will likely be IT’s main contribution to enabling growth. In business areas that drive growth—innovation, marketing, sales, and customer service—up to 80% of IT enablement opportunities relate to business intelligence, collaboration, or the customer interface. Yet, the insight deficit means that efforts to improve information management may lead to worse, not better, decision making. Insight IQ: A Unique View of How Organizations Exploit Information for Decision Making To give IT leaders practical guidance on overcoming the insight deficit, we built an Insight IQ index to measure individuals’ ability to find and analyze information to make better decisions. Insight IQ has three elements: information attainability, information usefulness, and knowledge worker capability. Using the model, we surveyed almost 5,000 knowledge workers at hundreds of organizations globally. We used the survey to quantify the value of overcoming the insight deficit and pinpoint the actions needed to realize this value. (See page vii to learn how your organization can take the Insight IQ diagnostic). The benefits of overcoming the deficit are significant. Functions with the highest Insight IQ perform, on average, 24% better than their peers across a wide range of metrics. All three components of Insight IQ are vital to achieve this goal—if any are missing, the chance of failure is 65% or more.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Core information governance activities such as information architecture, business stewardship of data, and an analytics center of excellence are important drivers of Insight IQ. Most organizations know this and already have initiatives in place. What is more surprising is the impact of three capabilities directly related to the knowledge worker: information usability, knowledge worker skills, and the identification of emerging knowledge worker information needs. These three capabilities are often overlooked, yet all directly impact Insight IQ. 1. Enable Information Usability Only 50% of knowledge workers find information from corporate sources to be in a usable format. The problem will only get worse as the number of information sources, uses, and users continue to increase. In this diverse environment, trying to improve usability by offering self-service usually disappoints as few self-service capabilities are flexible enough to meet knowledge worker needs. To overcome this challenge, the best companies offer a choice of analytic tools and deploy information filtering and visualization capabilities. They develop a deeper understanding of how, when, and why information will be used by specific user segments, and they vary quality standards accordingly. Offer Greater Tools Choice—Greater choice in analytic tools drives Insight IQ. The greatest increase comes from moving from one standard tool for the enterprise to tools aligned to specific business units or functions. Rationalization programs for analytic tools often over-standardize, leaving employees without necessary capabilities and opening the door to unofficial tools. In response, progressive organizations, such as Blue Cross Blue Shield of North Carolina offer knowledge workers a portfolio of tools aligned to a list of analytic capabilities. Visualize and Filter—Many knowledge workers complain about the time required to aggregate information from supposedly user-ready reports. Consequently, leading organizations invest in capabilities such as information visualization, aggregation, and filtering before they invest in more sophisticated analytics.
Executive Summary (Continued) Make Information Quality Transparent and Targeted—Only one-third of knowledge workers trust information from other functions in the organization. To overcome this distrust, progressive organizations such as GlaxoSmithKline clarify the information’s source, how it has been manipulated, its current quality level, and caveats about how it can be used. Others, such as PepsiCo, recognize that certain groups, such as C-level executives, have distinct patterns of information usage that need to be targeted for different-in-kind support.
Provide Tools for Collaborative Decision Making—Making biases explicit and listening to alternative points of view are characteristics of informed skepticism. To support these behaviors, IT can provide workflow tools that connect stakeholders involved in decision making, capture “wisdom of the crowds” input, and add contextual data. In addition, business leaders can help promote informed skepticism by introducing formal decision-making models and by embedding the right behaviors in performance expectations.
2. Support Big Judgment, Not Just Big Data
3. Observe and Question to Identify Opportunities
Exploiting information requires broader and deeper analytic skills across the organization. Most employees are now knowledge workers and spend on average 36% of their time collecting and analyzing information. However, only 38% of employees are “informed skeptics” who combine analytic skills with judgment to make sound decisions. These critical thinkers are comfortable with analytics but maintain a healthy skepticism about its limitations.
Most IT organizations are familiar with process automation projects where business needs are known and stable. In contrast, information needs are context dependent, dynamic, and may be unarticulated or even unknown. Solving this challenge requires anthropological skills that are in short supply in IT.
Develop an Analytics Training Curriculum—Two-thirds of knowledge workers have access to analytics training, but only 25% find the training effective. Training that focuses on the tools but does not address information sources or analytical techniques is ineffective. Leading organizations, such as Tiffany & Co., adopt a multipronged approach that supports knowledge workers across the information lifecycle and focuses on conducting analysis. This type of training has no natural organizational home, so IT and business leaders should collaborate to determine ownership. Hire Analysts Who Can Coach—Knowledge workers should be supported by business intelligence experts or analysts who act as force multipliers. While coaching skills are what make these analysts most effective at driving Insight IQ, organizations typically look for quantitative and business skills when hiring analysts. Leading organizations have changed their recruiting approaches to actively test for coaching skills.
Identify and Learn from Early Adopters—Traditional requirements gathering fails when assessing information needs as the needs are fast changing, diverse, and difficult to articulate. Instead, progressive organizations, such as Ford, surface information needs by segmenting knowledge workers by their information usage patterns and their stance toward technology adoption. They also observe early adopters as an early-warning mechanism for emerging needs. Observe and Question—Knowledge workers often struggle to articulate their information needs, and these needs differ across individuals, organizational cultures, and collaborative practices. To proactively identify pain points in end-user workflows, leading IT groups adopt an anthropological approach to identify opportunity, observe knowledge workers in diverse user settings and conduct open-ended interviews.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
v
vi
An Information Management Strategy on a Page Statement of Strategy Shift the focus of IT investment and skills toward information management with the goal of providing knowledge workers with attainable and useful information and boosting their capability to exploit that information for competitive advantage.
Top Initiatives 1. Launch an information management and analytics center of excellence. 2. Redesign IT's opportunity identification process to make it proactive and informed by observation of distinct knowledge worker segments. 3. Identify analytic capabilities used by knowledge workers, and offer a portfolio of tools to meet those needs. 4. Develop and hire usability and interface design skills in IT. 5. Coach knowledge workers to boost their analytic capability and foster informed skepticism. 6. Harmonize and integrate a small subset of information subjects where there is greatest enterprise need.
Key Information Management Metrics
Current (2011)
Percentage of IT budget devoted to information and analytics projects
23%
40–50%
Percentage of budget spent on knowledge worker capability
< 5%
> 10%
> 80%
< 40%
1–3
8–12
38%
59%
51
65
Percentage of information subjects targeted for harmonization and integration Target number of analytic tools Percentage of knowledge workers who are informed skeptics Average Insight IQ score
Underlying Assumptions and Beliefs 1. The number of opportunities to drive growth through information management will equal or outstrip the opportunities for process automation. 2. Many of our knowledge workers lack the skills and judgment to use information effectively for decision making. 3. Not all information needs to be harmonized or integrated at enterprise level. Similarly, some information needs higher levels of quality than others. 4. Our business partners will take the lead in information stewardship. 5. Knowledge worker reliance on external information sources and on unstructured information will continue to rise. Obtain a customizable template for a strategy on a page at www.cio.executiveboard.com. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Goal (2014)
Use the Insight IQ Diagnostic to benchmark the maturity of your information management and prioritize areas for investment.
Assess Insight IQ in Your Organization Overview of the Insight IQ Diagnostic
■■
■■
■■
■■
Diagnostic Benefits
The diagnostic measures Insight IQ using online surveys for knowledge workers and an IT leader. Organizations identify at least 30 knowledge workers per business function, and at least three business functions participate in the diagnostic.
1. Assess Readiness—Identify which business functions and teams in your organization are ready for additional investment in information management and analytics and be forewarned about areas where adoption may be low.
2. Set Priorities—Prioritize the technology investments, governance processes, and skills that have the greatest impact on Insight IQ.
Each survey takes approximately 20 to 25 minutes to complete. The diagnostic is included in your existing CIO Executive Board membership.
3. Communicate—Demonstrate the business leadership, communication, and change management efforts necessary to close your organization’s insight deficit. 4. Benchmark—Compare your organization’s Insight IQ against industry peers.
Key Deliverables Custom Diagnostic Report and Improvement Recommendations
Detailed results benchmarked against 5,000 knowledge workers worldwide
Guidance on practical improvement steps, including tactics
Executive Advisor® walkthrough of results and key recommendations
For more information, contact your account manager or e-mail [email protected]. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
vii
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
viii
Occasion for the Research
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
1
“Big data” is quickly becoming a reality as information volumes grow by 60% annually, and 36% of all work time is devoted to information collection and analysis.
■■
■■
MORE INFORMATION, MORE INFORMATION WORK Estimated Rise in Global Data Volumes, 2010–2015 Indexed to 100 1,050 1050
A 60% annual rise in information volume means that average storage per end user will increase from 2 GB to 20 GB in five years. Knowledge worker time spent collecting and analyzing information is similar across all functions, for example: –– Finance, 40% –– General Management, 35%
Overcoming the Insight Deficit
60%
R
CAG
660 410
100 2010
160
2011
260
2012
2013
2014
Time Spent Collecting and Analyzing Information
2015
Drivers of Democratized Decision Making
Percentage of Total Knowledge Worker Work Time
–– HR, 33%
1
Decisions are made closer to the market (e.g., product design, channel mix).
2
Decisions are more dynamic and varied (e.g., demand forecasts, discounts).
3
Knowledge workers have access to more information and better tools (e.g., customer segmentation and value analysis).
–– Marketing, 36% –– Production and Operations, 33% –– Sales, 27% ■■
Knowledge workers dedicate more time to collecting and analyzing information as decision making is democratized.
64% All Other Work
36% Collecting and Analyzing Information
n = 4,941 knowledge workers. Source: “All Too Much” The Economist, 27 February 2010; Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
2
To drive growth, IT must support less familiar areas of the business where the majority of opportunities are in information management, not process automation.
FROM PROCESS TO INFORMATION TO DRIVE GROWTH Breakdown of IT Enablement Opportunities by Business Process Percentage of Total Opportunities None
9% 9%
15%
■■
Information Management (Collaboration, Business Intelligence, Customer Interface)
18%
27%
28%
IT enablement opportunities are divided between:
Process Automation
–– Information Management—Includes collaboration, business intelligence, and the use of technology at the customer interface, and
30% 63%
28%
63% 60%
–– Process Automation— Processes enabled by enterprise systems such as ERP, PLM, HRIS, and CRM.
54%
46%
28% 18%
Finance and HR
“Information management means we encounter functions with little prior experience working with IT. But to contribute to growth, we have to understand these areas.” CIO Consumer Goods Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
■■
■■
Production and Supply Chain
Extensive experience working with IT Primarily drive efficiency and quality
Customer Service ■■ ■■
Marketing and Sales
13% Product/Service Innovation …but increased support for growth moves IT into much less familiar areas.
Little experience working with IT Primarily drive growth
Continued focus on process automation, will mean IT misses opportunities to drive growth… Note: Totals may not equal 100% due to rounding. Source: “Process Classification FrameworkSM,” APQC, September 2009, http://www.apqc.org/knowledge-base/documents/apqc-process-classification-framework-pcf-cross-industryexcel-version-51-0.
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
3
As 62% of knowledge workers lack the ability to apply “big judgment” to information they use for decision making, increased information availability may do more harm than good.
Overcoming the Insight Deficit
Without Big Judgment, Big Data Magnifies Risk not Opportunity Have Processes and Skills to Use Information Effectively for Decision Making Percentage of Knowledge Workers
Risks from Big Data When Big Judgment Is Lacking
38% Yes
62% No
■■
Exaggerated or overlooked opportunities
■■
Underestimated risks
■■
Disputed decisions
■■
Misleading performance measures
■■
Wasted knowledge worker time
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
“The biggest talent challenge we face is finding people with the skills to create insight from data.” VP, HR Global Food Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
4
Effective information management reframed around the knowledge worker drives improved decision making.
■■
■■
MEASURING THE INSIGHT DEFICIT The Three Components of Insight IQ
Insight IQ is a quantifiable outcome that can be measured at the level of the individual knowledge worker. Linking information management initiatives with higher Insight IQ will refocus information management on the important outcome.
Insight IQ entails the ability to find and analyze relevant information to drive actions and decisions effectively.
The right information is available and easy to find.
Information Attainability
Information Usefulness
Information is of a known quality and in a usable format.
Insight IQ
Knowledge Worker Capability
Knowledge workers have the analytic ability and predisposition to generate insight from information.
Source: Corporate Executive Board, IT Practice Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
5
The CIO Executive Board would like to extend its sincerest appreciation to the following organizations for their participation in this research.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
With Sincere Appreciation Partial List of Participating Organizations
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
6
The CIO Executive Board’s database spans a variety of organization sizes and industries and includes responses from almost 5,000 knowledge workers globally.
■■
■■
Survey administration involved in-depth collaboration with 22 member organizations. Additional surveying of knowledge workers at more than 1,000 organizations worldwide was conducted to supplement the dataset.
The Only Survey of Its Kind Respondents by Industry
Respondents by Function
Sample Distribution
Sample Distribution
2% Energy and Utilities 3% Telecommunication 4% Other 4% Aerospace/Defense, Automotive, and Transport 4% Government 5% Professional Services 5% Manufacturing 9% Chemicals, Pharma, and Health Care 9% Banking and Financial Services 10% Retail
2% Legal 3% Corporate Strategy 3% HR 18% Finance
6% Other
27% High Tech
6% General Management
18% IT
7% R&D and Engineering
18% Insurance
15% Sales, Marketing, and Retail
10% Customer Service 12% Production/ Operations/ Procurement
Respondents by Geography Sample Distribution 1% Other 2% Latin America 14% APAC
65% North America
18% EMEA CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
7
Overcoming the Insight Deficit
A Model of Insight IQ Insight IQ entails the ability to find and analyze relevant information, to drive actions and decisions effectively.
1 Attainability—Information is easy to find. ■■
■■
■■
The information that I need to do my job is available. I know which sources to go to for the information I need. I can find subject matter experts and other sources of tacit knowledge.
2 Usefulness—Information is useful. ■■
■■
Information Attainability
Information Usefulness
■■
■■
■■
Insight IQ
Information is useful and does not require modification or manipulation. I understand how information is defined. There is a single version of the truth for specific information. Information is directly relevant to business decisions and processes. Information is timely, up-to-date, precise, and accurate.
Knowledge Worker Capability
3 Knowledge Worker Capability—Knowledge workers have the analytic ability and predisposition to generate insight from information.
■■
■■
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
I rely on the results of analysis to support business decisions, processes, and deliverables. I am comfortable with advanced analytics.
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
8
ASSESSING THE MOST POWERFUL DRIVERS OF Insight IQ 2 Determine Drivers of Insight IQ
1 Deploy Surveys Knowledge Worker Survey (n = 4,941.) Knowledge workers from all major functions and business areas are surveyed on information usage.
IT Leader Survey (n = 22.) IT representative provides details on information management platforms, strategy, governance, and architecture.
3 Translate Findings into Practical Guidance ■■
Potential Drivers of Insight IQ
■■
1. Knowledge Worker Attitudes and Incentives 2. Training and Support 3. Staff Knowledge and Competencies 4. Information Quality 5. Information Usage Patterns 6. Demographics 7. Information Management Structures and Processes 8. Investment and Resource Level 9. Information Strategy and Governance 10. Technical Environment
Distill the practical implications borne out by the data. Provide real-world illustration of key insights via practitioner tools and tactics.
Illustrative Driver: Availability of Training on Conducting Analysis Average Insight IQ Score, Indexed ∆ = 29% (Maximum Impact)
129
100
Sample Questions Which of the following best describes the analytic tools available to you?
No Tools
Information Usefulness
Information Attainability
Multiple Tools
Insight IQ
How effective is the training provided by your organization on conducting analysis?
Very Ineffective
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
No Training on Conducting Analysis Available
Knowledge Worker Capability
Very Effective
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Yes, Training Available on Analysis
Existing Research
Occasion for the Research
9
Some knowledge workers have up to three times higher Insight IQ than their peers.
Overcoming the Insight Deficit
Finding 1: Insight IQ Varies Substantially Across Knowledge Workers Knowledge Worker Insight IQ1 Range Database Percentile Values of Insight IQ
Some knowledge workers demonstrate almost three times higher Insight IQ than their peers.
77 65
51
38 28
10th Percentile
25th Percentile
50th Percentile
75th Percentile
90th Percentile
n = 4,941 knowledge workers. 1
Insight IQ is measured as an index constructed from the three model components. Index scores ranges from 0 to 100, with higher scores reflecting higher levels of Insight IQ.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
10
There is wide variation in Insight IQ across knowledge workers within the same organization.
Variation Within and across Insight IQ1 Range by Organization
90th Percentile ■■
■■
Although not all knowledge workers need to reach even the average level of Insight IQ, small improvements can have a significant cumulative effect given the ubiquity of knowledge workers within most organizations.
Average 10th Percentile
100 90 80
Variation within organizations demonstrates that company-level factors do not explain all differences in Insight IQ; there are important factors at the functional or knowledge worker level as well.
70 Spread of Average Insight IQ Across Organizations
60 50 40 30 20 10 0 Representative Sample of Participating Organizations
n = 22 organizations. 1
Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
11
Business functions with a high Insight IQ demonstrate 24% better performance than functions with a low Insight IQ.
■■
■■
Performance results were measured at the functional level to minimize the impact of exogenous factors and self-reporting bias. A performance index of 12 questions across the four categories of self-reported data was created to compare top- and bottomquartile functional teams.
Overcoming the Insight Deficit
Finding 2: OVERCOMING THE INSIGHT DEFICIT Has a Measurable Payoff Indexed Functional Performance by Level of Insight IQ
Performance of Functions with the Highest Insight IQ
1.24x
Twenty-four percent higher functional performance Performance of Functions with the Lowest Insight IQ
x
Components of Functional Performance Index Relative to similar functions at industry peers: ■■ ■■ ■■
■■
Functional effectiveness Functional efficiency Function-specific performance (e.g., market share growth, employee engagement) Impact of analytics on effectiveness, efficiency, and performance
n = 4,941 knowledge workers. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
12
The CIO Executive Board controlled for demographic factors that impact Insight IQ during all analysis.
Finding 3: Three Demographic Factors Impact Insight IQ Potential Demographic Drivers What Matters ■■
Seniority
■■
Openness to new technologies
■■
Importance of analytics to job role
■■
Age
■■
Highest degree earned
■■
College major
■■
Organizational tenure
■■
Company revenue
■■
Level of regulatory burden
■■
Information intensity
■■
Industry
■■
IT organizational structure
■■
Geographic span of business
What Doesn’t Matter
The CIO Executive Board applied statistical controls to void the impact of demographic factors that drive Insight IQ.
Knowledge Worker–Level Factors
Organization-Level Factors
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
13
Overcoming the Insight Deficit
Finding 4: All Components of Insight IQ Matter Poor performance in any of the three components of Insight IQ significantly diminishes the probability of strong business performance. Type 1: Knowledge Workers Cannot Easily Access Information
Information Attainability
Information Usefulness
Information Attainability
Type 3: Knowledge Workers Cannot Translate Information into Action
Information Usefulness
Information Attainability
Type 4: High Insight IQ
Information Usefulness
Information Attainability
Information Usefulness
Insight IQ
Insight IQ
Insight IQ
Insight IQ
Knowledge Worker Capability
Knowledge Worker Capability
Knowledge Worker Capability
Knowledge Worker Capability
Probability of strong business performance1 if information attainability is low: 18%
1
Type 2: Knowledge Workers Are Wary of Analytics
Probability of strong business performance if knowledge worker capabilities are low: 27%
Probability of strong business performance if information usefulness is low: 35%
Probability of strong business performance: 80%
Values are calculated as the probability of knowledge workers demonstrating top quartile performance in indexed business performance via logistic regressions.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
14
The likelihood of realizing full business value from an information management initiative is only 5% to 35% if the organization does not invest in all three components.
■■
■■
NO SHORTCUTS TO DEFICIT REDUCTION Likelihood of Realizing Full Value from Information Management and Analytics Percentage of Likelihood Under a Typical Deployment Pattern 80%
Yet, many organizations focus on system investments and governance at the expense of knowledge worker engagement.
User Engagement Typically unfunded
Each component considered individually is necessary, but not sufficient, to achieving full business value.
Data Governance Total Annual FTE costs1: $3 M
27% Without effective information governance, the likelihood of realizing full value is only 5%.
System Investments Annual project spend1: $18 M 5% 0% A data desert—no information is available.
1
Staff openness to analytics boosts the likelihood of realizing full value from 27% to 80%.
Technology without governance— information is attainable but quality and relevance are low.
Enough information to be dangerous— information is useful but staff don’t know how to analyze it.
Full value realization— there is complete absorption of analytics for decision making.
Estimates for a company with revenue of $8 B.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
15
We analyzed the dataset to isolate the most important drivers of Insight IQ and translate the findings into practical guidance.
Overcoming the Insight Deficit
FINDING 5: TOP DRIVERS OF INSIGHT IQ
Potential Drivers of Insight IQ Tested 1. Knowledge Worker Attitudes and Incentives 2. Training and Support 3. Staff Knowledge and Competencies 4. Information Quality 5. Information Usage Patterns 6. Demographics 7. Information Management Structures and Processes 8. Investment and Resource Level 9. Information Strategy and Governance 10. Technical Environment
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Information Usefulness
Information Attainability Insight IQ
Knowledge Worker Capability
Overview of findings
Roadmap for the discussion
Existing Research
16
Partial List of tested drivers I. Information Management Structures and Processes 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.
Analytic Capabilities and Practices Shared Analytics Teams Centralized Companywide Central Analytics Group in IT, Reporting Directly to CIO Central Analytics Group in IT, Within Information Management Delivery Channels for Providing Information/Analytics Degree of Integration of Management of Analytics and Information Degree of Knowledge Worker Involvement with Other Functions/Groups Existence of Formal Analysis Processes/Frameworks Existence of Master Data Management Tool Existence of Metadata Structure (Standards and Models) Integration of External Sources of Information into the Data Warehouse Information/Analytics Requirements Incorporated into Other Projects Early IT Organization Structure (e.g., Centralized or Decentralized) Knowledge Workers Have Enough Time to Conduct Analysis Master Data Management Process Includes Error Detection and Correction Mechanisms of Delivery for Each Type of Information— Pushed or Pulled Prevalence of Analytics in Group Versus Individual Decisions Process for Sharing Information Responsibilities for IM Group (Including Who Owns What) Responsibilities of Central Analytics Group Use of Aggregation, Visualization, and Analysis Use of Agile Software Development Use of External Providers for Analytics
II. Training and Support
24. 25. 26. 27.
Analytic Mind-Set of Respondent’s Manager BU/Functional Performance Metrics Reported Regularly Formality of Decision-Making Process Leadership Attitudes/Behaviors Toward Fact-Based Decision Making 28. Overall Frequency of Using Analysis 29. Use of KPIs/Metrics to Track Organizational Performance 30. Performance Reviews Based on Measurable Goals 31. Performance Reviews Evaluate Analytic Ability 32. Time Spent on Analysis 33. Existence of Processes/Capabilities to Overcome Biases
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Shading Denotes Top 15% Most Impactful Drivers
III. Knowledge Worker Attitudes and Incentives
V. Quality of Information (Continued)
34. User Confidence in Information Quality (Cleanliness and Accuracy) 35. What Level of Confidence Do You Have in the Quality (Cleanliness and Accuracy) of Information from Each of the Following Functions? 36. Knowledge Workers Are Rewarded and Promoted Based on Outcomes of Their Analysis and Decisions. 37. Knowledge Workers’ Perceived Ease of Use of Information Contained in Central Repository 38. Knowledge Worker Preference for Push Versus Pull for Types of Information 39. Knowledge Worker Trust in Information Contained in Central Repository 40. Knowledge Worker Trust in Information from Other Groups/ Functions (and Reason) 41. Level of Blame for Relying on Inaccurate Prediction 42. Perception That Being Analytic Is Advantageous to Career Development. 43. Perception That Analytics Is Hard Due to Lack of Time. 44. Perceived Importance of Information Aggregation Capabilities 45. Perceived Importance of Information Filtering Capabilities 46. Perceived Importance of Information Visualization Capabilities 47. Visibility of Internal Opportunities That Require Analytic Skills
60. New Technologies Prescreened for Enterprise Data Model Alignment 61. Quality Standards Vary by Function 62. Quality Standards Vary by Information Type 63. Quality Standards Vary by Seniority 64. Satisfaction Level for Each Type of Information 65. Timeliness Standards Vary by Information Type 66. Use of Common Definitions and Vocabulary in Information Presentations
VI. Information Usage Patterns
IV. Investment and Resource Level 48. Approach Taken to Getting Approval for Investments in IM/ Analytics 49. Cost of/Investment in Analytic Capabilities (by Function) 50. Level of IT Investment in Capabilities to Support Group Decision Making 51. Number of IT Staff Responsible for Information Management 52. Technology Costs for Managing Information
67. Capabilities in Place to Obtain Cross-System/Functional Information 68. Degree of Integration of Cross-Functional Information 69. Frequency of Using Each Type of Information/Analysis 70. IT Support for Different Types and Sources of Information 71. Nature of Information/Analytics Use (e.g., Customer Analysis, BPO) 72. Need of Information from Different Functions 73. Number of Sources Required for Each Type of Analysis 74. Purposes of Information/Analytics Use (e.g., Decisions, Reporting, Pattern Spotting) 75. Relative Use of Different Types of Info. (e.g., Structured/ Unstructured, Internal/External) 76. Satisfaction Level for Mobile Access to Key Information Types 77. Time Spent on Information Visualization, Filtering, Aggregation, and Analysis 78. Types of Analyses Being Done (e.g., Predictive, Prescriptive) 79. Types of Information Being Used 80. User Feedback on Information Sources and Analytics Tools Collected 81. Users Information Needs Assessed via Surveys 82. Users Information Needs Assessed by Tracking Information Requests
V. Quality of Information
53. Existence of Automation of Information Quality Governance and Standards 54. Existence of Process for Updating and Retiring Information (Info. Lifecycle Management) 55. Existence of Standard Definitions for Information 56. Data Transformations Are Transparent to Users. 57. Frequency of Reviewing/Updating Analytic Models and Assumptions 58. Functional Information with Standard Definitions 59. Information Quality Regularly Reviewed/Audited
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
17
Partial List of tested drivers (Continued)
Overcoming the Insight Deficit
18
Shading Denotes Top 15% Most Impactful Drivers
VII. Staff Knowledge and Competencies
VIII. Information Strategy and Governance
IX. Technical Environment
83. Analyst Skills Valued 84. Analytics Staff Hired and Assessed for Training and Coaching Abilities 85. Assessment of Analyst Speed, Quality, and Responsiveness 86. Availability of Support in Finding or Analyzing information 87. Availability of Training on Finding or Analyzing Information 88. Consumption of Training on Finding or Analyzing Information 89. Existence of Training on Use of IM and Analytic Systems 90. Effectiveness of Training on Finding or Analyzing Information 91. High-Quality Training on Finding, Understanding, and Analyzing Info. 92. Inclusion of Analytics in Job Descriptions 93. Knowledge Worker Analytic Skills and Level 94. Knowledge Worker Awareness of Security Policy Related to Information Sharing 95. Knowledge Worker Degree of Proficiency in Using Analytic Tools 96. Knowledge Worker IT Skills 97. Knowledge Worker Knowledge of What Information Exists in the Central Repository 98. Knowledge Worker Perception of Ease of Use of Sharing Analysis/Info. Using Analytic Tools 99. Knowledge Worker Relies on Judgment When Disagreeing With the Results of Analysis 100. Knowledge Worker Understanding of Analysis Behind Strategic Decisions 101. Knowledge Worker Understanding of How Information is Useful in Other Functions/Groups 102. Knowledge Worker Understanding of Responsibilities of Other Functions/Groups (i.e., What They Do) 103. Presence of Analytics Coach (Formal or Informal) 104. Processes for Understanding Tool Usage and Workflows 105. Skills and Knowledge of Staff Responsible for Managing Information 106. Use of Hypotheses to Drive Analysis
1 07. 108. 109. 110. 111. 112. 113. 114. 115. 116. 117. 118. 119. 120. 121.
134. Analytic Tools Bought Off the Shelf or Developed In House 135. Analytics Solutions Deployed Using a Test and Learn Approach 136. Automated Decisions Implemented 137. Customizability of Analytic Tools 138. Enterprise Search Implemented 139. Existence of and Types of Collaboration Tools/Capabilities 140. Existence of Automated Information Capturing Capabilities 141. Existence of Central Repository 142. Existence of a Data Warehouse 143. Existence of In-Memory Technology Capabilities 144. Existence of Non IT-Supported Information Repositories 145. Degree of Integration of Operational Systems 146. Information Sources (e.g., Smart Networks, Sensors) 147. Knowledge Workers Have Access to Analytic Tools Other Than Excel 148. Knowledge Worker Can Choose from Multiple Analytic Tools Available 149. Legacy Systems 150. Number of Analytic Tools Being Used/Provided 151. Number of Features of Analytic Tools 152. Number of Information Repositories 153. Number of Operational Systems (by Function) 154. Tool Interface Design, Testing, and Enhancements 155. Use of Cloud Applications to Generate, Store, or Analyze Information
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
122. 123. 124. 125. 1 26. 127. 128. 1 29. 130. 131. 132. 133.
Occasion for the research
Approach Taken on Analytic Tool Selection and Distribution Approach Taken on Analytic Tool Selection and Distribution Clear Goals or Strategies for Social Media in Place Components of Analytic Strategic Plan Coverage of Analytics Strategic Plan Decision Stakeholders Degree of Standardization of Key Business Processes Degree of Sharing Information with Customers Degree of Sharing Information with Suppliers Documentation of Business Processes Existence of Analytics Strategic Plan Existence of Data Steward Role Existence of Dedicated Information Management Team Existence of Strict Security Policies Around Information Information Quality Standards and Policies Are Defined and Enforced Information Quality Standards and Policies Enforced by Info. Management Group Information Quality Standards and Policies Enforced by Shared Services Group Information Security and Access Policies Do Not Inhibit Attainability of Information Inclusion of Analytics in Broader Corporate/IT/Other Function Strategic Plan Information Architecture in Place or Planned Keeping Track of New Information Systems and Analytic Capabilities Organization Characteristic (e.g., Consensus-Based or Hierarchical) Organization Strategic Business Goals/Objectives Overall Ownership of IM Group Perceived Clarity of Roles and Responsibilities for IM Group Policies for Adding Information to a Data Warehouse Robustness of Enterprise Data Model
The Insight IQ Diagnostic
Overview of findings
X. Demographics 156. Function Responsible for Developing Analytics Strategic Plan 157. Knowledge Worker Career Background (Experience, Number of Functions Worked In) 158. Knowledge Worker Function 159. Knowledge Worker Seniority 160. Organization Industry
Roadmap for the discussion
Existing Research
1
CIO0361911SYN
CIO Executive Board IT Practice www.cio.executiveboard.com
Occasion for the research The Insight IQ Diagnostic
Overview of findings
24.7% 24.6%
15.0% 14.8%
Roadmap for the discussion
10. Surveying KWs to Determine Information Needs
2. Support Big Judgment, Not Just Big Data
3. Collecting KW Feedback on Information Sources and Analytics Tools
16.4%
18. Hiring Analytic Staff for Their Training and Coaching Abilities
13.7%
16. Considering KW’s Analytics Capability When Evaluating Performance
17.4%
11. Basing Performance Reviews on Measurable Goals
18.6%
8. Regularly Reporting Business/ Functional Performance Metrics
20.0%
7. Providing KWs with Access to Analytic SMEs
6. KWs Are “Informed Skeptics”
1. Enable Information Usability
5. Formalizing Decision-Making Process
25.2%
4. Training KWs on How to Find and Conduct Analysis
14.9% 14.7%
20. KWs Choosing from Multiple Analytic Tools
16.0%
19. Enabling Enterprise Search
17.2% 16.4% 16.2% 15.5%
17. Providing Information Visualization and Aggregation Tools
The Usual Suspects
14. Providing Visibility into Data Transformations
2. KWs Having Confidence in Information Quality
15. Defining and Enforcing Quality Standards
13. Having Central Analytics Group or COE
11. Having a Mature Data Model
9. Having an Information Architecture
1. KWs Having Enough Time to Conduct Analysis
The Top Drivers of Insight IQ1 3. Observe and Question to Identify Opportunities
30.3% 24.8%
16.8%
n = 4,941.
KW = Knowledge worker.
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Existing Research
Occasion for the Research 19
To help drive insight IQ, IT organizations will require new roles or skill sets in four areas.
Overcoming the Insight Deficit
20
FINDING 6: INSIGHT IQ HAS IMPLICATIONS FOR IT ROLES AND SKILLS IT Roles and Skills Required in a High Insight IQ Organization
1. Enable Information Usability
2. S upport Big Judgment, Not Just Big Data
3. O bserve and Question to Identify Opportunities
Information Architects
Information Insight Enablers
Corporate Anthropologist
Develops and maintains information architecture that enables knowledge worker productivity and decision making
Supports knowledge workers with insights, business intelligence, and management reports for effective decision making
Uses observational and experience mapping techniques to identify unmet needs for information and analytic capabilities and other technology functionality
User Experience Specialists INFORMATION TECHNOLOGY PRACTICE
CIO EXECUTIVE BOARD ™
The IT Talent Implications of the Future of Corporate IT A Guide to Filling Emerging IT Skills Gaps
Designs and configures user-centric interfaces and information visualization techniques based on a deep understanding of end-user requirements and behaviors
For full details about our analysis of emerging IT skills and roles, see The Talent Implications of the Future of Corporate IT study. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Business leaders outside IT have a vital role to play in creating a high Insight IQ organization.
Finding 7: Information Ownership is a Critical Business Role Guidance to Business Leaders for Driving High Insight IQ
■■
To highlight the criticality of business leader involvement in information management, Corporate Executive Board sent a summary of Overcoming the Insight Deficit to all members in all functions globally.
1. Take Ownership of Your Information Business leaders must determine what data is necessary, what analyses they want to run, how data is shared across organizational boundaries, and how they derive insight from data. EXECUTIVE GUIDANCE FOR 2011
Overcoming the Insight Deficit Big Judgment in an Era of Big Data
2. Put Information at the Heart of the Business Plan Tightly integrating information management plans with business strategic plans drives higher Insight IQ. This means deconstructing vague strategy into more stable business capabilities. 3. Create Incentives for Cross-Company Data Standardization Efforts Executives should create a strong governance process that cuts across the company, including a corporate “tax” to overcome the first-mover disadvantage. 4. Improve the Quality of Your Information Business leaders should own the maintenance of authoritative data sources, approve new and modified definitions, establish confidentiality levels, and define data-quality processes. 5. Centralize Management, Not Information Foundational analytic investments, such as a dedicated analytics team, training design, and information quality norms, benefit from centralization, create the oversight to sustain decentralized information sources.
To access copies of this report for business executives, visit http://www.executiveboard.com/executive-guidance/2011/Q3/index.html.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
21
Overcoming the Insight Deficit
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
22
Information management is not a new topic; most organizations have foundational capabilities in place.
Building on a Solid Foundation Existing Capabilities and Relevant Resources Architecture Roadmapping
■■
While necessary, these capabilities are not sufficient to boost an organization’s Insight IQ or to move IT’s capabilities from process automation to information management.
Target Architecture Roadmaps (Wells Fargo)
Business Architecture (KBC Bank)
Information Strategy and Stewardship Task-Specialized Data Stewardship (RBC Financial) Information Quality Data Quality Audit Team (Hewlett-Packard)
Information Health Scorecard (International Monetary Fund)
Process Breakpoint Identification (Intel)
Report Generation Business Metrics Selection Tool (Corporate Executive Board) Systems Integration and Rationalization End-to-End Simplification Business Case (Charles Schwab)
To access the Board’s existing resources, Visit the Information Management topic center at www.cio.executiveboard.com. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Applications Portfolio Rationalization (CNA Financial)
User Support Segmented Information User Support (Hallmark)
Occasion for the research
The Insight IQ Diagnostic
Overview of findings
Roadmap for the discussion
Existing Research
Occasion for the Research
23
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
24
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Enable Information Usability
25
One-half of all knowledge workers find that information from corporate sources is not in a usable format.
■■
Overcoming the Insight Deficit
THE USABILITY DEFICIT Information from Corporate Sources Is in a Usable Format Percentage of Knowledge Workers
Improve information usability by letting knowledge workers choose the tools they find easiest to use, prioritizing information visualization and offering greater transparency into quality levels.
50% No
50% Yes
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
26
Knowledge workers rely on a wide range of information sources, both internal and external.
■■
The proliferation of information sources creates challenges for information aggregation, presentation, and quality.
INFORMATION OMNIVORES Mean Number of Corporate Information Sources Used by Knowledge Workers at Least Weekly
Corporate Operational Systems
4.7
Corporate Unstructured
4.2
Use External Sources of Information in Day-to-Day Work Percentage of Knowledge Workers
24% No
76% Yes
n = 4,941 knowledge workers. CIO Executive Board IT Practice www.cio.executiveboard.com
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
27
The proliferation of information users, uses, and sources drives new usability problems.
28
INFORMATION EVERY WHICH WAY Usability Challenges
More Information Users
More Information Uses
Executives
Customer value analysis
Line managers
Risk management
Analysts
Mobile presence
Frontline staff
Executive dashboards
Customers and suppliers
Automated decisions
1
Inadequate Tools—No single analytic tool can support all needs and skills levels.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Internal
External
2
Inflexible Reports and Cubes— Predefined reports cannot answer all knowledge worker questions while cubes are slow and offer limited dimensions.
3
Centralized Ad Hoc Reporting Inefficiencies—Relying on a central group to run custom queries creates lags in analysis time and can be an inefficient use of resources.
Flexible, Transparent Information Access
More Information Sources
CXO KPI Identification
4
Information Quality Issues— Quality varies significantly by information source in ways that are not always transparent to the user.
The IT Skills
Appendix
■■
To avoid a free for all, the choice should be shaped by a deep understanding of the business questions knowledge workers need to answer.
USABILITY = Bounded FLEXIBILITY AND TRANSPARENCY
Capability-Driven Tool Portfolio
Goal: High Uptake of Self-Service Analysis
High
Flexible and Transparent Analytics
Value Associated with the Approach
Usability can be improved by providing knowledge workers with greater flexibility in tools choice, information access, and use.
CXO KPI Identification Data Cubes
Ad Hoc EDW Queries by Specialist
Data Free for All
Dashboards Standard Reports Low One Size Fits All
Choice Shaped by Business Need Approach to Usability
1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
29
If knowledge workers don’t find the analytic tools provided by IT useful, they have an array of other choices.
■■
Users have choice (official or not) Use Analytic Tools That Are Not Supported by IT
Reasons for Using Analytic Tools That Are Not Supported by IT
Percentage of Knowledge Workers
Percentage of Knowledge Workers
The most common reason for using an unofficial tool is lack of an official alternative, the second most common reason is ease of use.
IT Does Not Provide an Alternative Tool
–– This is a 1.5 times more frequent reason than the need for a more powerful tool.
51%
32% Yes The Unofficial Tool Is Easier to Use
45%
68% No
The Unofficial Tool Is More Sophisticated
29%
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
30
Overcoming the Insight Deficit
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Greater choice in selecting analytic tools to use is a powerful driver of Insight IQ.
■■
Choice has Value Maximum Impact on Insight IQ1
The greatest increase comes from moving from one standard tool for the enterprise to standard tools aligned to a specific business unit or function.
∆ = 6.0%
Only Analytic Tool Available to Knowledge Workers Is Excel
∆ = 16.0% Knowledge Workers Must Use a Single Tool That Is Standardized Across the Organization
13.7%
7.7%
Knowledge Workers Must Use a Single Tool That Has Been Selected for Their Business Unit or Function
Knowledge Workers Can Choose from Multiple Analytic Tools Available
(8.3%) ∆ = 13.1%
(21.4%)
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
31
Greater tool choice doesn’t mean chaos but rather providing a portfolio of tools that meets the full range of capabilities needed by different knowledge worker communities.
■■
■■
■■
Rationalization programs often try to over-standardize the analytic toolset, leaving knowledge workers without needed capabilities and opening the door to unofficial tool adoption.
Overcoming the Insight Deficit
Not too Few, Not Too Many Three Approaches to Standards for Analytic Tools
Increases knowledge worker productivity
“Capability-Driven” Align the Tool Portfolio by Capability
Reduces redundancy while acknowledging diverse needs, preferences, and skills Provides principled criteria for future tool selection
ROI of the Tool Portfolio
Number of Tools in the Portfolio
Progressive organizations rightsize their portfolio of analytic tools by identifying and aligning with capabilities. IT leaders should avoid applying the same criteria to analytic tools that they use for other applications rationalization efforts, as the optimal number of analytic tools is likely to be higher than for other categories of application.
“Dictatorship”
“Anarchy”
Rationalize to One Tool
Ungoverned Tools
No single tool provides all capabilities. Rationalization expense erodes savings. Having only one tool may encourage use of unofficial tools.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
Proliferation of tools leads to uncontrolled licensing costs. Different tools are used by different teams for the same purpose. ROI dips as tools have limited usage or unexploited functionality.
CXO KPI Identification
The IT Skills
Appendix
32
Capability-Driven Tool Portfolio
Overview A methodology for selecting a set of analytic tools that collectively provide 10 key capabilities to knowledge workers Executive Teaching Focus on helping knowledge workers use the right tool for the right purpose rather than aiming to minimize the total number of tools. Identify the analytic capabilities most needed by knowledge workers, and use the capabilities to assess current and future analytic tools to identify gaps and redundancy in the tool portfolio. Company Snapshot Blue Cross Blue Shield of North Carolina Industry: Health Insurance 2010 Employees: 4,600
BlueCross and BlueShield of North Carolina (BCBSNC) provides health care insurance products and related services to about 3.7 million members in North Carolina. BCBSNC is a licensee of the Blue Cross and Blue Shield Association and was founded in 1933. The company’s health plans include Blue Care (HMO) and Blue Options (PPO), as well as consumer-directed plans. BCBSNC also provides dental, life, disability, long-term care, and Medicare supplemental insurance, as well as prescription drug coverage.
Source: http://www.hoovers.com. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
33
In an internal survey of tools used for analytic functions, BCBS NC learned that its portfolio of analytic tools is misaligned and duplicative and used for suboptimal purposes.
Overcoming the Insight Deficit
A Plentiful but Ill-Fitting Portfolio Blue Cross Blue Shield of North Carolina Analytic Tool Environment 2009 One Tool Fits All Teams use one tool (often Excel or Access) for all needs, missing the chance to use tools that better meet their needs. Everyone Gets Their Own Tool Multiple tools are used for the same purpose by different teams, resulting in wasteful duplication. A Screwdriver to Hammer in Nails Tools are used for purposes for which they are not optimized. Tools with Unfulfilled Potential Tools have with limited usage or unexploited functionality.
“We found that many teams were using the one tool they knew for as many functions as possible, while better tools went unused.” Todd Norris Director of IM Delivery BCBS NC CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
34
Define analytic capabilities needed by knowledge workers across tools, not functionality within tools.
■■
■■
■■
■■
BCBS NC defines 10 capabilities supported by analytic tools. The company surveys knowledge workers to determine which tools meet which capabilities. The capabilities are reviewed annually and adjusted according to evolving knowledge worker needs. See page 37 for an overview of the survey process and page 38 to 39 for a list of the survey questions.
Ten Analytic Capabilities Capability
Description
1. Data Collection/ Conditioning/Validation
Gathering, integrating, consolidating, modifying, and validating data
2. Static/Printable Reporting
Reports are specifically designed to be consumed as a printout. These reports have no “interactivity” and are typically scheduled to run on a regular basis or have a few short prompts for the user to enter.
3. Self-Service Reporting
Reports/queries are given to users to run on demand. They can be interactive with prompts and drill downs.
4. Ad Hoc Report Development
Reports are developed as part of exploration and are typically for a one time use. These may turn into self-service or printable reports.
5. Advanced Analytics/ Data Mining
Analysis typically done by a small number of analysts who look for new business insights by examining relationships of information and information trends, this includes data and text mining.
6. Data Visualization
Not limited to charts, this includes any functionality providing faster consumption of information through “pictures” or presentations other than data grids.
7. Dashboards/Scorecards
Dashboards—On-screen visualizations that provide a quick status of key performance indicators specific to a role Scorecards—Use colors, indicators, and numbers to compare actual performance of key metrics against a target or budget. These are slightly more detailed than dashboards and less visual.
8. Geospatial Analysis
This is an array information across a geography to show relationships between data and locations.
9. Operational Reporting
This is not used to analyze performance, but used for operational reporting. Examples include sending an invoice to a customer or a letter to a member.
10. Publishing and Delivery
Report distribution including e-mail, print and mail, or using a business intelligence (BI) tool to distribute through a built-in or custom portal for providers, members, or internal staff.
Nonspatial analysis and mobile BI will be added as emerging capabilities during the next refresh. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
35
Maintain a portfolio of tools that supports all 10 capabilities.
A Portfolio of ENTERPRISE and Specialty Tools
■■
BCBS NC identifies capability gaps and helps teams use the tools that best provide the capabilities they need.
Publishing and Delivery
Operational Reporting
Geospatial Analysis
Dashboards/Scorecards
Data Visualization
Advanced Analytics/ Data Mining
Ad Hoc Report Development
–– Specialty Tools support one capability in one particular function; another six to nine of these tools may be needed.
Data Collection/ Conditioning/Validation
–– Enterprise Tools support multiple capabilities used for the same purposes across the company; between three and five of these tools should support most capabilities.
Self-Service Reporting
Tools are designated in one of two categories:
Static/Printable Reporting
■■
Overcoming the Insight Deficit
Enterprise Tools (3 to 5) BObj Web Intelligence SAS Core Crystal Reports
Speciality Tools (6 to 9) “While we will always retire tools where the ROI isn’t there, we are more concerned about helping people use the right tool for the right purpose and get away from using one tool for everything.”
Amisys MACESS Teradata SQL Assistant MCSource EDIWatch
Todd Norris Director, IM Delivery BCBS NC
Capability gaps
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
36
The process of capturing analytic capabilities by surveying knowledge workers requires executive sponsorship and clear communication.
IMPLEMENTATION GUIDE: CAPABILITY SURVEY ROADMAP Keys to Successfully Surveying Knowledge Workers on Their Capability Requirements
Only key analytic areas identified by this group are surveyed.
1. Obtain sponsorship and direction from VP–level information governance group.
Representatives are told that each survey should take less than an hour per analytic area to complete.
2. Identify user representatives in each division to administer the survey.
3. Hold an orientation session to make sure knowledge workers understand the intent of each question. “The process of surveying was just as important as the questions we asked. The survey was not intended to capture perfect information but to give informed guidance for next steps.” Susan Helm-Murtagh VP, Information Management and Analytic Services BCBS NC
4. Follow up surveys with an interview with each group to validate results and ask questions. Three Weeks for Entire Process
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
37
Overcoming the Insight Deficit
38
IMPLEMENTATION GUIDE: Capabilities Survey I. Tools and Technology 1. Indicate the tools used in your area, by capability.
2. N umber of people proficient with each tool and proficiency level (1 = general understanding of the tool; 5 = expert level)
X X
X
2 1
3
4
5
Other 2
Other 1
Publishing and Delivery
Feedback on BI Technical Skills:
X
X
Other 1 X Other 1 X Any other feedback on the tools used in your area Any other feedback on the tools used in your area
1 2
Tool 3
Tool B Tool C
Tool Tool 1 Tool 2 Operational Reporting
Geospatial Analysis (mapping)
X
Visualization
X
X
Dashboards/ Scorecards
X
Advanced Analytics/ Mining
X
Self Service
X
Ad hoc Reporting
Tool A
Static/Printable Reporting
Tool
Data Collection/ Conditioning
BI Capabilities
X
X
X
3. C omment on tools that are particularly critical to your area or feel free to mention tools you feel, if improved, could save you time or provide greater information insights. 4. C omment on activities you do in your area that you feel should not be your responsibility or you feel your team is not adequately trained to support.
II. Effort 5. Number of people who do analytical or BI development work.
6. Percentage of time spent by high-level activity
Number of full-time equivalent employees in your area
Collecting and assembling data for analysis
20%
Conditioning, filtering, or validating data for analysis
20%
Developing new reports, report solutions, or analysis
20%
How many FTE resources would you need to fully satisfy the requests of your team?
9 9
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
IMPLEMENTATION GUIDE: Capabilities Survey 7. O f the time your area spends collecting and assembling data, list the percentage spent on each activity. Activity
8. O f the time spent conditioning and filtering information, how do you spend that time? %
Activity
%
Pulling information from various sources
50%
Developing/applying filtering and rules unique to your needs
90%
Manipulating information in spreadsheets/MS Access/SAS
50%
Developing/applying filtering rules than are/could be used by others
10%
100%
Total
Other 1 Total
100%
9. L ist the main sources of information you use or plan to use in the near future.
10. How does your report solution development time breakout?
Source
Type of Information
Activity
Source 1
Claims, membership, provider info
Time spent developing a specific report for a specific question
20%
Source 2
Call center information
Time spent creating a solution to answer many questions
80%
Source 3
Call center forecasting, staffing models
Total
100%
%
Critical (1–5)
Supported (X)
Support Hours
III. Applications and Solutions
Solution 1
5
X
20
Solution 2
4
X
2
Solution 3
5
Solution Name
11. In the grid to the left enter the following information: ■■ Solution Name: type in a solution name. ■■
Critical: 1 = nice to have; 5 = mission critical in your area
■■
Supported: put an X if the solution is supported by your area
■■
Support Hours: monthly hours spent supporting the solution
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
39
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
40
■■
To avoid a free for all, the choice should be shaped by a deep understanding of the business questions knowledge workers need to answer.
USABILITY = Bounded FLEXIBILITY AND TRANSPARENCY
Capability-Driven Tool Portfolio
Goal: High Uptake of Self-Service Analysis
High
Flexible and Transparent Analytics
Value Associated with the Approach
Usability can be improved by providing knowledge workers greater flexibility in tools choice, information access, and use.
CXO KPI Identification Data Cubes
Ad Hoc EDW Queries by Specialist
Data Free for All
Dashboards Standard Reports Low One Size Fits All
Choice Shaped by Business Need Approach to Usability
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
41
Less than one-half of all knowledge workers find standard reports and dashboards useful.
■■
Overcoming the Insight Deficit
REPORTING THE END OF THE REPORT Reports and Dashboards That Are Pushed1 Are Valuable Percentage of Knowledge Workers
Standard reports rarely contain the right information depicted in the right way to be to useful knowledge workers as they make decisions. 51% No
49% Yes
n = 4,941 knowledge workers. 1
Information supplied in standard dashboards and reports at a predefined frequency, often distributed electronically through e-mail.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
42
The proliferation of information sources increases the value of information visualization, aggregation, and filtering.
■■
Putting it All Together Maximum Impact on Insight IQ1 of Information Presentation and Analysis Capabilities
14.9%
These capabilities can be defined as follows:
13.6%
–– Visualization: the graphical depiction of information
11.6%
∆ = 2.1x
–– Aggregation: the combination of information from multiple sources –– Filtering: the selective omission of information to improve relevancy and legibility
5.5%
Information Visualization Capabilities
Information Aggregation Capabilities
Information Filtering Capabilities
Advanced Analytics Capabilities
n = 4,941 knowledge workers. 1
CIO Executive Board IT Practice www.cio.executiveboard.com
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
43
It is nearly impossible to uphold all dimensions of quality in all information sources.
■■
■■
Organizations with less than perfect information quality have two options: hide information until it is fixed or use information selectively in its current state. See pages 74–75 for full definitions and tips for measuring each of the nine dimensions of information quality.
Overcoming the Insight Deficit
44
Can All Information Be Perfect All the Time? Common Dimensions of Information Quality
1. Usability—Information is presented in a usable format.
2. Precision—Information is sufficiently granular.
3. Currency—Lag time before updated information is accessible.
4. Accuracy—Information accurately reflects the real world.
5. N on-Duplication—Information is not duplicated.
6. T imeliness—Information is available when needed.
7. Validity—Information conforms to business rules.
8. Consistency—The information conforms to its definition.
9. C ompleteness—Information fields are complete.
Information quality has many dimensions; few information sources comply with all of them.
Option 2: Help Knowledge Workers Use Information in Its Current State
Option 1: Avoid Using Information Until All Quality Problems Are Fixed “Perfect” quality requires costly multiyear effort.
Extracts value from information faster Helps prioritize most important quality fixes
External information quality may be unfixable.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Despite the high impact on Insight IQ, only about one-third of knowledge workers trust information from other functions in the organization.
■■
■■
■■
■■
Trust No One Maximum Impact on Insight IQ1 of Knowledge Worker Confidence in Information Quality
Have Confidence in Information Provided by Other Functions Percentage of Knowledge Workers
25.2%
This lack of confidence translates into perceived low information quality and usability.
32% Have Confidence in Information from Other Functions
The most trusted function is Finance, followed closely by Legal, with 47.5% and 43% knowledge workers willing to trust information from these functions, respectively.
16.5%
Marketing and R&D functions were the least trusted functions, with only about 25% knowledge workers willing to trust information from these functions.
Average for All Top 10% Drivers
68% Don’t Have Confidence in Information from Other Functions
Knowledge Worker Confidence in Information Quality (Cleanliness and Accuracy)
See page 73 for additional details on how each function assesses the quality of its own and other functions’ data. n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
45
Lack of transparency in how information is collected, stored, or analyzed is the number-one reason why knowledge workers lack confidence in the quality of available information.
■■
Knowledge workers need clarity on where information came from and how it has been transformed.
Overcoming the Insight Deficit
Trust From Transparency Reasons for Lack of Confidence in Information Quality Provided by Other Functions Percentage of Knowledge Workers
Lack of Transparency in How Information Is Collected, Stored, or Analyzed
60.0%
Multiple Versions of the Same Information Exist
Lack of transparency is a more significant reason for lack of confidence than the underlying information quality problems.
42.0%
Lack of Common Vocabulary/ Standard Definitions
23.8%
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
46
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Flexible and Transparent Analytics Overview An approach to information management and architecture that provides the flexibility and transparency knowledge workers need to conduct the analysis they require within a universe of information related to key business performance questions Executive Teaching A lack of understanding of information sources, definitions, quality, and relationships makes it hard for knowledge workers to assemble the information they need to address key business performance questions. The diversity of knowledge worker information needs means that it is not possible to support them with predefined reports and cubes and central support for ad hoc queries is slow and resource-intensive. Instead IT should promote analysis freedom and confidence by offering a flexible architecture and transparency into underlying information sources. Component Teachings
1. Business Performance Question Mapping—Identify the universe of information knowledge workers require by mapping the business performance questions they need to address.
2. Flexible Data Analysis Architecture—Enable flexible, on-the-fly analysis capabilities through in-memory architecture organized around key business questions. 3. Contextualized Information Access—Provide knowledge workers with access to information organized by their top business questions. 4. Data Quality Transparency—Provide transparency into data quality and source to help knowledge workers decide when the data can and can’t be used. Company Snapshot GlaxoSmithKline plc Industry: Pharmaceuticals 2010 Sales: US$43.9 B Employees: 98,485
GlaxoSmithKline (GSK) is one of the world’s leading pharmaceutical firms. GSK produces medicines that treat major disease areas such as asthma, anti-virals, infections, mental health, diabetes, cardiovascular, and digestive conditions. In addition, the company is a leader in vaccines and is developing new treatments for cancer. GSK also markets other consumer products, many of which are among the market leaders.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
47
A knowledge worker survey revealed high dissatisfaction with the amount of time wasted trying to assemble data provided in disparate reports and dashboards.
■■
■■
GSK’s North American Sales and Marketing IT department set out to understand the lifecycle of information delivery to support business processes. The results revealed a need for better understanding of key business questions to improve the speed and flexibility of analysis capabilities in the sales team.
Overcoming the Insight Deficit
Assembling Data is Very Time Consuming Knowledge Worker Data Satisfaction Survey Results Illustrative Results Shown
Knowledge Worker Pain Points
Director Area Sales
Director Sales Support
Product Director
Field Sales Specialist
Field Sales Manager
1. Unanswered Questions 2. Missing Data 3. Can’t Access Data 4. Tools Aren’t Useful 5. Slow Response to Data Needs 6. Data Inconsistency and Inaccuracy 7. Need More Historical Data 8. Waste Time Assembling Data
No Expressed Problem in This Area
The most serious pain point was the amount of time knowledge workers spent assembling data before they could conduct analysis.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
I Experience This Problem, However It Does Not Have a Significant Impact on My Work I Experience This Problem, and It Has a Significant Negative Impact on My Work
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
48
Reducing wasted time assembling data demands improvements to data mapping, flexible access, and metadata transparency.
Why Knowledge Workers Waste Time Assembling Data
Assembling Data Is Very Time Consuming
Challenge
Root Causes
“Knowledge workers wanted to assemble and manipulate data to explore cause and effect, likelihood of recurrence, and predictive analysis, but in our old model that meant intervention by IT. We needed to find ways to let them do this themselves.”
Asking knowledge workers what data they need misses how the data should be assembled.
1 Solution Components
2
Business Performance Question Mapping
Knowledge workers don’t know which combination of data will answer their business questions.
Predefined reports and cubes answer a narrow range of business questions.
3
Flexible Data Analysis Architecture
Knowledge workers don’t understand the data sources’ quality, definitions, and business rules.
4 Contextualized Information Access
Metadata Transparency for Power Users
Rob Jester Director, Enterprise Business Architecture, Sales and Marketing GSK CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
49
To identify the universe of information knowledge workers require, start by isolating the top business performance questions they need to answer.
■■
■■
Asking knowledge workers what information they want leads to a siloed approach that produces information that requires further aggregation. Instead, the Information Architecture group examines the business questions knowledge workers need answers for; from there, they can identify the combination of information subjects and sources that are necessary to answer these questions.
Overcoming the Insight Deficit
First, Ask the Right Questions
1. Identify Top Business Performance Questions
Business surveys and focus groups lead to identification of top 400 business questions. Example question: What prescription details help to predict likelihood to fulfill prescriptions?
IT works with business stakeholders to group the 400 questions into 20 categories.
2. Group Closely Related Questions
Example question groups: prescription fulfillment
3. Map Questions to Data Subject Areas Qu.1 Qu.2 Qu.3 Qu.4 Qu.5 Qu.6 RX
Information architects combine business questions and data knowledge to develop a detailed Business Subject Area Data Map that documents the relationships between the questions and the data subject areas. Example subject areas: prescriber, prescriptions
Prescriber Claims Patient Product
“To understand how knowledge workers needed to assemble data, we first had to know the questions they were asking.” Rob Jester Director, Enterprise Business Architecture, Sales and Marketing GSK
Information architects identify related data sources.
4. Identify Related Data Sources
Example data sources: patient database, sales database, EDW, prescriber database CRM ERP
EDW HR
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
50
Help knowledge workers answer business performance questions by equipping them to assemble, analyze, and visualize information from multiple sources as they need it.
Flexible Analysis Architecture Data Analysis Tool Architecture Illustrative
Presentation Layer
3 Users can access and segment data through high-quality visualizations to get fast answers to their top business questions.
Flexible Analytical Layer
2 Relevant data is moved into the in-memory database of the analysis tool to enable fast processing across all possible data dimensions.
In-Memory Architecture
Data Sources
1 Data sources are selected based on the business questions documented by the data map.
Prescriber DB
Rx Sales DB
CRM
Enterprise Data Warehouse
Note: GSK uses QlikView as its reporting tool. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
51
Provide an online data map to help knowledge workers understand information availability and interrelationships.
■■
■■
■■
The data map provides an overview of available information and shows the connections between information.
SHOW HOW IT ALL FITS TOGETHER Clickable Data Map Aligned to Business Subject Areas Illustrative
Business Subject Areas: Sales and Marketing
Claims
Providing information access organized by business subject areas makes it easy for knowledge workers to identify the most relevant information for their question or role.
Patient
Payer
Rx
Detailed data concept models, fact sheets, and information flow charts lie under each business subject area, available for heavy analytics users in the organization.
Prescriber
Formulary
Business
Order
Contracts Marketing Events
Call Activities (Sales)
Visualizing Data Relationships—A product manager can visualize how claims data links back to Rx data through patient data, to help understand how to conduct analysis. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Flexible, Transparent Information Access
Non-Retail
Clicking on a business subject area provides additional information about the data quality, sources, and relationships.
CXO KPI Identification
The IT Skills
Appendix
52
Provide full transparency into information sources, flows, and quality to increase usefulness and knowledge worker confidence in the information.
■■
■■
FULL TRANSPARENCY MAKES DATA MORE USABLE Centralized Metadata Portal Illustrative
Knowledge workers can view this information through the data map.
GSK Connect Welcome to Data Point Home
Metadata and additional information models and training is available for IT and power users in the business.
Data Glossary
Subject Areas
Standard Reports
Data University
Prescription (Rx) Sales Data Subject Area Description: Metrics (business facts) related to both non-retail sales of drug products and retail prescription fills. Retail prescription data are collected at the national and subnational level.
Quality Transparency—A product manager can review the Rx Data Fact Sheet to learn about the accuracy of the data and become aware of any caveats in how it should be used for analysis.
A
Data Fact Sheet—Provides description of data types, sources, granularity, and quality levels Information Flow—Visually depicts flow of information from original source to reporting tools
?
Data Concept Model—Visually depicts logical data model for national prescriptions Data Owner—Click Here to contact.
Users who need more detail or clarification can easily contact data owners.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
Users can access training materials and information about registering for upcoming training.
The IT Skills
Appendix
Enable Information Usability
53
By providing the tools and contextual information needed for self-service, GSK reduces the time knowledge workers spend assembling information.
Overcoming the Insight Deficit
Faster Assembly, Less Help Required
Early Benefits Claims
Patient
Payer
Rx
IT and the rest of the business share an enterprise-wide view of customer data needs.
Example: Time to Complete Complex Analysis on Rx and Claims Data 30 Days
Simplified access to information 87% 4 Days
Reduction in time spent generating reports Traditional Approach
New Approach
The new approach eliminated the need to create 65 reports when analyzing patterns in the relationship between prescriptions and claims.
“Sales and Marketing have always had strong demand for information on a short turnaround. Now we are equipping them to get the information they need faster and with much less effort.” Rob Jester Director, Enterprise Business Architecture, Sales and Marketing GSK CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
54
Data Fact Sheets provide power users with metadata details to support their understanding of the data sources and quality levels.
Implementation Guide: Data Fact Sheet
Data Fields on the Data Fact Sheet ■■
Data Area Name
■■
Integration Requirements
■■
Data Area Type
■■
Dimension Requirements/Gaps
■■
Data Subarea
■■
Data Anomalies
■■
Data Description
■■
Data Security Requirements
■■
Data Objective
■■
Target Delivery Requirements
■■
Data Source/Vendor
■■
Current Usage
■■
Data Granularity
■■
Disaster Recovery
■■
Facts/Metrics
■■
Data Quality Requirements
■■
Source Delivery Mechanics
■■
Date Entered
■■
Source Delivery Frequency
■■
Data Architect
■■
Refresh Strategy
■■
Reference Documentation
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
55
Analytical power users can attend a Data University to deepen their understanding of data sources, relationships, and usability.
■■
■■
Overcoming the Insight Deficit
Implementation Guide: Data University Course Schedule Day One
Course
9:00 a.m.–9:10 a.m.
Welcome, Introductions, and Course Overview
GSK’s training group facilitates the training program, with data architects speaking throughout the sessions to provide subject matter expertise.
9:10 a.m.–9:30 a.m.
Data Use, Disclosure, and Confidentiality
9:30 a.m.–9:50 a.m.
Customer Dimensions
9:50 a.m.–10:05 a.m.
Break
10:05 a.m.–10:25 a.m.
Alignment Dimensions
10:25 a.m.–10:45 a.m.
Product Dimensions
Sessions focus on understanding the sources of data and relationships between information, enabling knowledge workers to independently apply judgment on data usage.
10:45 a.m.–11:05 a.m.
Static Views/Goals Data
11:05 a.m.–11:15 a.m.
Introduction: Retail
11:15 a.m.–11:45 a.m.
Retail, Dynamic Claims Lifecycle, and Consumer Rx data
11:45 a.m.–12:30 p.m.
Lunch
12:30 p.m.–1:50 p.m.
Retail, Dynamic Claims Lifecycle, and Consumer Rx data (Continued)
1:50 p.m.–2:15 p.m.
State Data Restrictions
2:15 p.m.–2:40 p.m.
Monthly Retail (IRx, PRx)
2:40 p.m.–3:10 p.m.
Weekly Rx
3:10 p.m.–3:25 p.m.
Break
3:25 p.m.–4:00 p.m.
Prescriber Track/Passport
4:00 p.m.–4:20 p.m.
CRx (Consumer Rx)
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
56
Implementation Guide: Data University Course Schedule (Continued) Day Two
Course
9:00 a.m.–9:05 a.m.
Welcome/Introduction
9:05 a.m.–9:35 a.m.
Targeting
9:35 a.m.–10:30 a.m.
Formulary/Payer Dimensions
10:30 a.m.–11:00 a.m.
Break
11:00 a.m.–11:25 a.m.
Call Activity
11:25 a.m.–11:30 a.m.
Non-Retail Introduction
11:30 a.m.–12:30 p.m.
IMS Health: Non-Retail Data
12:30 p.m.–1:15 p.m.
Lunch
1:15 p.m.–1:35 p.m.
Non-Retail Data
1:35 p.m.–1:55 p.m.
Specialty Sales Request System (SSRS)
1:55 p.m.–2:20 p.m.
Non-Retail Reports
2:20 p.m.–2:25 p.m.
National Data Introduction
2:25 p.m.–3:20 p.m.
IMS Health: National Audits Data
3:20 p.m.–3:30 p.m.
Break
3:30 p.m.–4:00 p.m.
IMS Health: National Audits Data (Continued)
4:00 p.m.–4:20 p.m.
National Audits Data from IMS Health
4:20 p.m.–4:50 p.m.
Dashboard Reports
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
57
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
58
■■
To avoid a free for all, the choice should be shaped by a deep understanding of the business questions knowledge workers need to answer.
USABILITY = Bounded FLEXIBILITY AND TRANSPARENCY
Capability-Driven Tool Portfolio
Goal: High Uptake of Self-Service Analysis
High
Flexible and Transparent Analytics
Value Associated with the Approach
Usability can be improved by providing knowledge workers greater flexibility in tools choice, information access, and use.
CXO KPI Identification Data Cubes
Ad Hoc EDW Queries by Specialist
Data Free for All
Dashboards Standard Reports Low One Size Fits All
Choice Shaped by Business Need Approach to Usability
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
59
The traditional approach to defining key performance indicators (KPIs) fails to meet CXO and operational needs simultaneously.
■■
■■
■■
Rolling up metrics from operations requires extensive systems and information harmonization and may still miss external or new information that is not used for operations.
Overcoming the Insight Deficit
Rigid Roll-Ups, Cumbersome Cascades Methods for Defining KPIs
Traditional Approach 1 Roll Up from Standardized Operational Metrics
Traditional Approach 2 Cascade Down from KPIs Aligned to Strategy
CXO: Reframe Strategic Questions Operational: Map Workflow Decision Points
Operational Metrics Misses external information required for strategy but not operations
Misses nonstrategic metrics required to manage operations
Identifies 18 to 20 CXO KPIs (estimated 50% overlap with operational metrics)
Takes too long and requires excessive harmonization of operations and/or systems
Requires ongoing modification of operational metrics as strategies change
Creates flexibility for CXO KPIs and stability for operational metrics
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Alternative Approach Differentiate CXO KPIs and Operational Metrics
CXO KPIs
Cascading down from strategy may be too narrow, as some operational metrics are needed that do not directly drive strategic outcomes. Progressive organizations recognize that neither approach is ideal and instead pursue differentiated approaches to CXO KPIs and operational metrics.
60
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
CXO KPi Identification
Overview A methodology to segment CXO and operational information needs to target the appropriate level of information depth, quality, and refresh rates for each EXECUTIVE TEACHING CXOs’ information needs differ from operational management needs. Executives require a broader perspective, need speed more than information quality, and have frequently changing needs; operational information needs are narrower, deeper, and less dynamic. Consequently, a single KPI framework and a single approach to information management will not meet the needs of both. Company Snapshot PepsiCo, Inc. Industry: Annual Revenue: Employees:
Food and Beverages US$60 B 294,000
PepsiCo offers the world’s largest portfolio of billion-dollar food and beverage brands, including 19 different product lines that generate more than $1 billion in annual retail sales each. PepsiCo’s main businesses—Quaker, Tropicana, Gatorade, Frito-Lay, and Pepsi Cola—also make hundreds of other enjoyable and wholesome foods and beverages that are respected household names throughout the world. With net revenues of approximately $60 billion, PepsiCo’s people are united by the company’s unique commitment to sustainable growth by investing in a healthier future for people and the planet, which the company believes also means a more successful future for PepsiCo.
Source: http://www.hoovers.com. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
61
As PepsiCo migrates from country-level to Europelevel management, its information and reporting needs change.
■■
Overcoming the Insight Deficit
Consolidated Information for a Consolidated Business PepsiCo Migration from Country-Level to European Sector Organization
The current reporting environment differs by country, making it difficult for Europe-level management to get the information they need.
Sector-Level Information Needs and Challenges
Objective
Italy
Spain
France
Russia
Identify the information needed to manage the sector. Sample Sector-Level Management Questions ■■
European Sector and Global Functions ■■ ■■ ■■ ■■
Procurement Finance Commercial Treasury
■■ ■■ ■■
■■
Planning R&D1 Food Technology1 HR1
■■ ■■
Country-Level Reporting Challenges ■■
■■
“Business and markets have changed, and following the recession historical information has lost its value, so we should forget about reporting and start thinking about information.” Austin Wildmore Enterprise Architect PepsiCo
■■
1
5,000+ applications and thousands of country-level reports Global chart of accounts but local definitions Up to eight weeks to aggregate metrics across Europe
Global functions.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Where do we get the best returns on trade promotion spend? Which distributors are most profitable for us? How can we consolidate transport costs across Europe?
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
62
The information needs of CXOs and operational managers are fundamentally different.
■■
■■
Executives require highlevel strategic information, and they tend to focus only on one or two objectives, which change frequently.
Diverse and Dynamic versus Focused and Stable
CXOs Role Characteristics
Operational managers require consistent, reliable, and deep drill downs and their metrics are relatively stable.
■■ ■■
■■
■■
Information Needs
CIO0361911SYN
■■
Sequential interest is focused on one or two priorities at any given time.
■■
Cross-cutting priorities affect multiple functions or BUs.
■■
Priorities change with strategy.
■■
Manage operational performance. Parallel interest have a portfolio of priorities. Focused responsibilities lie within one function or business unit. Priorities are stable.
Broad Perspective—Information from Diverse Internal and External Sources
Information from Limited Sources
Information Needs Change Frequently
Information Needs Are Stable
Where Necessary, Speed to Insight Over Quality and Depth
Quality and Depth Are Critical
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction
Business Unit and Functional Managers
Manage strategy, not operations.
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
63
IT should frontload its involvement in reporting by helping to reframe strategic questions to make the business outcomes measurable.
■■
■■
PepsiCo’s enterprise architects partner with IT business relationship managers and transformation directors in Finance to provide guidance during the annual strategic planning cycle. Their goal is to define make strategic objectives measurable.
Overcoming the Insight Deficit
Reframe Strategic Questions to Identify CXO KPIs Process for Identifying CXO KPIs Illustrative 1. Frontload Participation in Strategic Discussions ■■
■■
Enterprise architect supports information collection for annual refresh of three-year strategic plan. Proactive outreach anticipates information requests.
2. Reframe Strategic Questions to Make the Business Outcomes Measurable Two reframing questions ■■
■■
What business outcome are we trying to achieve? What can we measure to ensure we are making progress on that business outcome?
Original Strategic Goal Grow our business in Russia. What Business Outcome Are We Trying to Achieve? We want to become the market leader by exploiting production, distribution, and marketing synergies between our soda, snack, and juice businesses. What Can We Measure to Ensure We Are Making Progress on That Business Outcome? ■■
■■
3. Identify and Collect Information Needed ■■
“We are not trying to define strategy, but unless we can help make the strategy measurable we can’t provide the information the company’s leadership needs.”
4. Stop Measuring Objectives That Are Removed During the Strategy Refresh
■■
■■
Percentage of total sales consolidated with a single distributor in each region Percentage of total trade promotion spend directed to a single distributor in each region Percentage of revenue generated through a shared sales force Percentage of juice products carried by our soda distribution fleet Market share by region
Austin Wildmore Enterprise Architect PepsiCo CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
64
The CXO KPIs require an approach to information management that prioritizes speed and flexibility over quality and depth.
■■
Use the KPIs as a guide to avoid overinvesting in enterprise-level information standards or reports.
CXO KPIs Require a Different Approach to Information Management Implications of CXO KPIs for Information Management
Broad Perspective
1. Prioritize Sector-Level Information Standards of Enabling CXO KPIs Most information does not need to be harmonized across countries or functions. The CXO KPIs indicate where harmonization is needed. 2. Integrate Internal and External Information CXO KPIs often combine internal and external information.
Frequently Changing Needs
3. Retire Old Reports Reports fall out of use as strategies evolve so review and retire regularly. 4. Avoid Overinvesting to Meet Short-Term Needs CXO priorities change, so avoid overinvestment in gathering information that may only be needed for a year or so.
Speed and Brevity
5. Manual Effort Is Okay The need for quick information often requires the data to be pulled together manually. For short-term needs, automation may not be justified. 6. Drill-Downs Only by Exception XOs rarely need drill-downs—save time and effort by providing drillC downs on request rather than enabling them proactively.
“A lot of important information is now external. You need to approach the data with a specific question; you can’t try to aggregate it all.”
7. Accept Gaps in System Architecture Incomplete information is sufficient if it avoids delays. Set information management priorities around filling in the gaps.
Austin Wildmore Enterprise Architect PepsiCo CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
65
Highlight information gaps to drive to consensus on priorities for systems harmonization or integration.
■■
To establish priorities for change, PepsiCo enterprise architects use a model of core business components overlaid with system information to demonstrate to business partners the sources of gaps in CXO information needs.
66
Highlight (Don’t Hide) the Gaps Analysis of Sources of Information Gaps Illustrative
Russian Distributor Consolidation Report
Component Business Model—Russia To accurately measure sales through a common distributor and the reuse of distribution assets, we need to integrate operational systems for soda and snacks, and fill functionality gaps in our commercial and supply chain systems. Soda
Snack Foods
Juice
Dairy
Finance
Supply Chain
Commercial
Operations
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Vendor A
Vendor A
Vendor B
Vendor B
Vendor A
Vendor A
Vendor C
Vendor B
Gap
Vendor C
Gap
Unintegrated System
Flexible, Transparent Information Access
Vendor D
Unintegrated System
CXO KPI Identification
The IT Skills
Appendix
Identify operational metrics by identifying the information needed at key decision points in business processes.
MAP OPERATIONAL WORKFLOW TO FIND DECISION POINTS Decision Points in the Develop and Manage Sales Plans Process Illustrative
■■
■■
While CXO KPIs are identified by reframing strategic questions, PepsiCo takes a different approach to defining operational metrics used by managers below CXO level. These operational metrics are linked to decision points in business processes.
Process
Workflow
Decision Points
Information Needs
Manage Customer Sales
Manage Customers and Accounts
Perform Pre-Sales Activities
Manage Sales Orders
Perform Sales Calls
■■ ■■
Record Outcome of Sale
Decide on Discount Level
Decide Which Prospects to Call
■■
Close the Sale
Expected value Gap to quarterly goal Product availability
■■
■■
Average discount offered in quarter Previous prices paid by prospect
Source: “Process Classification FrameworkSM,” APQC, September 2009, http://www.apqc.org/knowledge-base/documents/apqc-process-classification-framework-pcf-cross-industryexcel-version-51-0. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
67
By focusing narrowly on CXO information needs, PepsiCo International significantly cut the time needed to gather new information and the number of legacy reports.
Overcoming the Insight Deficit
Better Decisions, Faster Time to Respond to Requests for New Reports
Reduction in Report Volume Illustrative
8 Weeks 1. Immediate elimination of 1,000+ legacy reports 2. Ongoing discipline to refresh or retire reports
< 1 Week
2009
2011(E)
2009
2010
2011
2012
“By asking the right questions, we can now track progress against our goals in Russia despite imperfect information. In the past we were swamped with metrics, few of which were useful. Now we only get those that matter.” Simon Black European Commercial Transformation Director PepsiCo
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
68
■■
The Information Architect role is critical to IT’s ability to tie business questions to the data knowledge workers need, to enable flexible analytical capabilities.
Information Architect
We estimate that demand for information architects will increase by 250% by 2015, the largest increase of any role in corporate IT.
■■
New-to-World Skills
Key Responsibilities ■■
■■
■■
■■ ■■
Conducts meetings with senior IT leadership and key business stakeholders across the organization to understand information needs
■■
■■
Analyzes the objectives of information consumers
Information analysis (structured and unstructured) Information integration
Other Important Skills
Visualizes and documents the inter-relations between information needs, technology, and design Devises metadata schemas for classifying content according to business needs Involved in prototyping and usability testing
■■
Information architecture development
■■
Information policy formation
■■
Information modelling
■■
Taxonomy creation
■■
Stakeholder management
Works with business stakeholders and IT project teams to define and promote enterprise information standards
Possible Backgrounds and Hiring Sources Most likely to have a background with a consultant or vendor specializing in information management or information-centric products Sourcing Difficulty
Seniority
Middle Management
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
69
A user-experience specialist collaborates with service managers and end users to understand and improve user experience and workflow for new and existing applications.
User-Experience Specialist Key Responsibilities
■■
■■
The role designs and configures user-centric interfaces for in-house and cloud applications.
New-to-World Skills
■■
Analyzes business and functional requirements
■■
Information visualization
■■
Creates user-centered design
■■
User-behavior analysis
■■
Improves the user experience
■■
Usability design
Visualizes and presents information in a userfriendly manner to end users
Other Important Skills
■■ ■■
This enables end users to access, visualize, and navigate information and analytics with ease. The skills required in this role match the primary skill sets of business analysts, data analysts, and software developers.
■■
Agile development
■■
Business domain analysis
■■
Functional requirements analysis
■■
Information architecture development
■■
Requirements management
■■
Usability design
Possible Backgrounds and Hiring Sources Likely to have a specialist background in behavioral science, graphic design, or product design; more likely to be found in a consultant or other specialized role than in a corporation Sourcing Difficulty
Seniority
Individual Contributor
For full details of this role, see the user-experience guru job descriptions on the CIO Executive Board website. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Overcoming the Insight Deficit
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
70
Key Takeaways: Enable Information Usability
Make Usability a Core IT Competency: IT requires new skills and processes to improve the usability of information and analytic tools. Drive knowledge worker productivity by letting them choose the tools they find easiest to use, by offering greater transparency into quality levels, and by increasing their ability to navigate and analyze information from diverse internal and external sources. At Organizations with a High Insight IQ...
At Other Organizations...
1. Knowledge workers choose from a portfolio of tools that cover key capabilities.
One toolset is provided for all purposes.
2. Information presentation aggregation and filtering are top priorities.
Investment in advanced analytics is prioritized over presentation.
3. Knowledge workers can see all the information available to them and the relationships between that information.
There is no single list of all available information.
4. Information quality levels are transparent to knowledge workers.
Knowledge workers cannot ascertain the quality of information available to them.
5. KPIs are defined by reframing strategic questions.
KPIs are rolled up from operational metrics or cascaded down from strategy.
6. Speed to insight is prioritized when enabling CXO KPIs.
No KPIs are enabled until the underlying information is complete.
7. IT organizations invest in information architects and user experience specialists.
IT staff lack information architecture or usability skills.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Enable Information Usability
71
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
72
Knowledge workers are universally more confident about the quality of information from their own function than they are about information from other functions.
APPENDIX: Keeping Things Close to Home Average Knowledge Worker Confidence in Information Quality from Their Own Function and in Information from Other Functions 68%
High Confidence in Information from Own Function
62%
High Confidence in Information from Other Functions
56%
54% 48%
45%
44% 38%
35%
34% 27%
Legal
27%
Finance
34%
26%
R&D and Engineering
HR
Customer Service
Sales
Marketing
The IT Skills
Appendix
n = 4,941 knowledge workers.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
Enable Information Usability
73
Overcoming the Insight Deficit
Appendix: Definitions of information Quality Dimensions
Definition
Metrics (Illustrative)
Monitors (Illustrative)
Ease of Measurement
Measurement Comments
Timeliness
The relative ability of data to support a given process within the timetable required to perform the process
Days between product launch and support document publish date
Manual quarterly audit of lag time between new product launches and publication of supporting documentation
Medium
Like currency, this depends on data flow information to understand whether performance goals are met.
Validity
The degree of conformance of data to its domain values and business rules
Percentage of product codes in use corresponding to valid product codes
Automated cross-check of product codes in customer database against master product database to test code validity
Medium
It is usually fairly straightforward to create automated monitors to measure this characteristic.
Consistency
The conformance of the meaning of data with its data definition (also called Definition Conformance)
Match rate between internal and external product nomenclature
Automated monthly audit of match rate between all internal and external product names, performed by macro-comparing Excel fields for exact match
Medium
Not widely used; this often reveals a root cause of accuracy problems.
Completeness
The characteristic of having values for the data fields
Percentage of complete customer entries with complete e-mail information
Automated scan of customer database to measure percentage with existing, valid e-mail address entries
High
This is easy to measure and is recommended as a starting point to illustrate why monitoring matters.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
74
Appendix: Definitions of information Quality Dimensions (Continued) Definition
Metrics (Illustrative)
Monitors (Illustrative)
Ease of Measurement
Measurement Comments
Usability
The degree to which the information presentation is directly and efficiently usable for its purpose
Customer satisfaction with format and clarity of inventory data
Survey of business data customers to determine satisfaction with level of usability of data
Low
Measurement is usually done through some sort of customer satisfaction survey.
Precision
The characteristic of having the right degree of granularity in the data values
Percentage of deliveries scheduled as an hourly level of precision
Automated scan of delivery schedules to determine percentage with precision of one or two hours
Medium
This characteristic is not widely used within Intel, and problems usually surface in usability tests.
Currency
The lag time between when information is updated and when it is knowable in a redundant database
Lag time between inventory scan-in and updates to supply chain planning system
Automated measurement of time stamp differences between inventory scan-in and data refresh
Medium
Measuring this requires information about the flow of information—server, ETL logs, etc.
Accuracy
The degree to which the data accurately reflects the real-world object or event being described
Percentage of active employee records with accurate home address information
Manual audit conducted (by calling or e-mailing a sample of employees) to assess accuracy of home address on file
Low
This is the hardest characteristic to measure automatically; usually requires manual audits.
Nonduplication
The degree to which a one-to-one correlation exists between records in a dataset and real-world object or events
Percentage of products listed more than one time in product database
Automated cross-check of product databases to determine duplicate entries
High
This characteristic is very easy to measure and can uncover basic problems in business processes.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction CIO0361911SYN
Tools Portfolio
Flexible, Transparent Information Access
CXO KPI Identification
The IT Skills
Appendix
Enable Information Usability
75
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
76
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
77
Unquestioning faith in analysis can be as damaging as overreliance on intuition.
Overcoming the Insight Deficit
Why Balancing Analysis With Intuition Matters
Risks from Overreliance on Intuition ■■
High
■■
Reliance on Intuition
■■
Unable to account for complex environments Highly subject to personal biases Hard to replicate, scale, or use to foster buy-in
Risks from Overreliance on Analysis ■■
■■
■■
May hinge on incorrect assumptions or inaccurate data Unable to adapt to discontinuous change Fails to account for situational context by looking at the data too narrowly
Low Low
High Reliance on Analysis
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
78
Knowledge workers fall into three categories based on their relative reliance on analysis and intuition.
■■
■■
Informed Skeptics like to make biases and assumptions explicit and often question their own thinking; they listen to alternative points of view without defaulting to consensus-based decisions. The most common profile is the Unquestioning Empiricist, who tends to rely excessively on analysis, so in many organizations the greatest risk comes from too much analysis, not too little.
Visceral Decision Maker 19% of the sample Average Insight IQ1 = 45
High
■■ ■■ ■■
Seldom trusts analysis Has weak analytic skills Makes decisions unilaterally ■■
Reliance on Intuition
■■
The Value of Big Judgment
■■ ■■
Informed Skeptic 38% of sample Average Insight IQ1 = 60 Applies judgment to analysis Has strong analytic skills Listens to others and is comfortable with dissent
Visceral Decision Makers are more rare but represent almost one-fifth of knowledge workers.
■■
■■
■■ ■■
The profiles are determined using cluster analysis to identify common groups of analytic skills and behaviors.
Unquestioning Empiricist 43% of the sample Average Insight IQ1 = 48
Trusts analysis over judgment Supplements analytics with precedent Values consensus
Low Low
High Reliance on Analysis
n = 4,941 knowledge workers. 1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
79
Informed Skeptics are most common among the more senior ranks.
Overcoming the Insight Deficit
Learned Skepticism Knowledge Worker Profile Distribution Percentage of Knowledge Workers
■■
■■
While knowledge worker profiles vary substantially by seniority, there is little difference by organizational tenure, indicating that institutional knowledge is not a distinguishing factor.
Unquestioning Empiricists Visceral Decision Makers
22% 37%
43%
47%
Informed Skeptics
9%
Similarly, education and extent of cross-functional experience do not matter. 14% 17% 21%
69% 50% 41%
32%
Individual Contributors
Midlevel Managers
Senior Managers
Executive Leadership
As knowledge workers gain experience, they increasingly rely on judgment to supplement data when making decisions. n = 4,941 knowledge workers. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Introduction
Informed Skepticism
Who Does What
Appendix
80
Unquestioning Empiricists represent at least one-quarter of knowledge workers in every function.
No Function Is Immune from Overreliance on Data Knowledge Worker Profile Distribution Percentage of Knowledge Workers
47%
57%
60%
Visceral Decision Makers are most heavily represented in Sales, Communications, and Customer Service.
30%
43%
30%
25%
19%
50%
51%
31%
12%
14% 18%
23%
16%
14%
18%
ns un ic G
en er al
M
m C om
58%
an ag em C en or t po ra te St ra te gy
47%
at io
39%
IT
37%
l
37%
Le ga
36%
ce
&D R
ar M
35%
na n
34%
29%
22%
g
48%
50%
39%
14%
18%
ke tin
41%
Fi
Unquestioning Empiricists most heavily reside in Marketing, R&D, and HR.
R
■■
25% 36%
H
■■
Informed Skeptics represent the majority in General Management and Corporate Strategy.
Sa Pr od le s uc t Pr ion oc / ur Op em er en s/ C us t to m er Se rv ic e
■■
Informed Skeptics
Visceral Decision Makers
Unquestioning Empiricists
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
81
Informed skepticism can be fostered through a combination of decision-making models, analytical skills training and coaching, and tools for collaborative decision making.
■■
■■
Some areas have little to do with IT, while others clearly require technology. Training and coaching represent “gray areas” of responsibility where CIOs should work with their business counterparts to determine ownership.
Overcoming the Insight Deficit
WHO DOES WHAT TO BUILD INFORMED SKEPTICS?
Activities and Responsibilities for Fostering Informed Skepticism
Clear Role for IT
Organizational “Gray Area”
Business Leadership Role (Unlikely to Involve IT)
Build Tools for Collaborative Decision Making
Provide Analytical Skills Training and Coaching
Define Decision-Making Models and Behaviors
Examples
Examples ■■
Build workflow tools that connect stakeholders involved in collaborative decision making.
■■
■■
Shift from up-front training on new tools to ongoing development of analytic skills. Hire and incentivize analytics staff on their effectiveness as coaches, not just analysts.
Examples ■■
■■
Introduce decision-making techniques that promote critical thinking and make biases and assumptions explicit. Embed analysis and collaborative decision making in competency models and performance criteria.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
82
Introduction
Informed Skepticism
Who Does What
Appendix
■■
Collaborative tools can enable more effective decision making.
IT Role: Tools for Collaborative Decision Making
Collaborative decision making, listening to others, and accepting dissent are characteristics of informed skepticism.
Essential Features of Collaborative Decision-Making Tools Wisdom of Crowds—Find and connect people across the organization to get broad input into ad hoc decisions.
Annotations and Note-Taking—Add context to data by annotating reports and integrate all comments in a note-taking platform.
Collaborative Workflow—Improve the quality of decisions by enabling group comments and conversations to reach consensus.
Integrated Workflow—Integrate information and collaborative processes into a single platform.
Information Freedom—Allow for a free flow of relevant and contextual information by moving data out of siloed e-mail platforms.
Decision Transparency—Develop a corporate decision repository to record discussions, information, and analyses around past decisions.
Sample Collaborative Decision-Making Tools
Do use collaborative decisionmaking tools to gather a wide range of opinions from experts across the organization.
EMC uses a collaboration platform from Jive to enable an internal idea-sharing and development community.
The UK Department for Work and Pensions uses collaborative tools from Spigit to create a virtual stockmarket where employees “invest” in ideas they believe have value.
DON’T use collaborative tools when speed is of paramount concern and one person’s expertise far exceeds that of others. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Macquarie University uses Yellowfin to improve information accessibility and research data visualization.
Source: MC Corporation, EMC|ONE: A Journey in Social Media, 2008; Dinham, Peter, “Yellowfin, Macquarie Uni Win Global BI Award” ITWire, 9 December 2011, http://www.itwire.com/ it-industry-news/market/51672-yellowfin-macquarie-uni-win-global-bi-award; Applications Executive Council, “Redefined Governance for Innovation and Improved Project Prioritization” Business 2.0, vol. 1, Corporate Executive Board, Washington DC, 2011, p. 83; Starner, Tom, “Profiting by Playing Games” Human Resource Executive Online, 27 October 2011, http://www.hreonline.com/HRE/story.jsp?storyId=533342586.
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
83
A knowledge worker’s employment history and educational background have less impact on their Insight IQ than providing access to training and analysts.
Overcoming the Insight Deficit
84
Shared Role: Analytical Skills Training and Coaching Maximum Impact on Insight IQ1 of Knowledge Worker Training, Support, and Background
24.7%
Factors that can be influenced in the near term have far greater impact than long-term factors such as educational background or employment history.
18.6%
2.4% 0.3% Training Available
Analyst Available to Provide Help
Has Held a Previous Role in Another Function
Science or Business Degree Program
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Effective information and analytics training is a strong driver of Insight IQ, but only 25% of knowledge workers experience effective training.
Only 25% Find Information and Analytics Training Effective Attendance and Effectiveness of Information and Analytics Training Percentage of Knowledge Workers
Total Knowledge Workers
100%
No Training Available
36%
Did Not Attend Training
15%
Attended Training but Didn’t Find It Effective
Attended Training and Found It Effective
Sixty-four percent of knowledge workers have access to training on information and analytics...
24%
25%
...but only 25% attend and find it effective.
n = 4,941 knowledge workers. Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
85
Shift IT’s focus from up-front training on new tools to ongoing support for conducting analysis.
Overcoming the Insight Deficit
86
TRAIN TO ANALYZE Maximum Impact on Insight IQ1 of Different Types of Information and Analytics Training
Training on how to conduct analysis has more than twice the impact on Insight IQ of any other training subject.
28.7%
13.2%
13.7%
Tool Functionality
Definitions and Terminology
Conducting Analysis
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Information Training Portfolio
Overview A training portfolio that provides ongoing support to knowledge workers for conducting analysis and using analytics tools, reports, and dashboards Executive Teaching Training on new tools, without ongoing support and reinforcement, is ineffective as knowledge workers struggle to apply the training to their jobs. Training that focuses on the functionality of the tools and does not cover information sources or analytical techniques is also ineffective. Adopt a multipronged approach to design training offerings that support knowledge workers across the tool lifecycle and focus on conducting analysis. Company Snapshot Tiffany & Co. Industry: Jewelry Stores 2010 Revenue: $3.1 B 2010 Employees: 9,200
Tiffany & Co., founded in 1837, engages in the design, manufacture, and retail of fine jewelry. The company also offers brand merchandise, including timepieces and clocks, sterling silver merchandise, crystal, custom engraved stationery, and fashion accessories. The company sells its goods exclusively through about 230 Tiffany & Co. stores and boutiques worldwide, its website, business-to-business accounts, and catalogs.
Source: http://finance.yahoo.com; http://www.hoovers.com. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
87
IT struggles to meet rapidly increasing demand from knowledge workers for deeper support on using information and conducting analysis.
■■
■■
Tools Training Falls Short
IT Challenges
Most existing training programs are triggered by the launch of a new tool. To meet this increased demand for analyzing information, IT ends up providing ad hoc training and support.
Overcoming the Insight Deficit
Cost of Failure
Up-front, one-size-fits-all training doesn’t recognize varied knowledge worker needs and learning styles.
■■
■■
Solution
Limited training penetration due to unmet varied needs Lower tool adoption
Knowledge workers fail to capture value because they don’t understand the data or know how to conduct analysis.
■■
■■
Under-utilized or misused analytics capabilities Unexpected support costs
1. Make training an ongoing process with a portfolio of training programs mapped across a tool’s lifecycle. 2. Deliver customized training programs using a variety of channels that map to diverse knowledge worker needs and preferences. 3. Provide training on understanding information and conducting analysis, not just using tools.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
88
Provide a portfolio of training offerings that supports knowledge workers across the tool lifecycle.
■■
■■
Effective analytics training is designed as an ongoing process, not just an event tied to the launch of a new tool. Offering a range of training options and channels ensures that training maps to diverse knowledge worker needs and preferences.
INFORMATION TRAINING PORTFOLIO Illustrative Training Focus Areas Build Awareness of New Tool
Type of Training BI Day
■■
On-site training days on BI/ analytics tools, new subject matter, reports, dashboards, query techniques
Video Training
■■
Online training modules for all staff globally Includes training on data subjects as well as a series of training on new data marts
■■
Classroom Training
■■
■■
Train on New Tool Functionality
Provide Support for Analysis and Information
Provide Support for Tools
Weekly program delivered to business unit data stewards, culminating in a project Primarily for new analysts or refreshers
Analytic Team Portal
■■
For communication to users and troubleshooting
Ongoing Analyst Support
■■
A BI specialist for two months after launch of new tool or information Data stewards for intradepartmental training and support
■■
Introduce New Analysis and Information
Ongoing training is provided across the tool lifecycle, with greater focus on supporting knowledge workers in using information and analytic techniques.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
89
Focus training on how to conduct analysis and use information, not just tool functionality.
■■
■■
■■
■■
Train on Analysis, Not Just Tools Business Intelligence Day Agenda
List of Online Video Training Topics
Illustrative
Illustrative
Tiffany & Co. uses BI Days to educate groups of knowledge workers about new information, reports, dashboards, tools, and query techniques available to them.
Agenda
Business subject matter experts and vendors are also invited to lead these sessions; BI Days are publicized internally by e-mail and office message boards.
11:00 a.m. How to Analyze Reports and Dashboards Standard offerings and ad hoc analysis
In addition, the information management group creates online videos to offer knowledge workers easy, on-demand access to training on tools and information. Training videos are typically 15 to 20 minutes long and range from general, beginner level to specific, advanced level.
Overcoming the Insight Deficit
Learning About Our Information—Six Video Modules
9:00 a.m. BI: The Journey to Here
■■
10:00 a.m. How to Analyze and Use the New Customer Information Hands-on training demos and tips
Enterprise data, data marts, general information
■■
Sales data
■■
General Ledger data
■■
Merchandising data
………
Noon Lunch Learning to Use Our Analytic Tools— Seven Video Modules
1:00 p.m. Overview of Enterprise BI and Data Warehouse Includes BI strategy
(Specific to Information Types)
2:00 p.m. Using Queries to Help You Make Decisions Tips and tricks 3:00 p.m. BI Technologies and New Tools
■■
Analytics tools
■■
Reporting tools
■■
Dashboard tools
…….
Facilitators: Information management group
Audience: 800 ad hoc knowledge workers, other viewers of analytics, reports and dashboards
Attendees: 100 business unit management staff Location: Large venue, agenda tailored to regional needs
Location: Online
Training Focused on Conducting Analysis or Using Information CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
90
Hire and incentivize analytics staff on their effectiveness at coaching, not just analyzing.
HIRE QUANTS WHO CAN COACH Maximum Impact on Insight IQ1 of Hiring Criteria for Analyst Skills
Skills That Organizations Rate as Most Important When Hiring Analysts Percentage of Organizations
■■
■■
Although organizations rate business knowledge skills as nine times more important when hiring analysts than coaching skills, business knowledge skills provide no opportunity for impact on Insight IQ.
Business Knowledge Skills
14.8%
6.8%
See page 98 for interview questions to use to test for coaching skills when recruiting new employees and page 99 for an example of a training program to develop coaching skills.
Quantitative Skills
0.0% Coaching Skills
Technical Skills
45%
36%
Quantitative Skills
Business Knowledge Skills
Technical Skills
Coaching Skills
(11.1%)
9%
5%
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
91
Develop specialized information analysis and insight skills to bolster IT’s ability to support knowledge workers.
Role Profile: Information Insight Enablers Profile of the Information Insight Enabler Role Key Responsibilities
■■
■■
A subset of the skills required in this role are the same as those often found in data analysts; consequently, data analyst is the most common background for the information insight enabler role. For full details of this role, see the information insight enabler job description at www.cio.executiveboard. com/Members/ ResearchAndTools/Abstract. aspx?cid=100236298.
Overcoming the Insight Deficit
■■
New-to-World Skills
Understands the decision-making process and the workflows of business unit heads and service managers
■■
Identifies knowledge workers’ information needs
■■
Represents information in a user-friendly manner
■■
■■
■■
■■
Identifies trends and patterns; generates insight for business units and senior leadership Develops framework and process to analyze unstructured information Performs market and customer research and analysis and creates dashboards and scorecards Teaches knowledge workers about their information and analytic tools
■■ ■■ ■■ ■■
Market and competitor analysis Information insight generation Information visualization Unstructured information analysis
Other Important Skills ■■
Business domain analysis
■■
Coaching and Training Skills
■■
Data warehousing
■■
Functional requirements analysis
■■
Global teaming and remote collaboration
■■
Information taxonomy creation
■■
Information modeling
■■
Usability design
Possible Backgrounds and Hiring Sources Likely to have a background in market or financial research or in analytics or statistics “Right now, we have lots of people who can build reports. In the future, I will need people who help business partners find the right insight.”
Sourcing Difficulty
Seniority
Individual Contributor
CIO US Construction Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
92
Informed skepticism can be fostered through the introduction of decision models and behaviors that encourage inference and judgment.
■■
■■
Business Leadership Role: Decision-Making Models and Behaviors Maximum Impact on Insight IQ1 of Decision Models and Performance Criteria
These are areas where IT has little influence and therefore requires business leadership.
24.6%
Performance metrics are transparent and in regular use and are supplemented by formal measures of analytic ability.
17.4%
16.4% 15.0%
Decision Processes Are Formalized
Functional Performance Metrics Are Regularly Reported
Performance Review Is Based on Measurable Goals
Performance Criteria Explicitly Include Ability to Conduct Analysis
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board IT Practice, Insight IQ Diagnostic 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
93
An insight-driven culture is best realized when business and decisionmaking processes are formalized.
■■
■■
Not all decision-making models will promote informed skeptics (e.g., models such as Six Sigma and lean that focus on reducing variation and ambiguity are less likely to lead to informed skepticism). See Appendix on page 97 for more details about applying critical thinking skills to decision making.
Overcoming the Insight Deficit
Sample DECISION-MAKING Models Essential Features of Decision-Making Models That Promote Informed Skepticism
Formalized Methodology—Mitigate biases and foster scalability by introducing a systematic decision-making process.
Nonlinear Methodology—Adapt to change by incrementally following decision steps that can be tested and revised at short intervals.
Critical Thinking—Improve decision making by combining judgment and common sense with normative decision-making processes.
Incorporate the Unknown—Actively engage with uncertainty by separating the known from the unknown in the decision-making process.
Explicit Biases and Assumptions—Surface and adjust for biases and document assumptions so their validity can be tested.
Sample Decision-Making Models Design Thinking Integrates intuition, reason, and imagination into strategic decision making
The Mintzberg Model Incorporates feedback loops to design an iterative and dynamic decision-making model
Collaborative Decision Making Connects disparate teams of experts across the organization to collectively analyze problems
The Cynefin Framework Reflects the complexity of strategic decisions by acknowledging ambiguity
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
94
Informed skepticism can be fostered by setting the right expectations through performance criteria.
■■
Competency models and performance criteria should also describe the analytical, skeptical, and collaborative behaviors needed for informed skepticism.
Sample Performance Criteria Leadership Criteria
Proficiency Criteria
■■
■■
■■
■■
Uses analytics to identify trends and works to understand the driving factors
■■
Uses analytics in developing plans to achieve target business results
■■
Shows good judgment in interpreting the results of analytics
■■
Creates an analytic culture and drives usage by building analytics into day-to-day work Holds team accountable for using analytic tools and encourages full but skeptical usage Encourages a collaborative culture and is open to dissent
Collaborates with others in analysis and decision making
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
95
Overcoming the Insight Deficit
96
Key Takeaways: Support Big Judgment, Not JUST Big Data
Foster Informed Skeptics: If knowledge workers cannot apply judgment to analysis, greater information access may do more harm than good. Therefore, IT and business leaders must both promote a skeptical attitude toward information and analysis through decision models, performance criteria, tools, and training. At Organizations with a High Insight IQ...
At Other Organizations...
1. Most knowledge workers are Informed Skeptics.
Most knowledge workers are Visceral Decision Makers or Unquestioning Empiricists.
2. Decision making is supported by collaborative tools.
There are no tools for collaborative decision making.
3. Analytical skills are developed.
Analytical skills are hired.
4. Training is an ongoing process, so a portfolio of training offerings is available at all times.
Training is an event, tied to the launch of a tool.
5. Training focuses on using information and conducting analysis.
Training focuses on using the tool.
6. Ability to coach is a key job requirement for analysts.
Analysts are hired and evaluated solely for their analytical skills and domain knowledge.
7. Decision models and performance criteria promote informed skepticism.
Decision models and performance criteria are lacking or promote unquestioning reliance on data.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Foster critical thinking skills to increase informed skepticism.
Appendix: Critical Thinking Skills
Nine Gates for Critical Thinking1 Prepared for First Data Corporation
1
1
Clarity Could you illustrate what you mean? Could you give examples?
2
Accuracy How could you find out if that is true?
3
Precision Could you be more specific? Could you give more details?
4
Relevance How does the data relate to the problem?
5
Depth What are some of the difficulties that may arise that we may need to deal with?
6
Breadth Do we need to look at this problem from another perspective?
7
Logical Does the data make sense together?
8
Significance What is the most important problem to consider?
9
Fairness Do you have biases that will affect the outcome?
Copyright Complete Intelligence LLC.
Source: Complete Intelligence LLC. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
97
Identify candidates with coaching skills by asking them to give examples of the following behaviors and competencies.
Overcoming the Insight Deficit
Appendix: Identifying Coaching Skills During Interviews Please describe a time when you... 1. Assessed Development Progress: Reviewed where an employee stood against their development goals
8. Gave Feedback on Personality Strengths: Gave an employee feedback during a performance review about their personality strengths
2. Helped to Draft an Individual Development Plan: Helped an employee create a plan to meet their development goals
9. Gave Advice from Your Own Experience: Gave an employee advice based on your experience
3. Ensured That an Employee Had the Necessary Skills/Knowledge: Ensured that an employee had the required skills and knowledge before they needed it 4. Ensured That a Project Provided Learning: Ensured that each project or assignment was a learning experience
10. Helped an Employee Apply New Skills/ Knowledge: Helped an employee apply new skills or knowledge in their work 11. Helped an Employee Find Training: Help an employee find the training they need 12. Passed Along a Development Opportunity: Passed along relevant development opportunities to an employee
5. Explained Performance Evaluation Standards: Helped an employee understand their performance review standards 6. Gave Feedback on Performance Strengths: Gave an employee feedback during a performance review about their performance strengths 7. Gave Feedback on Performance Weaknesses: Gave an employee feedback during a performance review about their performance weaknesses
13. Passed Along Job Openings: Passed along information about internal job openings to an employee 14. Provided an Employee with Experiences That Develop: Helped an employee obtain the experiences at work that helped them develop over time 15. Taught a New Skill or Procedure: Taught an employee a new skill, concept, or process
Source: CLC Learning and Development research; CIO Executive Board research.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
98
Introduction
Informed Skepticism
Who Does What
Appendix
Use this process to help employees learn how to be effective coaches.
Appendix: A Formula for Coaching Success BCBSMA’s Learning Process for Coaching Align Coach Profile with Leadership Style
Reflect on Ideal Coach Attributes
■■
■■
■■
Which coach from my own experiences do I admire most? What did this coach do and say that was effective, and how did he or she interact with me? What kind of goals did this coach help me to achieve?
■■
■■
■■
What are my natural strengths as a leader based on my DISC1 Leadership Assessment? How does my idea of an ideal coach align with my leadership style? How can I reevaluate my definition of an ideal coach to better align with my leadership strengths?
Factor Team Dynamics and “Coachability” ■■
■■
■■
■■
How is my coaching and behavioral style similar and different from my direct reports? How can I adjust or tailor my style according to the behavioral styles and needs of my direct reports? What are potential areas of tension of my coaching style and the differing behavioral styles and needs of my direct reports? What might my direct reports need more or less from me?
1
CIO0361911SYN
■■
■■
■■
+
Focus on the “Right” Coaching Style CIO Executive Board IT Practice www.cio.executiveboard.com
Learn Coaching Fundamentals
How should my coaching activities differ based on performance-, development-, or career-based objectives? How can I better delegate to make projects a coaching opportunity? What role does my coaching style have in teambased problem solving?
Address Common Coaching Pitfalls
■■
■■
■■
What are the common challenges of coaching? How can I manage the emotions involved in delivering difficult feedback? What are strategies to address and resolve conflict within my team?
Focus on the Most Impactful Coaching Activities
=
Increased Coaching Effectiveness
DISC refers to dominance, influence, steadiness, and conscientiousness.
Learning and Development Roundtable, “Awareness-to-Action Coaching Toolkit” Profiles of Coaching Programs and Tactics, Corporate Executive Board, Washington DC, 2007, p.29
Introduction
Informed Skepticism
Who Does What
Appendix
Support Big Judgment, Not Just Big Data
99
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
100
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
101
Collecting knowledge worker feedback on information sources and analytic tools has a 24.8% maximum impact on Insight IQ.
Overcoming the Insight Deficit
102
IT PAYS TO LISTEN Maximum Impact on Insight IQ1 of Knowledge Worker Feedback
24.8% 24.8%
16.5% 16.5%
Average for All Top 10% Drivers
Knowledge Worker Feedback on Information Sources and Analytics Tool Is Collected
n = 4,941 knowledge workers.
1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
IT needs to adopt a proactive collaborative and anthropological approach to identify opportunities to provide information and analytic tools.
Approaches to Identifying Opportunities to Provide Information and Analytic Tools
High
Progressive organizations engage knowledge workers earlier and more frequently throughout the project lifecycle using observation and experimentation.
1
Anthropologically Driven Need Identification
Value Associated with the Approach
■■
From LISTENING to OBSERVING
“Prosumer” Sounding Board
Use anthropological observation and interviewing techniques to discover information needs.
Observe progressive users to identify pain points and emerging solutions.
Agile Development Information Usability Surveys Requirements Gathering Low Feedback Collection
Proactive Observation Approach to Observation
1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
Observe and Question to Identify Opportunities
103
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
104
“Prosumer” Sounding Board
Overview A methodology to observe knowledge workers (both early adopters and mainstream users) to surface unarticulated and emerging information needs EXECUTIVE TEACHING Segment knowledge workers based on their information usage patterns and their stance toward technology adoption, then customize engagements to learn from each segment. Observe early adopters as an early-warning mechanism for emerging needs but balance this with input from the mainstream. Company Snapshot Ford Motor Company Industry: Auto Manufacturing 2010 Sales: 2010 Employees:
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
US$129.0 B 164,000
Ford Motor Company, a global automotive industry leader based in Dearborn, MI, manufactures and distributes automobiles across six continents. The company provides financial services throughout Ford Motor Credit Company.
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
Observe and Question to Identify Opportunities
105
Ford’s Digital Worker Office gathers user feedback on a regular basis to ensure continued relevance of collaboration services and communications.
■■
For additional background on the Digital Worker Office see the profile of Ford’s Digital Productivity Package in the IT Leader’s Guide to Collaboration at www.cio.executiveboard. com/Members/ ResearchAndTools/Abstract. aspx?cid=100159974.
Overcoming the Insight Deficit
DEVELOP User LISTENING POSTS to track evolving needs
Target Services Across Users
Key Challenges
■■
Solutions
■■
Engage Progressive Users
User needs vary within roles, making it difficult to scalably engage with the user base while ensuring services match with needs.
Allow users to self-segment based on preferred engagement and tech savviness to understand preferences for collaboration services, communications, and training.
■■
■■
IT misses the perspectives of the most tech-savvy users as they often bypass IT to deploy point solutions.
Ford creates a sounding board of progressive users to understand pain points and pressure-test IT plans.
“We want to make IT an enabler of user productivity, rather than play a ‘control and command’ role.” Scott Blow Manager, Digital Worker Office Ford Motor Company CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
106
IT allows users to select personas based on their needs and customizes engagement accordingly.
CREATE PERSONAS TO CUSTOMIZE ENGAGEMENT Personas
■■
IT creates personas based on user attitude toward technology and workflow needs to learn about challenges and target services and communications.
Engagement Strategy
1 Prosumers Willing to assume the risk of using new, uncertified, cutting-edge technologies with a light-touch support model
IT calls on prosumers to pilot new tools and suggest collaboration improvements.
Typical Demand Pattern—Employee-owned IT browsers
Early warning of emerging needs...
2 Senior Leaders ■■
■■
Users can select more than one of the five personas on the Digital Worker Office site. IT directs users to its site in training sessions and on the corporate intranet.
Company officers and others who require highend, auditable, discoverable collaboration services
IT dedicates a communications liaison and gathers recommendations on required collaboration tools and services.
Typical Demand Pattern—Support for off-site access to all systems
3 Information Creators Contribute information for decision making and need sophisticated knowledge management tools
IT knowledge management experts provide assistance with specialized needs such as domain search.
Typical Demand Pattern—Administrative responsibilities for team websites
DO provide users with incentives—such as early access to new tools—for providing feedback to IT.
DON’T invest equally in gathering input across all user segments; instead, favor scalable methods such as community sites over focus groups for users whose needs are easier to diagnose. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
4 Mobile Workers Knowledge workers who work primarily from outside the office
...is balanced with insight into mainstream demand.
IT connects users with each other to share problem-solving tips and device preferences.
Typical Demand Pattern—High-end access to information from outside the office
5 Standard Collaborators Consumers, reviewers, and processors of information
IT provides information about collaboration capability improvements.
Typical Demand Pattern—Occasional participation in meetings
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
The IT Skills Implications
Observe and Question to Identify Opportunities
107
Create a sounding board of progressive users to understand barriers to end-user productivity and identify innovative solutions.
Overcoming the Insight Deficit
108
CO-Opt CRITICS TO Identify Gaps Progressive User Sounding Board
Selection Criteria ■■
■■
The Digital Worker Office at Ford engages a subset of the prosumer segment to surface productivity pain points and drive innovation. To prevent technology evangelism, the facilitator guides users to diagnose the root cause of the underlying problem they are trying to solve with technology.
DO engage tech-savvy IT critics to identify capability gaps and push innovation.
DON’T overengineer conversations with users. Establish an informal dialogue to surface concerns about IT and understand new technologies users are experimenting with. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
■■
Independently solved technology constraints (e.g., obtaining access to enterprise data or e-mail on personal mobile device)
■■
Comfortable with voicing concerns or dissatisfaction with IT
■■
Understands the economics of niche versus enterprise solutions
■■
Demonstrates the drive to use new tools IT implemented
■■
Has a track record of recommending new technologies to IT
Progressive User Sounding Board Description The sounding board is a global, cross-functional group of 12 users that meets monthly. Role ■■ Articulate pain points. ■■ Provide early feedback on IT deployment plans. ■■ Participate in collaboration pilots.
Sample Discussion IT Facilitator
Sounding Board Participant
“Where do you see opportunities for improving productivity?”
“We need Device B to access enterprise e-mail while traveling.”
“Will Device A work just as well?”
“Device B has better functionality and interface.”
“What specific aspects are better?”
“Device B allows me to carry one device for corporate and personal use of e-mail and web.”
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
ANTHROPOLOGICALLY DRIVEN NEEDs IDENTIFICATION
1
OVERView The use of anthropological techniques to discover hidden information needs and opportunities for productivity improvement, by observing knowledge workers in their own environment and conducting open-ended interviews Executive Teaching Traditional approaches to requirements gathering fail when assessing knowledge workers’ information needs because the needs are fast-changing, diverse, and difficult to articulate. Alpha Company adopts an anthropological approach to opportunity identification. By observing knowledge workers in diverse settings and conducting open-ended interviews, Alpha Company gains a deeper understanding of knowledge worker needs and proactively identifies pain points in end-user workflows. Component Teachings ■■
■■
■■
Component 1: Direct Observation—Use anthropological observation techniques to surface unarticulated and context-dependent knowledge worker needs that are missed by more focused requirements-gathering tactics. Component 2: Interviews and Discourse Analysis—Conduct open-ended interviews with various stakeholders and analyze end-user stories beyond the immediate language to discover patterns of user experience. Component 3: Skills Implications for IT—Collectively, these observational and anthropological approaches require IT to develop a new set of anthropological skills.
Company Snapshot Alpha Company 2010 Sales: > US$10 B
1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Alpha Company is a leading technology provider.
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Observe and Question to Identify Opportunities
109
While knowledge workers often lack the ability to articulate their information needs, these needs also differ across individuals, organizational cultures, and collaborative practices.
Overcoming the Insight Deficit
CAN THE KNOWLEDGE WORKER SPEAK?
Needs Identification Scenarios
1
Identify Opportunities for Productivity Improvement
Surface Unarticulated Knowledge Workers’ Information Needs
Challenges
Information needs are context dependent and vary across individuals, organizational cultures, and collaborative practices.
Knowledge workers lack the ability or inclination to communicate their workflow or information needs.
Illustrative Examples
Sales Team Collaboration—Alpha Company found that the value of a sales team collaboration tool varied significantly across the year as collaboration declined in Q4.
Customer Relationship History—By observing sales teams in two regions, Alpha Company found that information on relationship history was far more valuable in one region than the other, due to cultural factors.
Mobile Sales Report Writing—Alpha Company found that trip reports written on the road were less detailed, took more time, and had more errors than those written in the office.
Information Source Diversity—Alpha Company discovered that sales teams with a certain profile of manager needed access to a broader variety of information sources when travelling.
“Anthropological research does not start from an ideal state to be achieved. Instead, it provides input to process redesign based on actual conditions on the ground.” Corporate Anthropologist Alpha Company 1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
110
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Traditional processcentric workflow mapping techniques fail to surface knowledge workers’ actual needs and behaviors.
■■
FROM WORKFLOW MAPPING TO ANTHROPOLOGY
Traditional Workflow Mapping Approach ■■
Most requirements-gathering techniques assume that there is only one right way to do something, and that this is the best solution for all knowledge workers in all contexts.
■■
■■
1
Alpha Company’s Anthropological Approach
Assumes there is only one best known method of working
■■
Uses standardized process discipline with pre-set rules for mapping workflows
■■
Solutions for knowledge worker needs aligned to stated needs and a predefined business case
■■
Assumes there may be several contextspecific and equally valid ways of working Uses observational techniques to identify roadblocks in day-to-day workflows and de-familiarization to mitigate biases Solutions based on unarticulated or unstated knowledge worker needs
When to Use an Anthropological Approach Workflows are intuitively executed and the result of many ad hoc workarounds.
Knowledge worker needs and workflows are context specific and difficult to articulate. “While end-user surveys show you what knowledge workers say they do, anthropological research tells you what knowledge workers actually do.”
The goal is to enable greater productivity and insight by understanding how knowledge workers work.
Corporate Anthropologist Alpha Company 1
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Observe and Question to Identify Opportunities
111
Use anthropological observation and interviewing techniques to surface unarticulated and context-dependent knowledge worker needs.
■■
■■
1
Overview of Common Anthropological Techniques
Scenario: Help sales reps frame needs in terms of their underlying challenges in accessing information, rather than the technologies they need.
While end-user surveys follow a predetermined trajectory with little room for maneuvering, openended interviews follow a conversational model that allows for immediate feedback.
DO engage knowledge workers with an open mind, without a desired business goal, process, or workflow in mind.
DON’T assume that one process is superior or that knowledge workers use the same workflow in all situations. 1
CIO0361911SYN
112
OBSERVE AND ASK ASSUMPTION-FREE QUESTIONS
Gain insights into knowledge worker requirements through direct observation in different user settings, organizational contexts, and across points of time.
CIO Executive Board IT Practice www.cio.executiveboard.com
Overcoming the Insight Deficit
Anthropological Observation Techniques
Open-Ended Interviewing Techniques
1. Contextualized Observation Observe sales teams in their natural environments, such as customer visits or in sales meetings. Look across internal divisions and at organizational and cultural contexts.
1. A cknowledge Expertise Neither pretend to know more about the subject matter than the sales team does, nor buy in to easy answers, but always ask follow-up questions.
2. Observation Across Space and Time Observe sales teams across multiple spaces and points of time, looking for differences that may relate to location or time of day.
2. Reiterate and Qualify Repeat what is heard to clarify meaning, but also ask whether these are the right questions to ask.
3. S elf-Reporting Ask sales reps to use written or audio-visual diaries to record their workflow and log stage gates that cause bottlenecks or stalls.
3. Read the Silence Look for ideas and actions that are missing from the sales reps’ narrative, and identify variability and consistency across knowledge worker experiences.
4. Auto-Reporting Record the use of collaboration and analytic tools, using real-time monitoring to measure response times and gain deeper insights into sales reps workflows.
4. Ignore Nothing In the context of unarticulated needs, anything may turn out to be significant.
Pseudonym.
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Screen for attributes such as intellectual curiosity and critical thinking to develop a set of core anthropological skills within IT.
■■
■■
While trained anthropologists are best qualified to conduct anthropological research, IT organizations should first identify employees with intellectual curiosity and critical thinking skills. Use behavioral interviewing techniques to locate and identify intellectual curiosity and observational and problem solving skills during the interview process.
CAPTURE THE CURIOUS Overview of Anthropological Skills
Technical Anthropological Skills
Skills Implications of Anthropological Observation General Skills
Specific Skills
■■
Critical thinking and intellectual curiosity
■■
Critical thinking and intellectual curiosity
■■
Social agility and sensitivity
■■
Empathy and open-mindedness
■■
Contextual understanding
■■
Contextual understanding
■■
Primary research and fieldwork methodologies
■■
Discourse analysis and intertextuality
■■
Identifying patterns of behavior
■■
■■
■■
Understanding the evolutionary and material context of end-user needs
CIO0361911SYN
■■
Storyboarding ■■
Where to Find These Skills
CIO Executive Board IT Practice www.cio.executiveboard.com
Skill Implications of Open-Ended Interviews
Interviewing skills with high levels of reflexivity Gathering, integrating, synthesizing, and analyzing data Storyboarding
Where to Find These Skills
■■
Anthropology Graduates
■■
Anthropology Graduates
■■
Human Factors Engineers
■■
Market Researchers
■■
User Interface Designers
■■
Investigative Journalists
■■
Interaction Designers
■■
Information Scientists
Introduction
Approaches to Opportunity Identification
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
Observe and Question to Identify Opportunities
113
IT organizations can embed anthropological skills in several existing or emerging roles.
■■
THE ANTHROPOLOGIST INSIDE US
Criteria for Locating Anthropological Skills
The ongoing focus on information management and user-behavior analysis will see the emergence of IT roles that require capturing knowledge worker requirements for effective decision making.
■■
Has domain knowledge from alignment with business units
■■
Has time outside specific projects
■■
Has seniority, credibility, and business engagement skills
■■
Links back to project, service management, or architecture teams
Alpha Company’s Approach Dedicated Corporate Anthropologist Role ■■
■■
■■
For full descriptions of the responsibilities and skill requirements for these roles, please see the CIO Executive Board’s IT Job Descriptions Database. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
Alternative Approaches Embed Anthropological Skills in Existing Roles
For large organizations For organizations with people-intensive processes (e.g., Sales, Prof. Services, R&D) When project warrants dedicated resources
Introduction
Approaches to Opportunity Identification
Embed Anthropological Skills in New-to-World Roles
Business Analyst
Collaboration and Social Media Evangelist
Business Liaison
Information Insight Enabler
Service Manager
User-Experience Designer
Solution Architect
Observe Progressive Users
Anthropologically Driven Need Identification
the IT Skills Implications
114
KEY TAKEAWAYS: Observe and Question to Identify Opportunities
Use Observations of Knowledge Workers in Addition to Observations from Knowledge Workers: Adopt an anthropological approach to identify opportunities and deliver analytic capabilities successfully. Identify and learn from early adopters, observe segments of knowledge workers across the organization and in diverse user settings, and conduct open-ended interviews to identify bottlenecks and pain points.
At Organizations with a High Insight IQ...
At Other Organizations...
1. IT is proactive in observing knowledge worker information needs.
IT listens to knowledge workers only when gathering requirements.
2. IT segments knowledge workers by their attitude toward technology and information usage patterns.
IT segments knowledge workers solely by role or seniority.
3. IT learns from early adopters about new analytic functionality.
IT restricts early adoption of new functionality.
4. IT observes knowledge workers in diverse user settings, locations, and points in time.
IT gathers requirements for information and analytic tools primarily through business process mapping.
5. IT conducts open-ended interviews to capture the voice of the knowledge worker.
IT communicates with knowledge workers through end-user surveys.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Observe and Question to Identify Opportunities
115
Overcoming the Insight Deficit
Overcoming the Insight Deficit 1. E nable Information Usability Key Insights from the Diagnostic
■■
■■
■■
2. S upport Big Judgment, Not Just Big Data
Provide a portfolio of analytic tools, as a single tool cannot meet the needs of all knowledge worker segments. Visualize how information from multiple sources can be combined to answer key business performance questions. Rather than seek “perfect information,” in all cases, make information quality transparent and targeted.
■■
■■
If knowledge workers cannot apply judgment to analysis, greater access to information may do more harm than good. Foster informed skepticism through tools for collaborative decision making, training, and new decision models.
3. O bserve and Question to Identify Opportunities ■■
■■
Create opportunities to improve knowledge worker productivity via observation, not by merely asking for requirements. Identify and learn from early adopters.
Case Examples Capability-Driven Tool Portfolio
Information Training Portfolio
“Prosumer” Sounding Board 1
Flexible and Transparent Analytics
Anthropologically Driven Needs Identification
CXO KPI Identification
Emerging IT Roles
Information Architects
User Experience Specialists
Information Insight Enablers
Corporate Anthropologists
Appendix: Ownership Models for Analytics Centers of Excellence 1
Pseudonym.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
116
CIOs face a range of options for central or local management of information and for management within IT or outside.
Information Management—How to Organize, Where to Locate?
Centralized (Dedicated) Managing information centrally avoids duplication, promotes integration, and provides a single view of the enterprise.
Diffused (Integrated with Other Activities) Information should be managed close to the people who use it to allow for speed and flexibility. Key Questions
Business Ownership Only information producers and users understand its meaning and value and only they can improve quality.
Business/ Centralized
IT Ownership IT manages the information sources and can offer a neutral perspective.
IT/ Centralized
Business/ Diffused
1. What is the right degree of centralization for information management and analytics? 2. How should responsibility be divided between business areas and IT?
IT/ Diffused
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
Appendix
117
Foundational analytic and information management activities benefit from centralization and create sufficiently strong oversight to sustain decentralized information sources.
Overcoming the Insight Deficit
CENTRALIZE Management, NOT INFORMATION Maximum Impact on Insight IQ1 of Centralized Models for Information Management 13.0%
10.5%
10.4%
6.9%
6.7%
0.0% Centralized Analytics Team
Centralized Knowledge Worker Training
Centralized Information Quality
Centralized User Support for Analytic Tools
Centralized Information Architecture
Organizations with a high insight IQ centralize information management and support activities…
Single Unstructured Information Repository
0.0% Single Data Warehouse
…which enables them to keep the information itself decentralized.
n = 4,941 knowledge workers. 1
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board IT Practice, Insight IQ Diagnostic, 2011. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
118
There is no one ideal model for an analytics center of excellence as the optimal location depends on a number of organizational factors.
Where Should An Analytics COE Reside? Factors Determining Organizational Location of Analytics Center of Excellence (COE)
1. Location of Analytics Expertise
IT is the main source of analytics expertise.
Other functions/business units have analytics expertise.
2. Organizational Diversity
IT is the largest central function.
Other central functions are well resourced or influential.
3. Nature of Analytics Use
Information is used to produce the product.
Information is used to price or sell the product.
4. Information Management Focus
Information management is focused on integration and quality.
Information management is focused on knowledge worker capability.
I. Analytics Coe Within It
II. Paired IT and Business COEs
III. Analytics COE Outside IT
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
Appendix
119
While many activities migrate between IT and a COE outside IT, information stewardship remains primarily a business responsibility in all models while source system management always remains in IT.
Overcoming the Insight Deficit
DIVISION OF LABOR BETWEEN IT AND THE REST OF THE BUSINESS
I. A nalytics COE Within IT
Activity
II. P aired IT and Business COEs
III. Analytics COE Outside IT
Source Systems Management Information Architecture Data Warehousing Strategy and Prioritization External Information Acquisition Reporting Knowledge Worker Training Analytics Information Stewardship
Examples
IT–Owned Activities Owned Elsewhere in the Corporate Center
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
120
Introduction
Information Management COEs
Skills Implications
Information management and analytics activities are centralized within IT.
I. Analytics COE WITHIN IT Information Management and Analytics Organization Structure
A. E ach major business unit is represented on the Data Governance Council; the council was set up as a business council, and IT’s participation on the council is purely as one of the represented business units.
CIO
A Data Governance Council
B. EDW and BI user support team addresses questions related to both business intelligence tools and content.
Enterprise Business Intelligence (EBI)1
EDW Content and Data Governance
C. B I solutions team develops reports and tools for users; as there is a strong focus on user self-service, the team typically builds templates for users to adapt rather than providing ongoing maintenance and support.
EDW Content and Architecture
Business Applications
EBI Asia Support and Development
Business Intelligence C
Asia Business Intelligence
BI Solutions
D
D. A pplication-specific and factory-related reporting is run outside the EBI group to cater to special requirements of these groups.
Factory Systems
PMO and Data Modeling
BI Strategy and Architecture
EDW and BI User Support
Advanced Analytics
Asia EDW Content
B
Real-Time and ApplicationSpecific Reporting
D
Real-Time and ApplicationSpecific Reporting
Reports to IT 1
Quality and engineering applications support teams are not shown on this chart but they also report into the executive director, Enterprise Business Intelligence.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
Appendix
121
Information management and analytics responsibilities are divided with a business COE acting as an interface between IT and the rest of business.
Information Management and Analytics Organization Structure
Analytics Resource Center (ARC)
B. ARC is supported by business subject matter experts and change agents from the lines of business. C. D ata Analysis COE reports to IT and derives data requirements and data delivery priorities from ARC’s prioritization and governance processes.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Data Analysis COE
BI Delivery and Adoption Team
Program Executive/ Group Analytics Lead
Data/Information Analysis Lead ARC Prioritization and Data Governance Processes
Analytics Advisory Board Program Management Office
B
B
Change Agents
Business SMEs
–– Support analysts across the organization in nonactuarial analytics, and –– Lead data governance efforts across the organization.
122
II. PAIRED IT AND BUSINESS COEs
A. K ey mandates for the Analytics Resource Center (ARC): –– Represent business partners in the development of BI and provide testing and training,
Overcoming the Insight Deficit
A
Analytics
Data Analysis
Business Intelligence
Data Governance Strategy
Reporting
Policy Committee
Data Delivery
Data Integration
C
Data Analytics Data Analysis
Process Team
Support
Aligned by BUs
Technical Lead
Metadata/ Governance
Aligned by Subject Areas
Business-Led Function That Reports to the Actuarial Group Introduction
Information Management COEs
Reports to IT
Skills Implications
Information management and analytics activities are centralized under a corporate function, shifting most of traditional information management activities out of IT.
■■
■■
■■
III. Analytics COE Outside IT Information Management and Analytics Organization Structure
Executive VP and Strategy Officer Enterprise Information Committee
At point A, touchpoints with IT are maintained through representation in committees and through regular meetings with the IT staff and CIO. At point B, a few technical information management– related roles such as data integration remain outside of the COE; information architecture responsibilities are represented on both sides. BCBSNC’s COE is being built iteratively, first by including the transactional activities (e.g., data sourcing, reporting, training) and some critical pure analytics functions (e.g., group analytics, care management).
Information Management and Analytic Services Analytics Center of Excellence
Analytic Services and Solutions
Analytic Delivery
Analytic and Information Management Advisory Committee
Continuous Improvement
A B
Informatics Solutions
Analytic Services
Solutions Portfolio Analyst Support Technology Architect
Informatics Solutions Analysts
Informatics Analysts
Portfolio and Process Analyst(s)
B
Information Strategy and Architecture Data Integration and Production Support
Informatics Solutions Developers
Corporate Function That Reports to the Strategy Group CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction
Information Management COEs
Skills Implications
CIO0361911SYN
Appendix
123
An analytics COE within IT requires new skills such as information visualization, analytics service management, and knowledge worker training and coaching.
Overcoming the Insight Deficit
Skill IMPLICATIONS OF Each COE MODEL I. A nalytics COE Within IT
Skills
II. P aired IT and Business COEs
III. Analytics COE Outside IT
Database Design and Mgmt. Info. Systems Design Info. Architecture Info. Integration Business Analysis Info. Visualization Analytic Service Mgmt. Strategy and Portfolio Mgmt. Info. Policy Formulation Analysis/Insight Generation Training and Coaching
IT Skills That Rise in Importance Business Skills That Rise in Importance
For a description of the key information management and analytics skills, please see page 125. CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Introduction
Information Management COEs
Skills Implications
124
Implementation Guide: SKILLS Descriptions
Analytics Service Management
The skills required manage analytics as an end-to-end service from service strategy to service delivery and improvement
Database Design
The skills required to produce a detailed data model for a database
Database Management
The skills required to create, access, and maintain a database
Information System Design
The skills required to analyze and design information systems, concentrating on entities and their attributes and interrelationships; also includes data modeling for individual databases and the corporate data model, as well as coordinating the definition of data across multiple distinct databases
Information Architecture and Development
The skills required to define how information and information systems support the objectives of the business
Information Integration
The ability to integrate information and content from multiple sources and channels
Business Analysis
The skills required to identify business needs and determine solutions to business problems
Information Visualization
The ability to depict information in intuitive and insightful ways
Information Policy Formulation
The skills required to establish guidelines for information dissemination and usage
Information Analysis and Insight Generation
The ability to understand the meaning of structured and unstructured business information and generate insight from it
Note: For detailed analysis on the emerging skill gaps and role changes for IT, please ask the CIO Executive Board staff or your account manager for a copy of our recent study on The IT Talent Implications of the Future of Corporate IT. You can also download this study from www.cio.executiveboard.com. CIO Executive Board IT Practice www.cio.executiveboard.com
Introduction
Information Management COEs
Skills Implications
CIO0361911SYN
Appendix
125
List of Questions Asked in the Insight IQ Diagnostic
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
126
I. Information Management Structures and Processes 1. Has your organization introduced new or enhanced information management or analytics capabilities in the past 18 months? 2. How are individuals/teams dedicated to analytics structured within the organization? –– Centralized companywide –– Partially centralized by business unit –– Partially centralized by function –– Decentralized—most teams/subfunctions have their own analytic experts 3. If your organization has a central analytics group or Center of Excellence, where does it reside? –– IT –– Business—Shared Services –– Business—office of CEO or President –– Marketing –– Finance –– Other 4. Which of the following capabilities are in place to improve decision making within your organization? –– Automated decisions –– Collective intelligence tools –– Decision making team structure based around diversity and independence –– Formal decision frameworks or guidelines –– Other 5. Is information primarily pushed to you or pulled by you? 6. Do rules exist around which information can be shared across the organization? 7. Has your organization undertaken the following in the past 12 months. –– Adding new information sources to the data warehouse –– Enhancing existing data source feeds –– Updating or modifying business rules used in standard reports –– None 8. Which best describes your organization’s approach to managing information and analytics? –– Information and analytics are managed completely
separately. –– Information and analytics are managed separately, but the owners of each are aware of the other’s processes and accommodate them as appropriate. –– Processes for managing information and analytics are partially integrated. –– Processes for managing information and analytics are fully integrated throughout the lifecycle. 9. Which best describes the structure of your organization’s information management group? –– Formal—within IT (with dedicated staff) –– Informal—within IT (i.e., a couple of individuals are responsible but is not their full-time job) –– Formal—within Shared Services –– Informal—within Shared Services –– Formal—part of broader Analytics Group –– Informal—part of broader Analytics Group –– Formal—within individual BUs/functions –– Informal—within individual BUs/functions –– Exists, but is integrated into day-to-day tasks of broader IT –– Doesn’t exist 10. Who owns each of the following information management responsibilities? –– Information architecture –– Information lifecycle management –– Information quality and standards –– Knowledge and content management –– Analytics tools –– Enterprise data warehouse 11. If you have an Information Management group, what is the perceived clarity of its roles and responsibilities? 12. Indicate the degree to which the following are centralized. –– Data warehouse –– Analytical tools –– Unstructured information repositories –– Knowledge worker training in information use and analytics –– User support for analytical tools
–– Information ownership –– Information architecture –– Information quality 13. How are individuals/teams dedicated to analytics structured within the organization? 14. If your organization has a central analytics group or Center of Excellence, what are its responsibilities? –– Coordinate analytics initiatives. –– Share analytics best practices and provide training. –– Provide analytical expertise/capabilities. –– Deploy analysts where needed. –– Develop analytics strategic plan for the organization/ functions. –– Provide guidance/rollout of analytical tools/technologies. –– N/A 15. If your organization has a central analytics group or Center of Excellence, where does it reside? –– IT—Reports directly to CIO –– IT—Applications –– IT—Infrastructure/Operations –– IT—Information Management –– IT—Other, please indicate –– Business—Shared Services –– Business—Office of CEO or President –– Marketing –– Finance –– Other, please indicate –– N/A 16. Does your organization use an external service provider for analytics (excluding cloud providers)? 17. If you do use an external service provider for analytics, which of the following is the external service provider responsible for? –– Information architecture –– Information lifecycle management –– Information quality and standards –– Knowledge and content management –– Analytic tools and capabilities –– Enterprise data warehouse
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
127
I. Information Management Structures and Processes (Continued)
––
20. Do you use cloud applications for analyzing information? 21. What capabilities are in place at your organization to ease the capture of unstructured information in shared systems? . –– Automated processes (e.g., automated tagging, document uploads) –– Rules and guidelines on what information to capture and tag –– Rules and guidelines on how to capture information and tag –– Dedicated team or individual responsible for capturing, tagging and transferring information into shared systems –– Other –– None 22. For which of the following does your organization have defined processes in place? –– Create/Acquire –– Store/Share –– Find/Use –– Archive/Retain –– Dispose –– Manage/Control –– None 23. How do you define metadata in your data warehouse? –– For all information types –– For information that is shared between functions –– For information that is shared between business units –– For information that is deemed critical to the business –– For information that is used most frequently –– For information that is stored in multiple systems or repositories –– For information where there is frequent confusion regarding the definition –– For unstructured information –– For no types of information 24. Which best summarizes your current level of metadata management maturity? –– Incomplete business and technical metadata are
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
128
II. Training and Support
18. Do you use cloud applications for generating information? 19. Do you use cloud applications for storing information?
Overcoming the Insight Deficit
–– –– ––
maintained in pockets of solutions or documents, typically at a project level. Consolidation of some business definitions or technical information occurs, but governance processes are immature. Business and IT accountabilities and processes are starting to be formed to manage repositories of metadata. Business and IT are utilizing common metadata across departments and projects to improve understanding. Formal processes and accountabilities are in place for managing official repositories of metadata utilized for guiding information use.
25. Do rules exist around types of information that could be shared between levels and/or different parts of the organization? 26. How would you assess your organization’s current level of proficiency in the following Enterprise Architecture disciplines? –– Business Architecture –– Information Architecture –– Applications Architecture –– Technical Architecture
1. Is there an analyst available to you to help you find information or conduct analysis? –– There is an analyst embedded directly in my team. –– There is a team of analysts in my business unit or function. –– There is a companywide central team of analysts. –– There are informal analytical experts on my team. –– There is no one available to me for help in finding information or conducting analysis. 2. Does your organization provide training on finding information or conducting analysis? 3. If you attended training provided by your organization on finding information or conducting analysis, how effective or ineffective was that training in the following areas? –– Finding information I need –– Information definitions and terminology –– Conducting analysis –– Communicating and displaying information –– Using the functionality of the tools provided –– Where/how to get help 4. How formalized is the process for making decisions in your organization? –– Non-existent: No decision making processes exist. –– Ad hoc: Individuals and/or teams have processes that are followed irregularly. –– Undocumented: Individuals and/or teams follow processes that are not standardized nor documented. –– Defined: Processes used across business units and/or functions are documented but not enforced. –– Managed: Processes are documented and enforced. –– Optimized: Processes are documented, enforced, and measured to drive improvement. 5. Which of the following are in place to improve decision making within your organization? –– Automated decisions –– Collective intelligence tools –– Decision making team structure based around diversity and independence –– Formal decision frameworks or guidelines –– Other –– None
III. Knowledge Worker Attitudes and Incentives 1. Are you evaluated in your performance review on your ability to conduct analysis effectively? 2. To what extent do you agree with the following: I enjoy conducting analysis? 3. To what extent do you agree with the following: If I conduct analysis and disagree with the findings I will rely on my own judgment instead? 4. To what extent do you agree with the following: I have enough time to conduct analysis that is valuable to me in making decisions in my daily work? 5. To what extent do you agree with the following: Expertise in analytics increases my ability to advance in my desired career path? 6. To what extent do you agree with the following: I feel very comfortable conducting analysis and am able to meet all my information and analytical needs without external help? 7. When making important decisions in your organization, how critical are each of the following considerations to you, your direct manager, and your organization’s leadership? –– Information and Analytics –– Intuition –– Precedent –– External Advice 8. How effective is your direct manager in using analytics in carrying out his/her job responsibilities? 9. To what extent do you agree with the following: My performance review is based largely on metrics and achievement of measurable goals and objectives? 10. To what extent do you agree with the following: Performance metrics and progress against my business unit/function’s goals and objectives are reported to me on a regular basis?
11. How significant a contributor to your personal analysis and decision making are each of the following types of information? –– Centrally provided information available to and used across the organization –– Centrally provided information available to and used across my business unit/function –– Information generated by and managed by my subfunction or team –– Information I personally obtain through external sources 12. Which characteristics do you value most when using information and analytics in your daily work? –– Information Accuracy –– Information Timeliness –– Information Consistency –– Information Relevance 13. How important are the following capabilities when you make decisions? –– Visualization—The graphical or visual presentation of information (e.g., through static or interactive graphs, charts, maps, images) to make it easier for the users to understand the information contents and attributes –– Analysis—The process of examining information with the purpose of drawing conclusions about that information –– Filtering—The process by which redundant or unwanted information from an information stream is removed and only information that meets specific criteria (e.g., time, value, user groups) is displayed or presented –– Aggregation—The process by which information is gathered and expressed in a summary form, typically combining two or more information attributes or occurrences
15. How important is information and/or analytics in enabling you to do the following? –– Make Better Decisions –– Make Faster Decisions –– Improve Your Job Performance 16. How frequently do you regret decisions you’ve made where information/analytics contributed to the decision? 17. To what extent do you agree with the following: All of the information I need to do my job is available to me? 18. To what extent do you agree with the following: In my daily work, I rely heavily on the results of analysis (charts, graphics, reports, dashboards, etc.) to support my business decisions, processes and deliverables? 19. To what extent do you agree with the following: The reports and dashboards that are pushed to me are very valuable and I read, analyze and use them extensively? 20. To what extent do you agree with the following: The reports and dashboards that I pulled are very valuable and I read, analyze and use them extensively? 21. If information you would like to have as an input to making a decision is not available in a standard report or dashboard, how willing are you to go look for the information?
14. How effective a contributor is information and/or analytics to your organization’s ability to do the following activities? –– Product or service development –– Improving product or service profitability –– Evaluating and prioritizing investment proposals –– Developing a corporate or business unit strategy –– New market identification and market strategy development –– Identifying and targeting customer segments –– Forecasting demand for products, services, or resources
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
129
IV. Investment and Resource Level
V. Quality of Information
1. What is your organization’s total budget for the following projects/initiatives/capabilities in 2011? Include all spend both inside and outside of IT. –– Business Intelligence –– Information Management –– Collaboration
1. To what extent do you agree with the following: “The information from corporate sources is in a usable format and does not require modification or manipulation?”
2. At which groups are your analytics investments and initiatives targeted? 3. How many staff are part of the information management group? 4. Rank the following four capabilities in terms of effort and investment. –– Visualization –– Analysis –– Filtering –– Aggregation 5. How are analytics projects and initiatives approved? –– Very formal business case review –– Somewhat formal business case review –– No formal business case review –– Piggyback on other IT/business initiatives –– Other 6. Which of the following investment approval processes do you use for analytics investments and initiatives? –– A business-enabling capital investment –– An IT infrastructure investment –– An R&D investment 7. How are information management projects and initiatives funded? –– As a business-enabling capital investment –– As an IT infrastructure investment –– As an R&D investment 8. How are information management projects and initiatives approved? –– Very formal business case review –– Somewhat formal business case review –– No formal business case review –– Piggyback on other IT/business initiatives –– Other, please indicate
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
2. To what extent do you agree with the following: “I have a good understanding of the definitions of the information from corporate sources?” 3. To what extent do you agree with the following statement: “There is a single version of the truth across information from all corporate sources?” 4. To what extent do you agree with the following: “The information from corporate sources is directly relevant to business decisions, processes, and deliverables in my daily work?” 5. To what extent do you agree with the following: “The information from corporate sources is always timely and up-to-date enough to be useful in my daily work?” 6. To what extent do you agree with the following: “The information from corporate sources is always precise and accurate enough to be useful in my daily work?” 7. What level of confidence do you have in the quality (cleanliness and accuracy) of information from each of the following? –– Human Resources –– Finance –– Legal and Compliance –– Marketing –– Sales –– Customer Service –– Supply Chain and Operations –– Research and Development 8. For functions where you have low or no confidence in the quality of their information, what are the reasons behind your lack of confidence? –– Lack of transparency in how information was collected –– Lack of transparency in how information was stored –– Lack of transparency in how information was analyzed –– Multiple versions of the same information exist –– Lack of trust in systems –– Lack of common vocabulary/standard definitions
Overcoming the Insight Deficit
130
–– There are no means to assess or verify information quality –– Other 9. For which of the following has your organization established standard definitions? –– Information that is shared between functions –– Information that is shared between business units –– Information that is deemed critical to the business –– Information that is used most frequently –– Information that is stored in multiple systems or repositories –– Information where there is frequent confusion regarding the definition 10. Is each of the following information types defined in a data dictionary? 11. What steps does IT take to remedy information quality issues (such as duplicates, errors, outliers, contradictions, inconsistencies, missing values) in relation to the information that is entered or maintained in the data warehouse? –– There is an information quality framework that establishes quality standards and guidelines for information to be added or kept in the data warehouse or data marts. –– The information is fixed in the source systems before entering it into the data warehouse or data marts. –– The information is fixed both in the source systems and during information movement. –– The information is fixed in the data warehouse or data mart with the fixes fed back to the source systems. –– The quality of information in our data warehouse is reviewed or audited on an ongoing basis. –– The quality of information in our data warehouse or data mart is reviewed or improved as and when business users raise quality concerns. –– The information quality process is integrated into the ETL process to maintain information quality thresholds. 12. To what degree are information quality governance and standards automated through technology? 13. Which of the following describes the information quality standards within your data warehouse or data marts? –– All information has the same standards for information quality.
V. Quality of Information (Continued) –– Information for certain functions (e.g., Finance, HR, supply chain) has higher standards for information quality. –– Information that is used more frequently has higher standards for information quality. –– Information that has been deemed to be more valuable to the business has higher standards for information quality. –– Information that will be used for certain types of analysis has higher standards for information quality. –– Information that will be used by more senior or executive staff has higher standards for information quality. –– Information from certain corporate systems has higher standards for information quality. 14. Which of the following describes the information timeliness standards within your data warehouse or data marts? . –– All information has the same standards for information quality. –– Information for certain functions (e.g., Finance, HR, supply chain) has higher standards for information quality. –– Information that is used more frequently has higher standards for information quality. –– Information that has been deemed to be more valuable to the business has higher standards for information quality. –– Information that will be used for certain types of analysis has higher standards for information quality. –– Information that will be used by more senior or executive staff has higher standards for information quality. –– Information from certain corporate systems has higher standards for information quality.
VI. Information Usage Patterns 16. Who in your organization is primarily responsible for enforcing information quality standards and policies? –– Business leaders –– Subject matter experts in the business –– IT –– Information Management group –– Analytics group –– Shared services group –– No one 17. On which of the following characteristics do you tag information in your data warehouse or data marts? –– Individual author of the information –– Function that created the information –– Date and time the information was created –– Cleanliness and timeliness standards for the information –– Relevant governance processes –– Frequency information is used –– Purpose of the information –– Types of analysis information is used for –– Types of decisions information is used to make –– Standards reports or dashboards information is used in –– Other –– None
1. How frequently do you use each of the following? –– Workforce Information –– Financial Information –– Risk Information –– Customer Information –– Product, Materials, and Suppliers Information –– Research, Product Development and Testing Information –– Supply Chain Information –– Demand Information –– Web Information –– Consumer and Marketing Information –– Customer Service Information –– Sales Operations Information –– Economic and Industry Information –– Financial Market and Trading Information –– Assets and Maintenance Information –– Business Process Efficiency, Service Levels, and Quality Information 2. Which of information types are most important to you in your daily work? 3. Is your team or function responsible for owning and/ or generating a significant proportion of the following information types? 4. How frequently do you consult each of the following in your day-to-day work? –– Operational Structured Information –– Analytical Information –– Unstructured Information –– Undocumented information from colleagues or experts –– External company-endorsed sources –– External non-company endorsed sources
15. Who in your organization is primarily responsible for defining information quality standards and policies? –– Business leaders –– Subject matter experts in the business –– IT –– Information Management group –– Analytics group –– Shared services group –– No one
5. How many different sources of information/analysis do you use on a weekly or more frequent basis? –– Corporate operational systems –– Corporate unstructured information systems and document repositories, excluding e-mail –– Reports, dashboards, data/information warehouses, or data marts On average, how much of your time at work do you spend finding information or conducting analysis?
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
131
VI. Information Usage Patterns (Continued) 6. On average, how much of your time at work do you spend finding information or conducting analysis? 7. Of the time you spend finding information and conducting analysis, what percentage is spent finding information? 8. How frequently do you undertake each of the following types? –– Descriptive Analytics—The process of using analytics to describe what happened in the business (using reports and dashboards) –– Predictive Analytics—The process of using analytics to predict what will happen in the business (using data mining, pattern recognition and alerts, Monte-Carlo simulation, forecasting, predictive modeling) 9. How much time do you spend obtaining information or conducting analysis that does not ultimately lead to a decision or productively inform your or your colleagues’ work? 10. To what extent do you agree with the following: When I need specific information, I know where to find it? 11. To what extent do you agree with the following: In instances where someone in my organization has conducted similar analysis before, I am able to find documentation, lessons learned, or notes on their process and experiences? 12. To what extent do you agree with the following: For projects/tasks where someone in my organization has worked on something similar before, I am able to find documentation, lessons learned, or notes on their process and experiences? 13. Which of the following formats/channels do you use to access each information type? –– PC-based web browser –– Mobile devices –– E-mail –– Online system –– Paper 14. If you access information from mobile devices, how do you rate their value and functionality relative to when accessing them on a standard desktop or laptop?
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
132
VII. Staff Knowledge and Competencies –– Significantly higher value due to speed/convenience of mobile device accessibility –– Somewhat higher value –– Same value –– Lower value, as some information is missing –– Significantly lower value, as key information is missing –– N/A—I do not have mobile access. 15. Do you analyze information obtained from any of the following sources? –– Smart Networks and Sensors –– Radio-Frequency Identification –– Global Positioning System –– Collaboration Tools –– External Social Media 16. Do you provide information to corporate systems for others’ consumption (e.g., inputting information into a CRM system or sharing information/knowledge within a collaboration system or document repository)? 17. Do you have the opportunity to provide feedback on information sources and business intelligence/analytics tools for improvement or functionality enhancements? 18. How does your organization determine end-users’ needs for information with regard to tools, capabilities, quality, etc.? –– Link information needs to strategic decisions. –– Derive information needs from strategic goals or targets/ KPIs. –– Define information needs based on workflow processes. –– Keep up to date of market/industry/functional trends. –– Directly ask knowledge workers via survey. –– Directly ask knowledge workers via focus groups. –– Track information requests from knowledge workers to understand trends in information needs. –– Take in feedback from knowledge workers in terms of their satisfaction with information (to inform future needs).
1. Is conducting analysis explicitly mentioned in your job description? 2. Which of the following skills do you most value in staff responsible for conducting and providing analysis to you and your team? –– Business knowledge and understanding –– Quantitative skills –– Technical skills –– Relationship and advisory skills –– Training and coaching skills 3. How frequently do you interact with staff from each of the following functions for work-related purposes? –– Human Resources –– Finance –– Legal and Compliance –– Marketing –– Sales –– Customer Service –– Supply Chain and Operations –– Research and Development 4. To what degree are your decisions made independently? –– I make the majority of decisions completely independently. –– I take my peers’ perspectives into account but ultimately make decisions independently. –– I make decisions in conjunction with peers to incorporate the group’s perspectives. –– Decisions are largely dictated by my manager or other leaders. 5. Which of the following best describes the methods you use to conduct analysis? –– I develop specific theories that I then test using analysis. –– I have some theories in mind that I test using analysis, but also manipulate the information in other ways to see if interesting trends or insights emerge. –– I manipulate the information without a clear idea of what I expect to find, and see if interesting trends or insights emerge.
VIII. Information Strategy and Governance 1. Does your organization have a data steward role? 2. If you responded “Yes, formal data steward role,” how many data stewards does your organization have? 3. Indicate if the following are available to knowledge workers through self service. –– Data definition –– Name of the data steward –– Source of the data –– Details of transformations performed on the data –– None 4. To what degree do information security and access policies inhibit your ability to obtain information that is valuable/ important to your analysis or decision making? 5. How mature are business processes within your function/ business unit? –– Non-existent: No business processes exist. –– Ad hoc: Individuals and/or teams have processes that are followed irregularly. –– Undocumented: Individuals and/or teams follow processes that are not standardized nor documented. –– Defined: Processes used across business units and/or functions are documented but not enforced. –– Managed: Processes are documented and enforced. –– Optimized: Processes are documented, enforced, and measured to drive improvement. 6. Does your organization have a strategic plan for analytics and information management? –– Yes, we have a standalone strategic plan for analytics and information management. –– Yes, analytics and information management are incorporated into a broader IT strategic plan. –– Yes, analytics and information management are incorporated into a broader business strategic plan. –– No, we do not have a strategic plan for analytics and information management.
7. Which components and initiatives are included in your strategic plan for analytics and information management? –– Formalizing the structure, roles, and responsibilities for analytics –– Implementing information architecture requirements –– Implementing information governance requirements –– Implementing information quality standards –– Implementing new analytical tools or capabilities –– Conducting training on using analytical tools –– Developing knowledge worker analytical skills –– Developing an analytical culture –– Measuring analytical performance 8. Are your analytics and information management initiatives aligned to business strategic goals? 9. Which groups are involved in developing the strategic plan for analytics and information management? –– IT –– Finance –– Marketing –– Corporate Planning –– Business Shared Services –– Business Unit –– Standalone Information Management Group –– Sales –– Supply Chain –– Other, please indicate –– N/A—No Strategic Plan 10. If IT is involved in developing the strategic plan for analytics and information management, what role do the following play? –– CIO –– Project Management Office –– Enterprise Architecture –– Infrastructure –– Applications –– Security –– Dedicated Information Management Team –– Other, please indicate: –– N/A—IT not involved in strategic plan
11. Rank order the business motivations for making analytics investments . –– Product or service development –– Improving product or service profitability –– Evaluating and prioritizing investment proposals –– Developing a corporate or business unit strategy –– New market identification and market strategy development –– Identifying and targeting customer segments –– Forecasting demand for products, services, or resources –– Other 12. Does your Enterprise Architecture group have information architects? 13. How many information architects does your EA group have? 14. How would you assess your organization’s current level of proficiency in the following disciplines? –– Data Strategy and Governance –– Data Stewardship –– Data Standardization –– Master Data Management –– Data Quality Management –– Reporting and Analytics –– Information Lifecycle Management 15. When IT provides a new report or analytics tool, does the project plan include formal steps to change business processes or workflow to make better use of it? 16. Are the implications of information and analytics requirements incorporated into other IT projects? –– Yes, before the project has been approved –– Yes, when building out requirements for the project –– Yes, after the project has been deployed 17. Which of the following best describes your organization’s approach to master data management? –– No formal initiative is currently in place or planned around master data management. –– Master data is defined and managed at the individual system level. –– Siloed, department or group-level master data management strategy has been defined and established.
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
133
VIII. Information Strategy and Governance (Continued) –– A unified, enterprise-level master data management strategy has been defined and established. 18. Which of the following do you use as part of your master data management? –– Source identification –– Data collection –– Data transformation –– Normalization –– Rule administration –– Error detection and correction –– Data consolidation –– Data storage –– Data distribution –– Data governance –– None 19. Who owns or stewards master data subjects (e.g., customer, product)? –– IT –– Business unit –– A central or cross-functional master data management or governance group –– Other –– None 20. Does your organization have an enterprise data model for organizing information? 21. Which information types are included in the enterprise data model? 22. If your organization does use a data model, which of the following components are included in your data model? –– Data architecture –– Data modeling –– Data properties –– Data organization –– Data structure –– Data model theory –– N/A—no data model 23. If your organization does use a data model, is there a process for reviewing new applications, tools, and databases for alignment with the enterprise data model? –– Yes, before they are developed or purchased CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
Overcoming the Insight Deficit
134
IX. Technical Environment –– Yes, after they are developed or purchased –– No –– N/A—no data model 24. Does your organization have an information architecture in place? –– Yes—We have documented both the current and future state architectures for the enterprise and have a roadmap in place against which we are moving toward the future state. –– Yes—We have architected the current and future states for different business units or areas of the enterprise but have yet to aggregate into a unified whole. –– Yes—We have an initiative in place but are still in the early stages. –– No—We have a vision for what we want to do but no formal initiative. –– No—We don’t have an information architecture and don’t plan to create one. 25. Which of the following best describes your data governance processes? –– We have formal governance structures in place that are understood and adhered to across the enterprise. –– We have formal governance structures in place that are understood and adhered to, but only in parts of the organization. –– We have some formal and some informal governance processes in place; adherence is mixed. –– Governance processes are generally ad hoc and adherence is generally low across the organization. –– There are no formal and few informal governance processes in place. 26. Rate the following end user requirements that the Information Management group is working to solve in order of urgency). –– Information quality or accuracy –– Information timeliness –– Information consistency –– Information relevance –– Information accessibility or ability to search –– Usability of reporting and analytical tools –– Flexibility to use personally preferred tools, e.g. via cloud or mash-ups
1. Do you use any document repositories that are not provided and supported by corporate IT (e.g., Google documents, Dropbox, iDisk) for business purposes? 2. Do you use any business intelligence/analytics tools that are not provided and supported by corporate IT? 3. If you answered “Yes,” why do you use tools not provided and supported by corporate IT? –– IT does not provide an alternative tool. –– The non IT provided tool is easier to use. –– The non IT provided tool is more sophisticated. –– The rest of my team uses the non IT provided tool. –– Other 4. Which of the following best describes the analytical tools available to you? –– There is no tool available to me. –– I only have access to Excel. –– There is a single tool I have access to that is consistent across the entire company. –– There is a single tool I have access to that is consistent across my business unit/function. –– There are multiple tools with similar functionality I have access to and I am free to use the one I prefer. 5. Which of the following collective intelligence tools are currently being used at your organization? –– Discussion forms –– Blogs –– Wikis –– Prediction markets –– Ratings and online surveys –– Other, please indicate 6. Which of the following best describes your organization’s approach to selecting analytical tools? –– Tools are centrally selected and deployed across the organization. –– Tools are centrally selected and deployed across the organization, but we customize capabilities to meet the needs of different functions/business units. –– Tools are centrally selected, but different tools are deployed to different business units/functions based on their unique needs.
IX. Technical Environment (Continued) –– Business units/functions choose their own tools with guidance from IT and the information management and/ or analytics group. –– Business units/functions choose their own tools and IT/ Information Management are not/rarely involved in the decision process. –– Individual knowledge workers choose their own tools. 7. How many different analytical tools are in use? –– Across the company –– Maximum within a business unit –– Maximum within a function –– Per information type 8. How many different operational systems are in use? –– Across the company –– Maximum within a business unit –– Maximum within a function –– Per information type 9. How many different corporate information repositories are provided? –– Across the company –– Maximum within a business unit –– Maximum within a function –– Per information type 10. What is the total volume of storage in your organization (include primary, backup, and archive) in TB? 11. What is the annual percentage growth rate of total storage volume in your organization (include primary, backup, and archive)? 12. Which of the following emerging technologies and capabilities have you implemented? –– In-memory analytics –– Data mash-ups –– Cloud/SaaS BI or analytics –– Columnar databases –– Mobile BI –– Interactive visualization –– Enterprise search –– Semantic technologies –– Complex event processing –– Advanced analytics
13. Which of the emerging technologies and capabilities are in use (either broadly or narrowly)? 14. To what extent are analytics and information management solutions deployed using a test and learn approach? 15. Does your organization have an enterprise data warehouse? –– Yes—data from both structured corporate systems and unstructured information repositories are all stored in a single data warehouse. –– Yes—structured information from all corporate systems are stored in a single data warehouse, but Unstructured information is stored elsewhere. –– Yes—data from some corporate systems are stored in a single data warehouse, but there is a significant amount of data outside the data warehouse. –– No—data is stored in several data marts. 16. What is the total number of data warehouses within your organization?
contributing to its removal? Please check all that apply. –– No longer required to retain the information for regulatory and compliance purposes –– Information no longer has business value. –– Too costly to continue storing the information 20. Does IT integrate external sources of information into the data warehouse? –– Yes—we proactively search for external sources of information that could be valuable to the business and integrate it into the data warehouse. –– Yes—when the business recurrently uses an external source of information, we integrate it into the data warehouse. –– Partially—we make some external sources of information available internally, but it is kept separate from the data warehouse. –– No—we do not integrate external sources of information into our environment. 21. Do you have a master data management tool?
17. What is the total size of data warehouses within your organization in TB? 18. Which of the following best describes the information that is included in your organization’s data warehouse? –– All information generated by our organization is included in the data warehouse indefinitely. –– All information generated by our organization is included in the data warehouse, but it is retired and removed after a defined period of time. –– Only information deemed valuable to the business is included in the data warehouse. –– Only certain information is included based on technical feasibility. –– Only certain information is included based on business requests. –– N/A—not in data warehouse 19. If your organization removes information from the data warehouse after a period of time, what are the factors
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic
135
X. Demographics 1. What is your age? 2. What is the highest level of education you obtained? 3. In which subject or discipline did you achieve an undergraduate- or graduate-level university degree? 4. What is your tenure at your current organization? 5. Which of the following most closely describes your current level in the organization? 6. Which of the following most closely describes your current function in the organization? –– Communications –– Corporate Strategy –– Customer Service/Contact Center –– Finance and Accounting –– Financial Services –– General Management Functions –– Human Resources –– Information Technology –– Legal and Compliance –– Marketing/Market Research –– Production/Operations/Procurement –– Professional Services –– Real Estate and Facilities –– Research, Development, and Engineering –– Retail –– Sales 7. Indicate any additional functions you have worked in previously. 8. In what country do you primarily work? 9. In which country is your organization headquartered? 10. Where do you spend the majority of your time working? 11. Which of the following best describes your attitude toward new technologies? 12. Which of the following best describes your organization’s industry? 13. How geographically dispersed is your organization? 14. What is the ownership structure of your organization?
CIO Executive Board IT Practice www.cio.executiveboard.com CIO0361911SYN
15. How high is the regulatory burden faced by your organization? 16. What is the level of business integration at your organization? 17. What is the level of information intensity of each of the following at your organization? 18. What is the overall annual revenue (in millions of your currency) for your organization? 19. To what degree does your organization handle sensitive or confidential information? 20. How many employees does your organization employ? 21. In how many countries does your organization have operations? 22. Which of the following best describes your IT organization’s structure? 23. How many IT users does your organization support? 24. How significant a part of your job responsibilities is conducting analysis?
Overcoming the Insight Deficit
136
Related Documents
Executive Board
April 2020 12
2013 Bundy Cio Executive Briefing Online
April 2020 5
Insight
December 2019 74
Insight
November 2019 62
