Market Research -an Overview - Knol

MARKET RESEARCH – A BRIEF OVERVIEW Saktishree D.M.

INTRODUCTION The marketing research process includes the systematic identification, collection, analysis and distribution of information for the purpose of knowledge development and decision making. The reasons and times at which your company or organization might consider performing marketing research varies, but the general purpose of gaining intelligence for decision making remains constant throughout. As a company or organization, the overwhelming majority of research you are currently considering likely revolves around your customers: * * * * * * *

Current customers Prospective customers Lost customers Members Community Employees (internal customers) Shareholders (internal customers)

Whether you are creating a new marketing research program or perhaps revising an existing marketing research program, what are the steps you should take? While there are dozens of little steps along the way, each of those steps fits into one of the 6 major steps of the marketing research process. They are:

Step 1- Identifying and defining your problem If you are considering conducting marketing research, chances are you have already identified a problem and an information need. This step is always the start of the marketing research process. At this point, the problem will have been recognized by at least one level of management, and internal discussions will have taken place. Sometimes, further definition of the issue or problem is needed, and for that there are several tools you can use. The most common tools are internal and external secondary research. Secondary research intelligence consists of information that was collected for another purpose, but can be useful for other purposes. Examples of internal secondary research are sales revenues, sales forecasts, customer demographics, purchase patterns, and other information that has been collected about the customer. Often referred to as data mining, this information can be critical in diagnosing the problem for further exploration and should be leveraged when available and appropriate. The amount of internal secondary information that can be applied is typically limited. External secondary research is typically far more available, especially since the Internet age. Most external secondary information is produced via research conducted for other purposes, financial performance data, expert opinions and analysis, corporate executive interviews, legal proceedings, competitive intelligence firms, etc.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

Leading sources for external secondary research resources include:           

Newspapers/Magazine Articles (business and vertical trades) Television Newsletters Competitive Intelligence Firms Industry Reports Trade Associations Business Directories Government Publications & Websites Search Engines Competitive Websites Friends & Colleagues

Step 2- Developing your approach Once your problem is better defined, you can move onto developing your approach, which will generally be around a defined set of objectives. Clear objectives developed in Step 1 will lend themselves to better approach development. Developing your approach should consist of honestly assessing you and your team’s market research skills, establishing a budget, understanding your environment and its influencing factors, developing an analysis model, and formulating hypotheses. Project Analysis o o o o o

How difficult is the project to execute? Is it a large sample (500+) or small sample (<200)? Will the project need advanced analysis? What are the likely methodological approaches? Is in-depth and detailed reporting or executive summary reporting needed?

Skills Analysis o o o

Is there in-house market research available to meet project needs? Is the in-house market research expertise available during the given timeframe? What parts of the market research process can be handled internally?

Budget Analysis o o o o o

Is this a strategic problem/issue or a tactical one? Is it a $20,000 project or $200,000 project — what is the information worth? Where will the budget come from, and can it be shared between departments? Who are those most likely to benefit from the research, and likely those most willing to fund the project? In what timeframe will budget be available?

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

Environment o o o

What is the overall economic environment? What is the economic environment relative to your products/services? What is the governmental environment (regulatory, etc.)?

Overall Theory o o o o

What is your overall theory and hypothesis? What do you intend to prove or disprove? What actions are your company willing to take based upon survey results? What are the internal/external roadblocks that will need to be overcome to drive results?

Step 3- Research design Based upon a well-defined approach from Step 2, a framework for the designing your marketing research program should be apparent. This step is the most encompassing of all steps in the research process, requiring the greatest amount of thought, time and expertise — and is the point at which those less experienced with market research will obtain assistance from an internal market research expert or perhaps partner with an external marketing research provider. Since the intelligence eventually gained from the research is so closely related to the selected research design, this is the single most import step in the research process and the step most vulnerable to the typical research errors. Research design includes secondary information analysis, qualitative research, methodology selection, question measurement & scale selection, questionnaire design, sample design & size and determining data analysis to be used. Elements of Research Design Include: The Questionnaire Design Process Before you get to the questionnaire design process, you must have already taken Steps 1 & 2 in the market research process. Simply providing questionnaire design samples that might include several sample questions for inclusion in your questionnaire does not completely do justice to the science and art behind the questionnaire design process. Every company or organization that considers performing market research will have different issues, which is why it is so difficult to find a single questionnaire design sample. Since there are some similarities between more typical types of research projects, however, we have provided on this website several more general sample questions to consider including in your various survey. It is highly recommended that you and your team go through the entire questionnaire design process to make sure that any survey instrument you create will be an effective tool for gathering the information you need.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

The Questionnaire Design Process       

Determine the information needed Determine which survey methodology is most appropriate for your needs Specify individual questions to be asked Decide what question structure, scale, and wording is appropriate Properly order the questions within the questionnaire Proof and pretest survey with small sample to check performance Make changes based on pretest and execute survey

Measuring and Scaling Creating a survey questionnaire that is capable of effectively collecting accurate data is a difficult process with many opportunities for making some of the more common market research errors. Many less experienced market researchers may believe that creating a questionnaire is simply the act of coming up with questions and putting a pen to paper, but that is dangerous assumption. Creating a questionnaire requires as much science as art, and incorporating those two elements into a high-quality survey that will draw a good response rates while effectively collecting accurate data often takes time and experience. When creating a survey questionnaire, there are basic types of scale questions to have in your tool box. They are: o

o

o

o

Nominal — when numbers are used to identify objects, such as social security number, license numbers or daily customers. In this case, the number acts mostly as a data tag, typically for identification. Ordinal — when numbers are used to indicate the relative position, but not indicate the magnitude of the difference between those positions. An example of this would be rankings in which items are listed by priority, say first through fifth, or competitive events where the quantifiable difference in perception between #1 and #2 is unknown. Interval — when a rating scale is used and the zero point is arbitrary. An example of this is satisfaction scores (satisfaction of 3 on a scale of 1 to 5) as well as most other attitude and opinion questions, regardless of the scale used (3, 5, or 10 point). Unlike ordinal, the difference between each data point is fixed. Ratio — the most useful of all of the scales, ratio scales allow the researcher to incorporate each of the above listed scales into one (nominal, ordinal and interval). The key difference with ratio is that unlike the interval scale, it is anchored with an absolute zero point. Examples of ratio questions are market share, income group, age group, etc.

If you are creating a survey questionnaire from scratch, it is important to be mindful of these scales as each one lends itself to a particular type of data analysis.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

Sample Size Calculator When you know how accurate you want your data to be, and you know the size of your target population, then you need to calculate how big a sample you need to draw from that population in order to have survey results that can be projected onto the target population. See below for a basic explanation of the relevant statistic procedures. About the Sample Size Calculator Sometimes you know how accurate you want your data to be, i.e., you have a desired margin of error (see Basic Statistical Testing for how to calculate the margin of error), and you know the size of the population you want to get information on. Then you need to know how many people in that population you need to actually talk to in order to have results that can be projected onto the entire population with the desired amount of accuracy. To calculate the desired sample size or percentages, you’ll need to know error, and, if known, the estimated you don’t know, then you can guess scenario.

when your results are going to be in proportions your desired confidence level, desired margin of proportion or percentage you expect to find. If 50 percent, since a 50-50 split is the worse case

To calculate the desired sample size when your results are in means or averages, you don’t need to give an estimated result, but you will need to estimate a standard deviation for your data set. Basic Statistical Testing To understand the results of any survey, two calculations are necessary: the margin of error and the statistical significance of any differences found. To calculate the margin of error … To understand how accurate your survey results are, one must calculate how much error is likely given the size of the sample you are surveying in relationship to the total population. The margin of error is the amount of error one could expect to find, due to just chance, above or below the actual figure obtained in the survey results. To calculate an acceptable margin of error, you must first select the confidence level you want for your results, and you must enter the sample size you are surveying and the total size of the population from which the sample is drawn. An important guideline is the confidence interval or confidence level. That tells you just how confident you can be that the error rate you find reflects simply errors due to chance or sampling variation and not actual differences in the total population of interest. If you want to be very sure of your findings – say, if life-or-death decisions are to made as a result – then you will want a 99 percent confidence level. That means that you can be 99 percent confident that there is only a so-many percent likelihood (your margin of error) that the differences do not reflect actual differences

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

but are due to chance. If you are looking for directional advice, you may want to go with a 90 percent confidence interval, as that will give you many more statistically significant results. Most commonly, a 95 percent confidence level is used. The sample size is the number of people you are surveying. A sample is a portion of a total population. Sometimes you know the actual number of the total population, say if you are surveying a percentage of your customers or association members. When you don’t know the total but you know it’s more than the number you are surveying, then you will want to select an infinite population. Most of the time, you will probably want to calculate a margin of error for the percentage or proportion of the sample choosing a particular answer in the survey. Sometimes, however, when you’ve asked your respondents for a number – such the number of hours they are on the Internet – then you will want to look at the average or mean number, and calculating the margin of error for a mean is somewhat different. To do this, you still must know the confidence level you desire, the sample size and the population size, as you do for calculating margin of error with proportional or percentage data. But you also must know the standard deviation for your data set. The standard deviation is the square root of the variance, which is based on the sum of squared differences between each score and the mean. Most calculators and spreadsheet programs can calculate the standard deviation for your data set. Once you know your margin of error, you can say this about your data: If one were to pull 100 samples from the population and ask each group the same questions, you can be certain that 95 percent of the time (or whatever your confidence level) you will get answers that are within five percent (or whatever your margin of error) of the answers you got this time. To test whether the difference between two results is significant … Two proportions or percentages, or two means, may be far apart in actual numbers but not so far apart as to be statistically significant if they come from samples of far different sizes. To be sure the difference is due to your independent variable – the issue or action you are testing to see if it makes a difference in your dependent variable – you must test the statistic to make sure the difference could not be the result of chance or sampling variation. A simple way to determine whether two proportions or percentages are truly different is to conduct a z-test. To calculate the z-score, one needs to know the desired confidence level, the two actual percentages, the size of the two samples used and the size of the total population. One also should be aware of whether one sample is included in the other sample (called a subsample), which occurs, for instance, when a sub-group is being compared with the total. For instance, when one wants to compare the percentage of one group answering a particular question yes with the percentage of the total sample answering yes, then the z-score must be calculated differently than when one compares the percentages of two different groups. The best way to determine whether two means, or averages, are truly different is to conduct a t-test. To calculate the t-statistic for means, one must know the desired

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

confidence level, the two means, the two sample sizes, the total population size, and the standard deviations for each mean. When the z-score for two proportions or the t-score for two means are high enough, then one can say with some (90%, 95% or 99%) confidence that the difference between the two is due to the action of the independent variable on the dependent variable and not simply due to chance or sampling variation. Step 4- Collecting the data Often called data collection or survey fielding, this is the point at which the finalized questionnaire (survey instrument) is used in gathering information among the chosen sample segments. There are a variety of data collection methodologies to consider. Selecting which is the most appropriate data collection methodology for a particular research project takes place during Steps 2 & 3 of the market research process. Data collection typically begins with field testing the final questionnaire with a small portion of the respondent sample to make sure it is gathering information correctly. Then data collection can be fairly automatic throughout the remainder of the data collection process. When quota groups and/or sample subgroups are being screened for, data collection will require more oversight, maintenance time and cost. Regardless of the data collection methodology chosen, the data collection process often takes from 25 percent to 50 percent of the total time needed to complete a research project. Market research data collection methods:         

Computer Assisted Telephone Interviewing (CATI) Internet survey Interactive Voice Response (IVR) Mail survey Mall intercepts Traditional telephone interviewing Internet panel Mail panel In-home panel

Step 5- Performing data analysis Any questionnaire data analysis will depend on how the questionnaire was constructed. Less complex questionnaire data analysis can be handled with any of a number of office suite tools, while more complex questionnaire data analysis requires dedicated market research analysis programs. Types of statistical data analysis that might be performed are simple frequency distributions, crosstab analysis, multiple regression (driver analysis), cluster analysis, factor analysis, perceptual mapping (multidimensional scaling), structural equation modeling and data mining. The more complex the needed level of statistical data analysis is, the more time and cost it will take to execute.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

Advanced Statistical Analyses ANALYSIS

DESCRIPTION

EXAMPLE APPLICATION

Multiple Regression (Driver Analysis)

Describes the relationship of each variable in a set (and the set of variables as a whole) to a single variable.

Determine key "drivers" of overall customer satisfaction with your service.

Cluster Analysis

Identifies homogeneous sub-groups within a much larger group of respondents.

Identify customer profiles or market segments, groups of customers or potential customers who make similar decisions and perceive products and services similarly.

Factor Analysis

Reduces a complicated data matrix into its more basic structural essentials.

Uncover basic dimensions employees use to evaluate how satisfied they are working for your organization.

Perceptual Mapping (Multidimensional Scaling)

Extracts multiple dimensions from a variable set and positions concepts within that space.

Visualize how customers mentally organize competitors in your product or service category and your brand's position relative to your competitors.

Structural Equation Modeling

Tests how well observed data confirm an entire theoretical model.

Describe the process by which customer loyalty is built for your particular product or service category.

Data Mining

Detects useful and sometimes unexpected patterns among variables in a data set.

Increase revenues by selling your products.

cross-

Step 6- Reporting and presentation Market research reporting and presentation is easily the second most important step, if not the first. Any business critical information and knowledge that comes from your market research investment will be limited by how it is presented to decision makers. There are as many reporting styles as there are market research reports, but some are definitely better than others, and there are definitely trends to be aware of.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

TOOLS & CALCULATORS

Basic Statistical Testing Already discussed in Step - 3 Sample Size Calculator When you know how accurate you want your data to be, and you know the size of your target population, then you need to calculate how big a sample you need to draw from that population in order to have survey results that can be projected onto the target population. See below for a basic explanation of the relevant statistic procedures. About the Sample Size Calculator Sometimes you know how accurate you want your data to be, i.e., you have a desired margin of error (see Basic Statistical Testing for how to calculate the margin of error), and you know the size of the population you want to get information on. Then you need to know how many people in that population you need to actually talk to in order to have results that can be projected onto the entire population with the desired amount of accuracy. To calculate the desired sample size when your results are going to be in proportions or percentages, you’ll need to know your desired confidence level, desired margin of error, and, if known, the estimated proportion or percentage you expect to find. If you don’t know, then you can guess 50 percent, since a 50-50 split is the worse case scenario. To calculate the desired sample size when your results are in means or averages, you don’t need to give an estimated result, but you will need to estimate a standard deviation for your data set.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

COMMON ERRORS Common Marketing Research Errors Since the usability of any market research depends upon the accuracy of the results, error control plays a critical role in the research process. Even experienced marketers and market researchers easily make several common mistakes while taking the six steps of the research process. Avoiding many of the simpler marketing research errors takes only common sense, but avoiding many of the more complex mistakes requires a much deeper level of awareness. 1. Sampling error 2. Non-sampling error a. Non-response error b. Response error i. Researcher error 1. Problem definition 2. Population definition error 3. Sample design error 4. Questionnaire error a. Structure error b. Language error (ambiguous, leading, assumptive, etc.) c. Measurement/scale error 5. Data analysis error 6. Reporting error ii. Interviewer error 1. Questioning error 2. Cheating error 3. Population definition error iii. Respondent error 1) Sampling error — the estimated inaccuracy of the results of a study when a population sample is used to explain behavior of the total population. 2) Non-Sampling error — all the sources of bias or inaccuracy in a study besides sampling error. Examples: leading by the interviewer, recording/data entry errors (see below) 3) Non-Response error — the estimated inaccuracy that results from a systematic difference between those who do and do not respond to the measurement instrument. 4) Response errors — the estimated inaccuracy that can be introduced potentially by the researcher, the interviewer or the respondent. 5) Researcher errors — the error that the researcher can make in survey design & execution throughout the 6 Steps of the research process.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

6) Problem definition — the error made during Step 1 of the research process when the researcher misinterprets, misunderstands or does not properly define the issue/problem and related information need. 7) Population definition error — the difference between the actual population relative to the issue/problem and the population as defined by the researcher. For example, estimating that the total target population is 50,000 when it is actually 10,000. 8) Sample design error — the estimated inaccuracy between the properly defined actual target and the population sampled. For example, mistakenly assuming an out of date telephone directory contains all current businesses, when many new businesses will have started/move into the area and others will have closed. 9) Questionnaire error — the total of errors made when creating the survey instrument (see below). 10) Questionnaire structure error — the error made when the structure and layout of the survey instrument leads to inaccurate responses. For example, when aided brand recall (brand identified to the respondent) questions are asked before unaided recall (brand not identified to the respondent) questions are asked, you have a questionnaire design that will significantly impact unaided recall results. Another example would be asking probing questions regarding viewpoints on potentially negative experiences before asking an overall satisfaction question, where overall satisfaction would be incorrectly affected by the recent recall of potentially bad experiences. 11) Questionnaire language error — the error made when the researcher uses incorrect language (ambiguous, leading, assumptive, etc) in the survey instrument so that respondents are influenced in their answers. Language errors severely limit the validity and usefulness of those questions and are the most common error for inexperienced marketing researchers. 12) Questionnaire measurement/scale error — the estimated inaccuracy that occurs when improper measurement and scaling techniques are used in the survey instrument. There are far too many measurement/scale errors to list here — the key is that for each type of question, there is a measurement/scale that is most appropriate, and sometimes it is intuitive and sometimes not. The right type of measurement/scaling question to use will depend upon the information being collected and the analysis that will be performed. A very common measurement error is not controlling order bias, which occurs when a list is not randomly rotated for a given question and respondents may exhibit a tendency to select the first few answers from a list over others. 13) Data analysis error — the error that occurs when analysis is incorrectly executed. Simple mathematical errors are common, which is why data analysis should be checked over by more than one qualified person for quality. A more significant data analysis error is when simple frequency reporting (straight number percentage reporting) is executed when far greater information can be mined from the results (often inexpensively) through additional analysis such as cross-tabulation analysis, multiple regression (driver analysis), cluster analysis, factor analysis,

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

perceptual mapping (multidimensional scaling), structural equation modeling tests, etc. 14) Reporting error — the best approach and program design combined with the best analysis is only as good as the researcher’s capability to synthesize and report on the results. The most common reporting error by far is the improper representation of the significant findings in a format conducive to creating management understanding and buy-in of survey results. It could be something as simple as poor language syntax to as complex as choosing the wrong results to report or not choosing the best way to graphically represent the results. More common in the current environment is not selecting the best delivery vehicle. For example, a quality online reporting system is much preferred when distributing results across a company that is geographically spread out. 15) Interviewer error —when a live interviewer improperly affects responses (see below) 16) Questioning error — when the interviewer improperly leads the respondent in any way through personal bias or any other improper delivery of questions. 17) Cheating error — when the interviewer falsifies any results. 18) Population definition error — when the interviewer does not randomly select potential respondents according to the methodology specified. 19) Respondent error — when respondents purposefully or mistakenly give incorrect answers to survey questions.

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

MARKETING RESEARCH COSTS

Market Research Costs Whether you are performing a market research project on your own or outsourcing it entirely, chances are you will need to present a market research proposal in order to justify the costs. The cost for conducting market research varies so widely, however, that obtaining a single sample research proposal is pretty much impossible. Why? Well, there are key reasons that every market research proposal looks different from the next. As with everything, the number one determining factor for estimating market research costs is how much you are willing and able to do in-house. There is a trade off, since the more you are capable of doing in-house, the less the overall project will cost you in hard dollars, but the more it will cost you in time and needed expertise. Many companies and organizations have a hard time determining what they are truly capable of and therefore have a hard time deciding what they can do in-house Another determining factor is that, while there is always a best approach and methodology for a particular market research project, there are usually several options to consider. It is critical to be truthful regarding budget, timeframe and a number of factors when putting together a RFP if you plan on outsourcing and soliciting market research proposals. What are some of the costs to consider? For Quantitative Step 1- Identifying and defining your problem & Step 2. Developing your approach o

Mostly consulting costs at this stage

Step 3- Research design o o o o o

Strategy design Questionnaire design Acquiring sample Incentives Programming or printing the survey

Step 4- Collecting the data o o o

Data collection Data cleaning Editing & coding of open-ended responses

Step 5- Performing data analysis o

Data analysis

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007

Step 6- Reporting and presentation o o o o

Executive summary Detailed reporting Online reporting Formal presentation

For Qualitative Step 1- Identifying and defining your problem & Step 2. Developing your approach o

Mostly consulting costs at this stage

Step 3- Research design & Step 4. Collecting the data o o o o o o o

Strategy design Discussion guide Recruiting participants Incentives Facility charges Eecording Food & drink

Step 5- Performing data analysis o

Data analysis

Step 6- Reporting and presentation o o o o

Executive summary Detailed reporting Online reporting Formal presentation

Market Research – A brief overview \ Saktishree D.M. \ Sep 2007