Marketing Research Notes Chapter11

RESEARCH METHODOLOGY

LESSON 11: ISSUES IN QUESTIONNAIRE Student’s today we will be continuing with the questionnaire design. Today we will be studying the Intensity Questions , bias in questions, bias in volunteer samples and levels of measurement , reliability and validity of findings. We have studied in the last lecture that the questionnaire is the means for collecting your survey data. It should be designed with your data collection plan in mind.

Strongly Disagree 1. Disagree 2. Undecided 3. Agree 4. Strongly Agree 2. On the whole, judges are honest.

Each of its three parts should take advantage of the strengths of questionnaires while minimizing their weaknesses. Each of the different kinds of questions is useful for eliciting different types of data, but each should be constructed carefully with welldeveloped construction guidelines in mind.

Strongly Disagree

Properly constructed questions and well-followed survey procedures will allow you to obtain the data needed to check your hypothesis and, at the same time, minimize the chance that one of the many types of bias will invalidate your survey results.

4. Strongly Agree

Before studying the types of bias first we will deal with the Intensity quest

A person who feels that laws are unjust would score lower than one who feels that they are just. The stronger the feeling, the higher (or lower) the score.

Intensity Questions and the Likert Scale As I have told you before,the intensity question is used to measure the strength of a respondent’s feeling or attitude on a particular topic. Such questions allow you to obtain more quantitative information about the survey subject. Instead of a finding that 80 percent of the respondents favour a particular proposal or issue, you can obtain results that show 5 percent of them are strongly in favour whereas 75 percent are mildly in favour. These findings are similar, but the second type of response supplies more useful information. The most common and easily used intensity (or scaled) question involves the use of the Likert-type answer scale. Likert-type answer scale It allows the respondent to choose one of several (usually five) degrees of feeling about a statement from strong approval to strong disapproval. The “questions” are in the form of statements that seem either definitely favourable or definitely unfavourable toward the matter under consideration. The answers are given scores (or weights) ranging from one to the number of available answers, with the highest weight going to the answer showing the most favorable attitude toward the subject of the survey. Illustration The following questions designed to measure the amount of “anti- law” feelings this : 1. Almost anything can be fixed up in the courts if you have enough money.

11.556

1. Disagree 2. Undecided 3. Agree The weights (shown by the numbers below the answers) are not shown on the actual questionnaire and, therefore, are not seen by the respondents.

The scoring is consistent with the attitude being measured. Whether “agree” or “disagree” gets the higher weight actually makes no difference. But for ease in interpreting the results of the questionnaire, the weighting scheme should remain consistent throughout the survey. One procedure for constructing Likert-type questions is as follows (adapted from Selltiz, et al., 1963; pp 367-368): 1. You being an investigator collect a large number of definitive statements relevant to the attitude being investigated. 2. You conduct and score a pretest of your survey. The most favourable response to the attitude gets the highest score for each question. The respondent’s total score is the sum of the scores on all questions. 3. If you are investigating more than one attitude on your survey, intermix the questions for each attitude. In this manner, the respondent will be less able to guess what you are doing and thus more likely to answer honestly. 4. Randomly select some questions and flip-flop the Strongly Agree-Strongly Disagree scale to prevent the respondent from getting into a pattern of answering (often called a response set). The intensity question, with its scaled answers and average scores, can supply quantitative information about your respondents’ attitudes toward the subject of your survey. A number of studies have been conducted over the years attempting to determine the limits of a person’s ability to discriminate between words typically found on rating or intensity

© Copy Right: Rai University

69

RESEARCH METHODOLOGY

scales. The results of this research can be of considerable value when trying to decide on the right set of phrases to use in your rating or intensity scale. When selecting phrases for a 4-, 5-, 7-, or 9-point .

Ilustration A survey shows that 60 percent of those returning questionnaires favour a certain policy. If the survey had a 70 percent response rate (a fairly high rate as voluntary surveys go), then

Likert scale, you should choose phrases that are far enough apart from one another to be easily discriminated, while, at the same time, keeping them close enough that you don’t lose potential information.

• the favourable replies are actually only 42 percent of those

You should also try to gauge whether the phrases you are using are commonly understood so that different respondents will interpret the meaning of the phrases in the same way. An obvious example is shown with the following 3 phrases: Strongly Agree, Neutral, Strongly Disagree.

• These are easily discriminated, but the gap between each choice is very large.

• How would a person respond on this three-point scale if they only agreed with the question being asked?

• Since little can be done to estimate the feelings of the nonreturnees, especially in a confidential survey, the only solution is to minimize the number of non-returns. The non returns can be minimized by:

• Use follow-up:These letters are sent to the non-respondents after a period of a couple of weeks asking them again to fill out and return the questionnaire. The content of this letter is similar to that of the cover letter. • Use high-level sponsorship

• People tend to reply to surveys sponsored by organizations

• There is no middle ground between Strongly Agree •

questioned (60 percent of the 70 percent who replied), which is less than 50 percent!

and Neutral. The same thing is true for someone who wants to respond with a mere disagree.

• Your scales must have enough choices to allow respondents to express a reasonable range of attitudes on the topic in question, but there must not be so many choices that most respondents will be unable to consistently discriminate between them. Bias and How to Combat it Like any scientist or experimenter, surveyors must be aware of ways their surveys might become biased and of the available means for combatting bias. The main sources of bias in a questionnaire are:

• a non-representative sample –leading questions and non returns

• question misinterpretation and mistrustful .

they know or respect. If possible, use the letterhead of the sponsor (sponsoring the research) on the cover letter.

• Make your questionnaire attractive, simple to fill out, and easy to read. A professional product usually gets professional results. • Keep the questionnaire as short as possible.

You are asking for a person’s time, so make your request as small as possible

• Use your cover letter to motivate the person to return the questionnaire One form of motivation is the have the letter signed by an individual known to be respected by the target audience for your questionnaire. In addition, make sure the individual will be perceived by the audience as having a vested interest in the information needed.

• Use inducements to encourage a reply.

Now we will discuss these in detail. Surveyors can expose themselves to possible non-representative sample bias in two ways.

These can range from a small amount of money attached to the survey to an enclosed stamped envelope. A promise to report the results to each respondent can be helpful. If you do promise a report, be sure to send it.

1. The first is to actually choose an non-representative sample. This bias can be eliminated by careful choice of the sample .

Proper use of these techniques can lower the non-return rate to acceptable levels.

Non-representative Sample

2. The second way is to have a large number of non- returns.

Misinterpretation

• The non-return bias (also called non-respondent bias) can

The second source of bias is misinterpretations of questions.

affect both the sample survey and the complete survey.

• The bias stems from the fact that the returned questionnaires are not necessarily evenly distributed throughout the sample. The opinions or attitudes expressed by those who returned the survey may or may not represent the attitudes or opinions of those who did not return the survey.

• It is impossible to determine which is true since the nonrespondents remain an unknown quantity.

• Misinterpretation of questions can be limited by clear

instructions, well constructed questions, and through judicious pilot testing of the survey. • Biased questions can also be eliminated by constructing the questions properly and by using a pilot test. • Finally, bias introduced by untruthful answers can be

controlled by internal checks and a good motivational cover letter. • Although bias cannot be eliminated totally, proper

construction of the questionnaire, a well-chosen sample, follow- up letters, and inducements can help control it 70

© Copy Right: Rai University

11.556

Conclusions Warranting Some Confidence

Now we will illustrates the many diverse, and sometimes powerful factors that influence survey findings as a result of using volunteers in a survey.

1. Volunteers tend to be from smaller towns than nonvolunteers, especially when volunteering is for questionnaire studies.

The exclusive use of volunteers in survey research represents another major source of bias to the surveyor - especially the new surveyor.

2. Volunteers tend to be more interested in religion than nonvolunteers, especially when volunteering is for questionnaire studies.

• Although it may not be immediately evident,

3. Volunteers tend to be more altruistic than non- volunteers. 4. Volunteers tend to be more self-disclosing than non volunteers.

• It is nonetheless empirically true that volunteers, as a group,

possess characteristics quite different from those who do not generally volunteer. • Unless the surveyor takes these differences into consideration

before choosing to use an exclusively volunteer sample, the bias introduced into the data may be so great that the surveyor can no longer confidently generalize the survey’s findings to the population at large, which is usually the goal of the survey. Conclusions Warranting Maximum Confidence

1. Volunteers tend to be better educated than non- volunteers, especially when personal contact between investigator and respondent is not required. 2. Volunteers tend to have higher social-class status than non volunteers, especially when social class is defined by respondents’ own status rather than by parental status. 3. Volunteers tend to be more intelligent than non- volunteers when volunteering is for research in general, but not when volunteering is for somewhat less typical types of research such as hypnosis, sensory isolation, sex research, small-group and personality research. 4. Volunteers tend to be higher in need for social approval than non -volunteers.

5. Volunteers tend to be more maladjusted than non volunteers, especially when volunteering is for potentially unusual situations (e.g., drugs, hypnosis, high temperature, or vaguely described experiments) or for medical research employing clinical rather than psychometric definitions of psychopathology. 6. Volunteers tend to be younger than non- volunteers, especially when volunteering is for laboratory research and especially if they are female. Conclusions Warranting Minimum Confidence

1. Volunteers tend to be higher in need for achievement than non-volunteers, especially among American samples. 2. Volunteers are more likely to be married than non volunteers, especially when volunteering is for studies requiring no personal contact between investigator and respondent. 3. Firstborns are more likely than later borns to volunteer, especially when recruitment is personal and when the research requires group interaction and a low level of stress.

Conclusions Warranting Considerable Confidence

4. Volunteers tend to be more anxious than non -volunteers, especially when volunteering is for standard, unstressful tasks and especially if they are college students.

1. Volunteers tend to be more arousal- seeking than nonvolunteers, especially when volunteering is for studies of stress, sensory isolation, and hypnosis.

5. Volunteers tend to be more extraverted than non- volunteers when interaction with others is required by the nature of the research.

2. Volunteers tend to be more unconventional than nonvolunteers, especially when volunteering is for studies of sex behavior.

Borg and Gall (1979) have suggested how surveyors might use this listing to combat the effects of bias in survey research. For example, they suggest that:

3. Females are more likely than males to volunteer for research in general, more likely than males to volunteer for physically and emotionally stressful research (e.g., electric shock, high temperature, sensory deprivation, interviews about sex behavior).

The degree to which these characteristics of volunteer samples affect research results depends on the specific nature of the investigation.

4. Volunteers tend to be less authoritarian than nonvolunteers. 5. Jews are more likely to volunteer than Protestants, and Protestants are more likely to volunteer than Roman Catholics. 6. Volunteers tend to be less conforming than non-volunteers when volunteering is for research in general, but not when subjects are female and the task is relatively “clinical” (e.g., hypnosis, sleep, or counseling research).

11.556

For example, a study of the level of intelligence of successful workers in different occupations would probably yield spuriously high results if volunteer subjects were studied, since volunteers tend to be more intelligent than non-volunteers On the other hand, in a study concerned with the cooperative behavior of adults in work-group situations, the tendency for volunteers to be more intelligent may have no effect on the results, but the tendency for volunteers to be more sociable could have a significant effect. It is apparent that the use of volunteers in research greatly complicates the interpretation of research results and their general ability to the target population, which includes many individuals who would not volunteer.

© Copy Right: Rai University

71

RESEARCH METHODOLOGY

Bias in Volunteer Samples

RESEARCH METHODOLOGY

Appendix – A

•

over 60

8. Other comments:Created by Women’s Enterprise Society of BC 2

Sample Questionnaires ( Market Research)

In each of these cases, the business owners gain valuable information to help them make major decisions about their businesses. Remember that if the results of the survey aren’t very positive, you need to find out WHY. The questionnaire is used as a guide. It doesn’t mean you can’t go into

B. This survey was done by a business man interested in opening public storage buildings.Before he committed any time and money to the project, he sent a questionnaire to consumers within a 15 mile radius of the proposed site.

business.

1. Are you presently renting any public storage space? Yes _____ No_____ If no then go to question 2

A. The first questionnaire is for a select group, the customers of Speedy Photos. The owner conducted the survey during a one week period, reaching both weekday and weekend customers. Speedy Photo Survey In order for us to serve our customers better, we would like to find out what you think of us. Please take a few minutes to answer the following questions while your photographs are being printed. Your honest opinions, comments and suggestions are extremely important to us. Thank you, Speedy Photo 1.Do you live/work in the area (circle one or both) •

Close to home

Close to work 2. Why did you choose Speedy Photo (circle all that apply) • •

Convenient

•

Good service

•

Quality

•

Full-service photography shop

•

Other

Public Storage Questionnaire

If yes, then continue with 1a. 1a.Where are you currently renting storage space (name and address) 1b. How many times a month do you visit your storage space? _______ 1c. Is your storage space heated? Yes ______ No______ 1d. Approximately how much space are you renting? __________square feet 1e. Do you think you’ll need additional space in the future Yes ______ No ______ 1f. Are there any changes or improvements you would like to see in your present storage space arrangement? If yes, what would you like to see? 2. Are you planning on using any public storage space? Yes ______ No ______ 2a.If you are planning to rent public storage space or may rent such space, how far of a distance are you willing to travel to use your space? ______miles 2b.Approximately what size storage space would you need? ______square feet

3. How did you learn about us? (circle one)

2c. How much monthly rent would you be willing to pay? $______per square foot/month

•

newspaper flyer/coupon

•

passing by

•

recommended by someone

Name: Title:

•

other

Address:

•

2d.Would you require heat for your space?

4. How frequently do you have film printed? (please estimate)

Thank you very much for your co-operation

• ______ time per month

Created by Women’s Enterprise Society of BC 3

• ______ other

C. This questionnaire was developed by a woman who was interested in selling southwestern jewelry made by Native Indians.

5. Which aspect of our photography shop do you think needs improvement?

Southwestern Jewelry Questionnaire 6. Our operating hours are from 8 am to 5:30 pm weekdays and Saturdays from 9:30 am to 6 pm. We are closed on Sundays and legal holidays. What changes in our operating hours would be better for you? 7. Your age (circle one)

72

•

under 25

•

26-39

•

40-59

1. Have you ever purchased or received southwestern jewelry? Yes ______ No ______ 2. Have you ever purchased or received southwestern jewelry made by native Indians? Yes ______ No ______ If Yes, what type of jewelry?

© Copy Right: Rai University

11.556

3. Would you be interested in purchasing the above mentioned jewelry made by native Indians? Yes ______ No ______ 4. Do you know where to shop for such jewelry? Yes ______ No ______ 5. When buying jewelry, what do you value the most? On a scale of 1 through 5, list in order according to your preference. One represents your most valued choice. Craftsmanship_____ Cost _____ Uniqueness _____ Other _____

4. Do you believe that our competitors prices are too high? (two-choice)_____ Yes _____ No 5. What price would you be willing to pay for this product/ service? (two-choice) Note: This is an important question to ask because the answer will affect one’s sales revenue projections ____ $10 - 20 ___$20 - 30 6. Which of the following services would you like to see offered? Choose one. (multiple choice) ____ loans program ____ mentoring ____ counselling ____ research ____ other

Examples of Poor Survey Questions

D. The last questionnaire was developed by a woman who wanted to open a fitness center and offer one-on-one training

Do you like this hotel? (This does not give any valuable information, but it could be reworded, “What do you like about this hotel, what don’t you like about this hotel?)

Fitness Center Questionnaire

How do you rate the service received?

1. Do you exercise Yes ______ No ______

____ poor ____ fair ____ good ____ very good ____ excellent (This should have an even number of choices)

If no, please answer questions to Part A If yes, please answer questions to Part B

Which of these services would you be interested in?

A. Please check reasons for not exercising: ____Lack of time ____Lack of motivation ____Cost ____No convenient fitness centers ____medical reasons B. Check the type of exercise you do:

____ loans ____ mentoring ____ business counselling ____ information referral (This question should have an “other” category) What beverages do you drink? ____ Milk ____ coke ____ non-cola drink ____ coffee ____ tea ____ juice

____aerobic ____Nautilus ____Free weights ____running ____Swimming

(This question is too broad. Most of us will have drunk some of these at some time. Is the respondent to check a number of boxes or only one?

____Other, please specify ____________________________ C. Check you age group

Source www.Wesbc01companyELibraryMarketResearchSample

_____under 25 _____ 26-35 _____over 35 D. Where do you normally exercise?

Questionnaires.pdf

_____ at home _____ fitness center E. How far do you live from ( town of proposed center)?

Notes

_____ in town _____ 10-15 miles _____ out of town F. Do you think your town needs a fitness center? Yes _____ No _____ G. Would you be interested in one -on -one training? Yes _____ No _____ H. Please note any other suggestions or comments you might have. Created by Women’s Enterprise Society of BC 4

Examples of Good Survey Questions 1. How do you rate the convenience of our location? (ranking) _____ poor _____ good _____ very good _____ excellent 2. Please rank the following factors in the order of important to you when making a buying decision for this service (1 being most important, 5 being lease important) (multiple choice & ranking)____ price ____ referral ____ location ____ availability ____ guarantee ____ other 3. Are there any other services you would like to see offered? (open-ended)

11.556

© Copy Right: Rai University

73

RESEARCH METHODOLOGY

Necklace____ Ring _____ Bracelet _____ Earnings _____ Other _____