Survey New Project

SURVEY RESEARCH: A survey is part of a survey respondent. effectively to gain research is basically individuals.It has

social interaction between the researcher and the Therefore, the researcher must communicate participation and thoughtful responses.Survey a tool for obtaining information from a sample of

 Many purposes  Many techniques  Many different people studied  Often not done correctly

Interaction Divided into: 1. Contact 2. Response 3. Follow-up 4. Questions Facing the Researcher Which method to use? 5. How to draw a sample? 6. How to write questions? 7. How to design the questionnaire? 8. How to code, analyze, and report the results? Threats to Survey Research 1. Too many surveys in general 2. Sales calls masquerading as surveys 3. Inadequate disclosure by pollsters 4. Respondent deception Unscientific Methods 1. Casual or selective observation 2. Over-generalization from few observations

3. Purposive samples 4. Filtering observations through one's ideas 5. Fax polls 6. Newspaper polls 7. Mall intercept surveys Key Issues in Survey Research 1. Writing Good Questions 2. The "Art" of Asking Questions 3. Representativeness 4. Standardized Measurement 5. Overview: Survey ProcessDefine the problem 6. Define the population 7. Define the type of data that are needed 8. Determine method (mail, phone, Internet, other) 9. Determine resource needs 10.Background Research 11.Writing the Questions 12.Design the Questionnaire 13.Pretest the Questionnaire 14.Sampling 15.Data Collection (“Fielding Period”) 16.Data Entry (newer alternatives) 17.Data Analysis 18.Report Writing 19.Presenting Results

Steps to Survey ResearchDefine problem and hypothesis – recall scientific method  Planning of survey  What do you hope to answer?  What type of survey?  Writing of survey questions  Layout and response choices  Refinement of survey  Who to sample?  Analysis of results  Reporting results  Defining problem  Perhaps the most important part  Is a survey the appropriate research tool?  Advantages:  often quick and easy  Can assess many different variables at once  Surveys can be either quantitative or qualitative (or both)  Disadvantages  It is descriptive research – not causal  Surveys are self-report.  Attitude-behavior consistency – people often behave differently than they think.  E.g., helping behavior

 People often do not know why they act the way they do  Nisbett & Wilson (1976)

 Self-serving bias  Planning of Survey: Types of surveys Questionnaires – written questions  Can be mailed or administered in person  Convenient and cheap  Subjects can’t ask for clarification  Generally, very poor response rates.  Better for sensitive questions  Telephone surveys / interviews  Often are computer assisted - CATI  Allows for clarification  Inexpensive  Very time consuming  Difficult for some subjects to keep something in memory if not in front of them.  People are annoyed by phone surveys  Potential for experimenter bias.  Face-to-face interviews – oral questions  Plethora of data can be gathered.  Provides for flexibility in gathering responses  Requires large amounts of time  Potential for experimenter bias  Focus Groups  Advantages  Exploratory; gets people to talk about their attitudes and perceptions

 In-depth information  Can use various resources  Good for qualitative research  Disadvantages  Not efficient  Only get data from small sample  Must have a good moderator  Sensitive topics are difficult  Layout of survey questionsIf your questions are bad, your whole research project is bad:  Garbage in Garbage out  Most important rule: Keep it simple  Use plain, simple language  1. Avoid jargon (including technical terms), slang, and abbreviations  Do you like your job?  Do you experience satisfaction in accomplishing the daily activities involved with your employment?  2. Avoid ambiguity, confusion, and vagueness  E.g., what is your income? 

3. avoid emotional language and prestige bias  Al Gore argues that global warming is a big problem on our planet. Do you agree?

 4. Avoid double-barreled questions– asking two questions but expecting only one response.  Example: Do you think students should be required to write research papers and take exams in every class?  5. Avoid leading questions

 Ex. Don’t you think you’re boss is a great guy  Survey questions should never convey your personal opinions.  You’re not trying to influence them.  Ethics in surveys  Push polls – surveys designed to influence personal opinion, not simply to assess it.  6. Avoid asking questions that are beyond respondents’ capabilities  Why do you think Rwandan tanks fire on the Congo?  Depends on your audience.  7. Avoid false premises – do not begin with a premise that the respondents may not agree with.  Ex. Because the war in Iraq has been a disaster, should we pull out our troops?  8. Avoid asking about future intentions  What will you do in the future?  Very poor predictor of behavior, especially hypothetical situations.  Attitude-behavior inconsistency

 Problems with Survey Research  Sampling/Selection Issues.  Response Issues.  Response Scales.  Sampling/Selection Issues   

Random

sampling. (What is it?) Samples:



Must be representative of the population in question. (Why?) So that information (results) can be generalized to entire population (External validity).



Random samples are expensive, timeconsuming.





 





(Why?) Because each individual in

 population has an unbiased chance of  being in the sample.  Alternative method: Stratified sampling.  Divide population into smaller units.  Use random sampling on the smaller  units.

 Response Issues  May be problems with accuracy of  response due to:  Response Style/Response Bias:  Response Acquiescence.  Response Deviation.  Social Desirability.  Volunteer Participation.

 Response (Measurement) Scales        

Likert

(type). Forced Choice/Multiple Choice. Ranking. Sentence Completion. Fill in blanks. Verbal Descriptions. Written Descriptions. Verbal Protocols.

Threat to survey research

Excessive Interviewing With over 300,000 interviews conducted each year, in a country with only 2,600,000 citizens aged 15 or older, it is likely that a high proportion of the New Zealand population has been interviewed at some time in the recent past, and inevitable that some people will be interviewed more than once. Recent studies support this conjecture. In a 1987 survey of Hamilton residents, Phee and Taylor (1988) found that 70% of their sample had been interviewed in the past 12 months. A nationwide MRL survey in 1990 found that 64% of respondents had been surveyed before, 46% within the last 12 months. Neither of these studies reports the number of times respondents had been interviewed. In 1989, Brennan found that 60% of a national random sample had been surveyed in the past 12 months; 40% had been surveyed at least twice, while 10% claimed to have been surveyed five times or more (Brennan 1991). Sugging 'Sugging', or selling under the guise of research, is expressly forbidden by the Code of Practice of the Market Research Society of New Zealand, and by the Code of Standards of the New Zealand Direct Marketing Association. Despite this, sugging is prevalent, and of concern to both the MRS and NZDMA because it brings both industries into disrepute and

may contribute to respondent resistance (Opinions 1990). Brennan (1991) found that respondent attitudes towards survey research were indeed more negative among those who had been 'sugged'. According to an MRL study (Opinions, 1990), 8% of their national sample claimed to have been sugged. The main culprits were sellers of encyclopedias or other educational books, time-share apartments, vacuum cleaners, insurance and superannuation. Sugging was much more prevalent in Brennan's (1991) study; over 30% of respondents claimed to have been sugged, and over 60 products associated with sugging were identified. Again, the main culprits were time-share, encyclopedias, and insurance. Each of the above studies was conducted in a different year, using different survey methods, and a different sampling frame. It is therefore impossible to judge whether the level of exposure to surveys, or sugging, is increasing or not. To address this issue, a standard set of questions has been included in the Palmerston North Household Omnibus Survey, conducted annually by the Marketing Department, Massey University. This note reports the findings of that survey for 1990 and 1991. Different types of surveys •

attitudinal

•

behavioral

•

information gathering (economic surveys)

•

valuation

•

Advantages to surveysGathering Information –

Efficient for collecting large amount of information

–

Flexible to collect wide range of information •

•

•

attitudes, values, beliefs, and past behaviors.

Questions –

Standardized - relatively free from several types of errors

–

Researcher can ask more complex questions

–

Diminished response effect •

willing to divulge more info w/o face-to-face contact

•

less likely to try to impress interviewer)

Administration –

Relatively easy to administer

• – •

Computer-based – automatic data entry

Economy in data collection

Analysis –

Statistical techniques to determine validity, reliability, and statistical significance

http://en.wikipedia.org/wiki/Statistical_survey •

•

•

Disadvantages to surveysGathering Information –

Questions must be general enough for all respondents

–

Non-literate or illiterate populations •

only English-speaking populations

•

hard-to-reach populations – “vulnerable populations”

Questions –

Can’t determine subjects’ motivation, memory, and ability to respond

–

Not appropriate for studying complex social phenomena

–

Subjects may not answer honestly

–

No control over participant interpretation

Administration –

Low response rates

–

Respondents usually self-selected •

•

Uncertainty about who actually filled out the questionnaire

Analysis –

Statistical techniques to determine validity, reliability, and statistical significance

•

Survey research – componentsSurvey design

•

Sampling

•

Implementation

•

Analysis and reporting

Survey designTechnical background Preliminary survey design •

•

introduction –

tell respondents pFocus-groups

–

Also important as qualitative research tool

–

Ex: focus-group 9/10/2001 – Poughkeepsie, NY

–

One-on-one cognitive interviews

–

verbal protocols

–

think alouds

–

retrospective reports

–

Pre-testing survey

–

survey instrument

–

implementation method

–

analysis planning

–

Parts of a survey

–

Introduction

–

Filtering questions

–

Content questions

–

Socio-demographic questions

–

Debriefing questions

–

Other considerations

–

Instructions

–

Formatting

–

urpose of the survey

providing information

•

how to ask the correct questions

•

response categories

•

socio-demographic information

•

de-briefing questions

•

SamplingPopulation, Units, Subjects and Samples

•

–

Population: entire group of people about which information wanted.

–

Units/subjects: Individual members of the population are called units

–

Sample: part of population examined

Error –

Sampling: not surveying all elements of population

–

Coverage: not allowing all members of the survey population to have an equal or nonzero chance of being sampled

http://www.gseis.ucla.edu/courses/ed230a2/notes2/sampling.html •

Types of Samples –

Voluntary response sample

–

Convenience (accidental, haphazard) sample

–

Simple random sample

–

Stratified random sample

–

Cluster random sample

–

Multistage sample design

Implementationmethods for survey implementation –

telephone

–

in-person

–

internet •

Knowledge Networks type of access

•

ResearchExec

–

mixed mode – e.g., telephone/mail

–

mail •

the mail survey “package” – Dillman Total Design Method (TDM) –

pre-contact

–

cover letter

–

survey instrument

–

return envelope

–

incentive

–

reminder post-card

–

follow-up

•

Analysis and reportingQA/QC

•

response rate – AAPOR

•

socio-demographics of pop / sample / respondents

•

analysis methods

•

–

content analysis

–

factor analysis

–

latent class variable analysis

quantitative analysis –

summary data

–

basic statistical analysis

–

econometric modeling

– Characterstics of Good survey research Quatitative Self monitoring

Systematic Imparial Contemprory Theory basd Representative Replicable

Types of survey Cross sectional Longitudinal Trend Panel Time cohort