3 - Research Design

Research Design and Methods Research Design FEKD62

Ralf Müller Tomas Blomquist School of Business and Economics Umeå University

Research Design • framework or plan for a study • Used as a guide in collecting and analysing data • A blueprint, followed in completing a study

1

Types of Research Design • Exploratory • Descriptive • Causal Note: • The distinctions are not absolute – a study may serve several purposes • The design of the investigation should stem from the problem Churchill 1999

Relationships of Research Designs

Descriptive Research Exploratory Research Causal Research

Churchill 1999

2

Exploratory Research Purpose of exploratory research: • Formulating a problem for more precise investigation and for developing hypotheses • Establishing priorities for further research • Gathering information about the practical problems of carrying out research on particular conjectural statements • Increasing the analyst’s familiarity with the problem • Clarifying concepts

Churchill 1999

Exploratory Research … is appropriate to any problem about which little is known … is to develop tentative explanations, not demonstrating the viability of a given explanation … is characterized by flexibility in methodology use … is particularly indicated for: • Literature search • Experience surveys • Focus groups • Analysis of selected cases

Churchill 1999

3

Literature Search in Exploratory Research • Easy way to discover hypotheses in the work of others • May involve conceptual literature, trade literature and / or published statistics • Serves for the discovery of ideas and tentative explanations of a phenomenon • Demonstrating the explanation is better left to descriptive and causal research Churchill 1999

Experience Survey in Exploratory Research • Taps the knowledge and experience of those familiar with the subject being investigated • Careful selection of respondents needed, because the nature of the experience survey is on gaining insight into the relationship between variables and not to get an accurate picture of current or best practices. • Provocative ideas and useful insights are more valuable than statistics of the profession Churchill 1999

4

Focus Groups in Exploratory Research • Consist of 8-12 members knowledgeable in the subject under investigation • Goal is to: • Generate hypotheses for quantitative test • Generate information to structure questionnaires • Provide overall background information • Secure impressions on new product concepts • Interact through various possible means, e.g. meeting, email group using Delphi techniques etc. Churchill 1999

Analysis of Selected Cases in Exploratory Research • Involves the intensive study of selected cases of a phenomenon • Done e.g. by examining existing records, observation, unstructured interviews • Focus is on seeking explanations – not testing explanations • Requires the integration of many diverse bits of information into a unified interpretation • Cases that show sharp contrasts or have striking features are most useful (such as best and worst cases, or benchmarking) Churchill 1999

5

Descriptive Research Purpose of descriptive research is to: • describe the characteristics of certain groups / samples / populations • Estimate proportions in specified populations • Make specific predictions

(Churchill 1999)

Descriptive research often follows exploratory research to describe the particularities of the properties identified during in the exploratory step.

Descriptive Research • Done using rigid methods with clear specifications of the who, what, when, where, why and how of the research • Frequently use pilot studies to test the data collection tool and analysis techniques • Data collection often done through structured interviews or questionnaires

(Churchill 1999)

6

Classification of Descriptive Studies True Panel Longitudinal Omnibus Panel

Descriptive Studies Cross-Sectional

Sample Survey

(Churchill 1999)

Cross-Sectional Descriptive Studies • most common and most familiar

• uses a representative sample of elements from a population, often for a sample survey • characteristics of the elements are measured once, i.e. it provides a snapshot of the variables under investigation •Disadvantage of sample surveys are: • High level questions, not very deep, which allow for statistical analysis • Expensive in terms of time and money • Technical skills requirements of the researcher (Churchill 1999)

7

Longitudinal Descriptive Studies •involve panels, i.e. a fixed sample of elements, which are repeatedly measured over time, i.e. it provides a movie of the variables under investigation • panel members are relatively constant over time • True panel: repeatedly measured on the same variable • Omnibus panel: repeatedly measured, but on differing variables •True longitudinal analysis (aka time series analysis) can only be performed on the true panel •Main disadvantage of panels is that they are non representative

(Churchill 1999)

Causal Research A change in one variable brings about a change in another variable Four conditions for causal relationships: • Time sequence – cause occurs before effect • Covariance – two variables are related • Non-spurious association – relationship is not caused by another variable • Theoretical support – logical explanation exists for the cause- effect relationship

8

Causal Research Causality tested through • Lab experiments • All variables are controlled • Influence of external variables minimized • Internal validity maximized • Field tests • Most credible results • Influence of ‘real world’ settings taken into account • External validity maximized

Main Research Designs Exploratory Research Uses

• • • • •

Types

• • • •

Descriptive Research

Formulate problems more precisely Develop hypotheses Establish priorities Eliminate impractical ideas Clarify concepts

•

Literature search Experience survey Focus groups Analysis of selected cases

•

•

•

•

Causal Research

Describe characteristics of certain groups Estimate proportion of people in a population who behave in a certain way Make specific predictions

•

Provide evidence regarding the causal relationship between variables by means of: o Concomitant variation o Time order in which variables occur o Elimination of other possible explanations

Longitudinal study o True panel o Omnibus panel Sample survey

• •

Laboratory experiment Field experiment

9

Choosing the Right Design Design type depends on research question: If research question asks for: • discovery or clarification: use exploratory design • description of quantities, amounts, or extent of variable relations: use descriptive design • Statements on cause and effect: use causal design Other factors that influence the design decision: • Accessibly and quality of required data • Ethical questions • Time, cost, and researchers experience

Key Choices of Research Design Researcher is independent

vs

Researcher is involved

Large samples

vs

Small numbers

Testing theories

vs

Generating theories

Experimental design

vs

Fieldwork methods

Verification

vs

Falsification

Easterby-Smith, Thorpe, & Lowe, 1991

10

Choosing the Research Approach

• Deductive • Testing theory • Inductive • Developing theory

11

Deductive Research • Based on methods from natural science • Existing theory used for hypotheses development • Data collection through surveys, observation, Questionnaires, interviews etc. • Analysis often done using statistical methods, e.g. multivariate data analysis techniques • Theory development through confirmed or rejected hypotheses • Replicable research

Inductive Research • Understanding of a new / unknown phenomena • Data collection through interviews, observation, diary methods • Analysis approach holistic, inductive, pattern recognition / development • Theory follows data • Difficult to replicate

12

Research Strategies • Experiment • Survey • Case study • Grounded theory • Ethnography • Action research

Experiment • Tests a hypothesis • Controls external variables e.g. through selection of individuals from known populations • Tests impact of different levels of Xvariable(s) (cause) on Y-variable (effect) • Yields highest levels of precision in measurement

13

Survey • Mostly used in deductive research, but has potential for inductive research as well • Often done through questionnaires with standardised data which can be analysed using quantitative methods • Time consuming because of need for questionnaire design, pilot test and analysis • Can also be used for in-depth structured interviews

Case Study Research … is a comprehensive research strategy which attempts to examine a contemporary phenomenon in its real life context (Yin). Uses a variety of data collection methods to test existing theory or develop new theory. Therefore it can be based on: • Expert knowledge of previous research and a priori hypothesis development which precede data collection • The intent to create scientific generalisations through the deep understanding of issues, contexts and interpretations

14

Grounded Theory An inductive, theory-discovery methodology that allows the researcher to develop a theoretical account of the general features of a topic while simultaneously grounding the account in empirical observations or evidence (Glaser and Strauss 1967) • Often done using interviews or observations to identify patterns for development of a grounded theory. This theory can be tested for generalisation using e.g. a large scale survey.

Ethnography • A form of participant observation • Allows to interpret the behaviour of individuals in their real world context • Inductive research approach • Time consuming

15

Action Research … is to describe, understand, explain and change simultaneously • Intended for management of change projects, executed by a combination of researcher and practitioner • Researcher is part of the organization / project • Is a cyclical approach for the improvement of organizational changes and the associated actions over time

16

Dilemmatics

Runkel & McGrath 1972

Dilemmatics

Runkel & McGrath 1972

17

Example of a research design

Research Questions (Example) Q1:How do organisational structure, relational norms and project risk impact preferences for communication frequency, contents and media of project sponsors and managers in their formal communication during implementation of IT projects? Q2: How effective is the formal communication between project sponsors and project managers during IT project implementation?

18

Research Hypotheses (Example) H1: There will be a significant difference between communication preferences of project sponsors from buyer firms and those of project managers from IT seller firms in terms of preferred communication frequency, contents and media. H2: There will be a positive relationship between the extent of organic organisation structure and communication frequency. The more organic the organisation, the higher the frequency in communication. H3: There will be a negative relationship between the level of organic organisation structure and the use of lean media. Bureaucratic organisation structures will favour written reports, while organic structures favour more interactive media like verbal communication or face-to-face meetings.

Research Hypotheses (Example) H4: There will be a negative relationship between the extent of organic organisation structure and the use of quantitative measures, like quality metrics and earned value numbers. Bureaucratic structures will favour quantitative data. H5: There will be a positive relationship between project goal equivocality and media richness; rich media (e.g. face-to-face meetings) will be selected for communications in projects with unclear goals, and lean media (i.e. written communication) in projects with clearly defined goals. H6: There will be a positive relationship between clearness of methodology and the use of written media for communications.

19

Example of Design and Process • underlying philosophical stance is that of post-positivism • starting point is a set of hypotheses derived from the literature review • the degree of researcher involvement in the research process is minimized through use of survey techniques and structured interviews • knowledge is gained though use of quantitative and qualitative methods, i.e. surveys and interviews, to triangulate and validate research findings and provide them to the scientific community for test and public scrutiny.

Example of Design and Process Preparation

Execution

Conclusion

Literature Review, establishing Hypotheses and Questions

Conclusions

Research Design

Interview Analysis and Results Triangulation

Focus Groups

Interviews

Survey Pretest

Survey Analysis

Worldwide Survey

time

20