Lecture Notes Research Chapter 10-16

46

Dr. William Kritsonis Educational Research Lecture Notes Chapter 10 – Quasi-Experimental and Single-Case Designs 1. A researcher does a study examining the effects of a preschool program. The researcher uses a nonequivalent comparison group design. The researcher finds that the cognitive growth of the experimental group is greater than that of the control group. Unfortunately, the researcher later finds that in general children who live in the area where the experimental group is located tend to grow faster cognitively than children who were from the area where the control group is located. When the researcher this problem, he/she discovered what threat to the internal validity of the study? Answer: Selection-maturation effect 2. For a treatment to be deemed effective when used in the context of an a-B-A single case design, the following has to occur: a) Behavior should change as the treatment is implemented b) Behavior should return to baseline levels when the treatment is removed.

47 3.

In changing-criterion design, changes in criterion are best done when the previous criterion is met and the behavior is stabilized.

4.

The non-equivalent comparison group design is a quasiexperimental design, in which, for reasons of practicality, we cannot insure that the control and experimental groups are equivalent to each other when the experimental begins. The major interpretational difficulty imposed by this design is being sure that any differences between groups at the end of the experiment are due to the independent variable’s influence and not due to preexisting group differences.

5.

A treatment effect is demonstrated in the regression discontinuity design by a discontinuity in the regression line.

6.

The history effect is a primary threat to the interrupted time-series design.

7.

The nonequivalent comparison-group design consists of an experimental and a control group but participants are not randomly assigned to the groups.

8.

A form of the nonequivalent comparison-group design is recommended when it is not possible to randomly assign participants to groups.

48

9. A threat to internal validity in the nonequivalent comparison-group design include the following: a) Selection-maturation effect b) Selection-history effect c) Selection-regression effect 10. Selection-history

effect is a threat to internal validity in the nonequivalent comparison-group design.

11. How

many variables should be changed at a time when conducting a single-case design? Answer: 1

12. In

quasi-experimental research designs, causal interpretations can be made only when rival explanations have been shown to be implausible.

13. In

single-case research, “baseline” refers to the rate of response established prior to the experimental intervention.

14. Changing-criterion

design can be used when the goal is to create a step-by-step increase (or decrease) in the amount, accuracy, or frequency of some behavior over a period of time.

15. Selection-maturation

effect occurs in a comparison group design when one of the two groups of participants grows or naturally develops faster than the other group.

49 16. An

observation of a dependent variable response prior to any attempt to change this response is known as the baseline.

17. The

most frequently used quasi-experimental design is the nonequivalent comparison-group.

18. A

baseline is used as the standard against which change induced by the treatment is assessed.

A baseline is the occurrence of a response in its freely occurring or natural state. A baseline is first obtained prior to the administration of a treatment. 19. A

single-case experimental design in which the response to a treatment is compared to baseline occurring before and after the treatment is call an A-B-A design.

20. In a single-case design, the researcher hopes that the behavior of the participants prior to the administration of a treatment condition is not highly variable. 21. The A-B-A design rules out history by demonstrating that the dependent variable response revert back to the baseline when the treatment is withdrawn. a)

Nonequivalent comparison-group design would use analysis of covariance during data analysis.

50 b)

The interrupted time-series design can also be viewed as an A-B design.

c) It is important to change only one variable at a time in single case designs because of the following reasons: d) Changing one variable allows isolation of the cause of the change e) Changing more than one variable at a time confounds those independent variables. 22.

What is the difference between A-B-A design and A-B-A-B design? a.

23.

Answer: A-B-A-B allows the reintroduction of the treatment condition during the last phase.

The following are phases in the A-B-A design: a. Baseline measurement b. Introduction of treatment c. Removal of treatment

24.

Researchers can attempt to eliminate the threat of bias from the selection-maturation effect in the nonequivalent comparison-group design by matching experimental and control participants on important variables.

25.

Group-comparison designs and single-case designs are both equally effective in conducting educational research.

51

Dr. William Kritsonis Educational Research Lecture Notes Chapter 11 – Nonexperimental Quantitative Research

1.

An example of a spurious relationship is when the number of police officers and the number of crimes are positively related.

2.

A researcher studies the relation between early reading and later school achievement. The research decides that a potentially extraneous variable in the relationship is IQ. In developing groups for the study, the researcher pairs each child who was an early reader with a child of the same IQ level who was not an early reader. The control technique the researcher used was matching.

3.

Partial correlation analysis involves examining the relationship between two or more variables controlling for additional variables statistically.

52

4.

The directors of a graduate program in educational research wish to see what types of jobs their graduates take after they finish their program. They randomly sample students form the program and have them fill out questionnaires with items asking about types of jobs they have had. They also are asked to describe the roles they play in their current positions. This project is best described as having a descriptive objective.

5.

When research is done to test hypotheses and theories about how and why phenomena operate as they do, then the primary purpose of such research is explanatory.

6.

The variable the research matches to eliminate it as an alternative explanation is called a matching variable.

7.

Longitudinal research designs include panels and trends.

8.

The positive correlation between teachers’ salaries and the price of liquor is spurious and due to a thirdvariable.

9.

Partial correlation and analysis of covariance are considered a special case of the general linear model.

10. Retrospective

research starts with the dependent variable and moves backwards.

53 11. The

method of working multiple hypotheses refers to a technique for identifying rival explanations.

12. GLM

refers to the General Linear Model.

13. The

post hoc fallacy is making the argument that because A preceded B, A must have caused B.

14. The following are steps in conducting nonexperimental research: a) Determine research problem and hypotheses b) Analyze data c) Interpret results 15. If

a research finding is statistically significant then the observed result is probably not due to chance.

16. The following are necessary conditions for causation: a) The relationship condition b) The temporal antecedence condition c) The lack of alternative explanation condition 17. The following independent variables cannot be manipulated in a research study: a) Gender b) Ethnicity c) Intelligence and other traits

54 18. Causal

modeling is a form of explanatory research in which the researcher develops a theoretical model and empirically tests the model to determine how well the model fits the data.

19. Non-experimental

research in which the primary independent variable of interest is categorical is sometimes called causal-comparative research.

20. The

experimental approach is the strongest for establishing that a relationship is causal.

21. Is causal-comparative stronger than correlation for establishing that a relationship is causal? Answer: One CANNOT say without additional information (i.e., it could be either depending on how well the researcher established the three necessary conditions for cause and effect. 22. Statistical

control is the most commonly used technique for controlling for extraneous variables in nonexperimental research.

23. It is best to use the method of working multiple hypotheses when planning for a research study. 24. Matching

can be done when you independent variable is categorical or quantitative.

55 25. If

a correlation coefficient is .96, we would probably be able to say that the relationship is strong and statistically significant.

26. What happens in a completely spurious relationship once the researcher controls for a confounding third variable? Answer: The relationship between the original variables will get weaker or, if the original relationship if fully spurious, it will disappear (i.e., the original relationship will become zero as measured by a correlation coefficient. 27. The

most problematic condition in non-experimental research is Condition 3: The relationship between variable A and variable B must not be due to some confounding extraneous variable.

28. A

cross-sectional study is NOT a form of longitudinal research.

Note: Panel and trend studies are a form of longitudinal research. 29. Observing a relationship between two variables is NOT sufficient grounds for concluding that the relationship is a causal relationship.

56 30. Panel

study is a type of longitudinal research that studies the same individuals over an extended period of time.

31. Explanatory

research tests hypotheses and theories in order to explain how and why a phenomenon operates as it does.

32. The

Pearson product moment correlation measures the degree of linear relationship present between two variables.

57

Dr. William Kritsonis Educational Research Lecture Notes Chapter 12 – Qualitative Research 1.

Unique case orientation is a characteristic of qualitative research.

2.

Phenomenology has its disciplinary origins in philosophy.

3.

The primary data analysis approach in ethnography is holistic description and search for cultural themes.

4.

Bracketing is the term used to describe suspending preconceptions and learned feelings about a phenomenon.

5.

A researcher studies how students who flunk out of high school experienced high school. The research found that it was common for such students to report that they felt like they had little control of their destiny. The researcher’s report that this lack of control was an invariant part of the students’ experiences suggests that lack of control is an essence of the “flunking out” experience.

6.

Shared beliefs are the specific cultural conventions or statements that people who share a culture hold to be true or false.

58

7.

Norms are the written and unwritten rules that specify appropriate group behavior.

8. The following are advantages for studying multiple cases: a) Multiple cases can be compared for similarities and differences b) Multiple cases can more effectively test a theory than a single case c) Generalizations about population are usually better when based on multiple cases. 9.

Shared values are the standards of a culture about what is good or bade or desirable or undesirable.

10. Phenomenology

is the study of human consciousness and individuals’ experience of some phenomenon.

11. The following are characteristics of qualitative research: a) Design flexibility b) Inductive analysis c) Context sensitivity 12. Grounded

theory is a general methodology for developing theory that is based on data systematically gathered and analyzed.

13. Selective

coding is the final stage in grounded theory data analysis.

59

14. Naturalistic

inquiry is a major characteristic of qualitative research that refers to studying real world situations as they unfold naturally.

15. As

a qualitative research approach, phenomenology’s primary goal is to gain access to individuals’ inner worlds of experience.

16. Ethnography

is the type of qualitative research that describes the culture of a group of people.

17. The

grounded theorist is finished analyzing data when theoretical saturation occurs.

18. An

example of instrumental case study and/or collective case study is when the researcher focuses primary interest on understanding something more general than the particular case.

19. Ethnocentrism

is judging people from a different culture according to the standards of your own culture.

20. The following are characteristics of qualitative research: a) Design flexibility b) Dynamic systems c) Naturalistic inquiry Note: Deductive design is not a characteristic of qualitative research.

60

21. Collective

case study involves the studying of multiple cases in one research study.

22. The following apply to qualitative research: a) Data are often words and pictures b) Uses the inductive scientific method c) Involves direct and personal contact with participants Note: Qualitative research does not end with a statistical report. 23. The difference between ethnographic research and other types of qualitative research is that ethnographers specifically use the concept of “culture” to help understand the results. 24. Emic

perspective is a term that refers to the insider’s perspective.

25. In

data analysis of the grounded theory approach, axial coding is the step that focuses on the main idea, developing the story line, and finalizing the theory.

26. The following are 4 major approaches to qualitative research: a) Ethnography b) Phenomenology c) Case study d) Grounded theory

61

27. In

“phenomenology,” a well written report will be highly descriptive of the participants’ experiences and will often elicit in the reader a feeling that they feel as though they are experiencing the phenomenon themselves. This experience is called a vicarious experience.

28. A

researcher wants to study a Native American group in New Mexico for a six month period to learn all they can about them so he/she can write a book about that particular tribe. The researcher wants the book to be accurate and authentic as well as informative and inspiring. The researcher will likely be conducting ethnographic research.

29. The

emic perspective refers to an internal view of reality.

30. Ethnography

is used to describe cultural scenes or the cultural characteristics of a group of people.

31. Terms

such as “geeks,” “book worms,” “preps,” are known as Emic terms.

32. When

a researcher identifies so completely with the group being studied that he or she can no longer remain objective we have what is called going native.

33. The

etic perspective is external.

62 34. The

emic perspective is the native or insider’s perspectrive; the etic perspective is an external, social scientific perspective.

63

Dr. William Kritsonis Educational Research Lecture Notes Chapter 13 – Historical Research 1.

A researcher was interested in studying why the “new math” of the 1960s failed. The researcher interviews several teachers who used the new math during the 1960s. These teachers are considered primary sources.

2.

External criticism is the process of dealing with concerns over the authenticity of a source.

3. A researcher studying the history of medical education finds a manuscript that purports to be from the 14th century. Before the researcher uses the source, he/she goes to three other experts who help identify whether the manuscript is authentic or not. Authentification of the object is referred to as external criticism. 4. A historical researcher studying the implementation of the “new math? During the 1960s uses as a source a text written on the subject by a critic who was a mathematics teacher during that time period. As the researcher examines the document, he/she discovers that the data the individual based their conclusions on was falsified. Hence, the conclusions drawn were erroneous. The researcher’s analysis of the document to check its accuracy is referred to as internal criticism.

64

1.

A researcher is interested in studying approaches to teaching writing in schools during the 1800s. The researcher discovers a grammar book, but there is no author or copyright date in the book. The researcher examines the typeface in the book as well as the writing style. After investigating further, the researcher finds a reference to the book from a teacher’s diary from the 1800s. The diary also mentions an author’s name. After further searching the researcher is able to identify the author of the book. The researcher was engaged in the sourcing process.

2. Historical research is conducted for the following reasons: a) To identify the relationship that the past has to the present. b) To evaluate and record accomplishments of individuals or entities c) To enhance understanding of the culture in which we live d) To uncover the unknown 3. Historical research is interpretative.

65

4. The following are steps in the process of conducting historical research: a) Preparing a report or narrative exposition b) Identifying a research topic and formulation of the research problem or question c) Data synthesis d) Data collection and/or literature review 5. Oral histories can be based on the following: a) Interviews with people b) Stories and tales c) Songs 6. In historical research, a primary source consists of the following: a) First hand accounts by witnesses to events b) Can consist of sources that include original maps, diaries, transcripts of minutes of a meeting, and photographs 7.

In historical research secondary sources are generally considered less useful than primary sources.

66

8. In evaluating historical research sources, external criticism… a) Can involve the use of carbon dating and handwriting experts b) Helps determine the validity, trustworthiness or authenticity of a source c) Can involve use of historical linguists’ knowledgeable with the writing style of the period 9.

Contextualization refers to the process of identifying when and where an event took place.

10. Internal

criticism is the process of determining the reliability or accuracy of the information contained in the sources collected.

11. Presentism

in historical sources is the assumption that the present-day connotations of terms also existed in the past.

12. “Comparing

documents to each other to determine whether they provide the same information or reach the same conclusion” is known as corroboration.

13. The

“identification of when and where an event took place” is known as contextualization.

14. Three

heuristics for evaluating documents are corroboration, sourcing, and contextualization.

67

15. In historical research, data synthesis usually includes the following: a) Defining and interpreting key words, phrases and terms b) Chronologically ordering eventgs c) Differentiating between how people should behave and how they did behave d) Maintaining a distinction between intent and consequences Note: In historical research data synthesis usually does not include inferring causation based on simple correlation.

68

Dr. William Kritsonis Educational Research Lecture Notes Chapter 14 – Mixed Method and Mix Model Research 1.

Mixed research occurs when quantitative and qualitative techniques are mixed in a single study.

2.

Mixed model research occurs when quantitative and qualitative approaches are mixed within or across the stages of the research process.

3.

Mixed method research occurs when a quantitative phase and qualitative phase are included in the overall research study.

4.

The use of a qualitative research objective, collection of qualitative data, and performance of quantitative analysis is an example of across-stage mixed model research.

5.

The use of a quantitative research objective, collection of quantitative data, and the use of quantitative data analysis is known as mono-method research.

6. Mixed researchers typically use the pragmatist philosophy. 7.

The pragmatist philosophy is most closely associated with the compatibility thesis.

69 8.

A weakness of quantitative research is that the researcher’s categories that are used might not reflect local constituencies.

9.

A weakness of qualitative research is that the results are more easily influenced by the researcher’s personal idiosyncrasies.

10. A

weakness of mixed research is that it is more time consuming and expensive.

11. If

a researcher develops a questionnaire that includes closed-ended and open-ended items then they have conducted within-stage mixed model research.

12. A

mixed method notation is as follows: “QUAL quan”

13. A

mixed design notation is as follows: QUAL + QUAN

This means qualitative and quantitative are given equal status and they are done concurrently. 14. What would this mixed method design be called: Qual QUAN This means a dominant-status sequential design. 15. What would this mixed method design be called: QUAL QUAN

70

This means equal-status sequential design. 16. What would this mixed method design be called: Qual + QUAN This means dominant-status concurrent design. 17. What would this mixed method design be called: QUAL + QUAN This means equal-status concurrent design. 18. The following can serve as a purpose or rationale for mixed research designs: a) Triangulation b) Complementarity c) Development d) Initiation e) Expansion 19. Complementarity

is a mixed research method that seeks elaboration, enhancement, illustration, and clarification of the results from one method with the results from the other method.

71

20. The following two questions must be answered in order to select one of the specific mixed method designs: a) Is the qualitative or quantitative paradigm going to be given priority, or will they be given equal status in the study? b) Should the qualitative and quantitative components be carried out concurrently or sequentially? 21. A researcher wants to understand why people are willing to handle snakes as part of their church activities in several rural churches in Tennessee, Alabama, and Georgia. The researcher makes qualitative observations at several churches but also hands out fairly lengthy instruments that quantitatively measure several constructs (IQ, religiousity, self-esteem, and a political attitudes construct). The researcher does some mixing during the discussion of his report about how the qualitative and quantitative insights are related. Which design is this? Equal-status concurrent design.

72

Dr. William Kritsonis Educational Research Lecture Notes Chapter 15 – Descriptive Statistics 1. What is the median of the following set of scores? 18,6,12,10,14 Answer: 12 2.

Approximately 68% of scores fall within one standard deviation of the mean in a normal distribution.

3.

The denominator (bottom) of the z-score formula is the standard deviation.

4. The predicted score for someone with 10 years of education and an aptitude test of 5 is 80. 5.

The standard deviation is the square root of the variance; a measure of variability; an approximate indicator of how numbers vary from the mean.

6. Hypothesis testing and estimation are both types of descriptive statistics. 7. A set of data organized in a participants(rows)-byvariables (columns) format is known as a “data set.” 8.

A line graph uses vertical bars to represent data.

73

9.

The goal of descriptive statistics is to focus on summarizing and explaining a specific set of data.

10. The

mode is the most frequently occurring number in a set of values.

11. As

a general rule, the mean is the best measure of central tendency because it is more precise.

12. Descriptive

statistics focuses on describing or explaining data. Going beyond immediate data and making inferences is a function of inferential statistics.

In other words, focusing on describing or explaining data versus going beyond immediate data and making inferences is the difference between and descriptive and inferential statistics. 13. Why are variance and standard deviation the most popular measures of variability? Answer: The variance and standard deviation are the most stable and are foundations for more advanced statistical analysis. 14. Regression

analysis is the set of procedures used to explain or predict the values of a dependent variable based on the values of one or more independent variables.

74 15. The

mean is the value calculated when we want the arithmetic average.

16. Scatterplots

are used when we want to visually examine the relationship between two quantitative variables.

17. The

median is often the preferred measure of central tendency if the data are severely skewed.

18. H

– L is the formula for range.

19. A

raw score that has been transformed into standard deviation units is called a z score.

20. The following are measures of variability: a) Variance b) Standard deviation c) Range Note: The median is not a measure of variability. 21. The following are common measures of central tendency: a) Mode b) Median c) Mean Note: The Range is not a common measure of central tendency. 22. What

is the median of this set of numbers:

75

4,6,7,9,2000000? Answer: 7 Note: Remember, the median is the middle score. 23. What is the mean of this set of numbers: 4,6,7,9,2000000? Answer: 4000,005.2 Note: Remember, the mean is the arithmetic average. 24. Percentile

rank is interpreted as the percentage of scores in a reference group that falls below a particular raw score.

25. The

median is the middle point.

26. The

median is a measure of central tendency that takes into account the magnitude of scores.

27. If

a test was generally very easy, except for a few students who had very low scores, then the distribution of scores would be negatively skewed.

28. One 29. The

dependent variable is used in multiple regression.

median represents the fiftieth percentile, or the middle point in a set of numbers arranged in order of magnitude.

76

30. If

a distribution is skewed to the left, then it is negatively skewed.

31. In

a grouped frequency distribution, the intervals should be mutually exclusive and exhaustive.

32. When a set of numbers is heterogeneous, we cannot place more trust in the measure of central tendency as representing the typical person or unit. 33. Non-overlapping

categories or intervals are known as mutually exclusive.

34. To

interpret the relationship between two categorical variables, a contingency table should be constructed with either column or row percentages, and if the percentages are calculated down the columns, then comparisons should be made across the rows. If the percentages are calculated across the rows, comparisons should be made down the columns.

77

78

Dr. William Kritsonis Educational Research Lecture Notes Chapter 16 – Inferential Statistics 1. The following symbol represents a population parameter:

σ 2. If we drew all possible samples from some population, calculated the mean for each of the samples, and constructed a line graph (showing the shape of the distribution) based on all of those means, what would we have? Answer: A sampling distribution 3. A calculation at the 95% confidence interval indicates the following: a) The process we used will capture the true parameter 95% of the time in the long run b) We can be “95% confident” that our interval will include the population parameter c) We can be “5% confident” that our interval will not include the population parameter.

79

4. What would happen (other things equal) to a confidence interval if we calculated a 99 percent confidence interval rather than a 95 percent confidence interval? Answer: The confidence interval will become wider. The following statement sounds like a null hypothesis: There is no difference between male and female incomes in the population. 5.

6.

The analysis of variance is a statistical test that is used to compare two or more group means.

7.

The standard deviation of a sampling distribution is called the standard error.

8. Hypothesis testing and estimation are the two key branches of the field of inferential statistics. 9.

A sample is a subset of a population.

10. A

statistic is a numerical characteristic of a sample and a parameter is a numerical characteristic of a population.

11. A sampling distribution is based on the following: a) Sample means b) Sample correlations c) Sample proportions 12. As

a general rule, researchers tend to use 95 percent confidence intervals.

80

13. A

researcher is usually interested in supporting the alternative hypothesis when he/she is engaging in hypothesis testing.

14. When p (.05 is reported in a journal article that we read for an observed relationship, it means that the author has rejected the null hypothesis (assuming that the author is using a significance or alpha level of .05). 15. When p ).05 is reported in a journal article that we read for an observed relationship, it means that the author has accepted the null hypothesis (assuming that the author is using a significance or alpha level of .05). 16. Confidence

limits are the values that mark the boundaries of the confidence interval.

17. Type

II error results if we fail to reject the null hypothesis when the null hypothesis is actually false.

18. A

good way to get a small standard is to use a large sample.

19. An

example of a point estimate is as follows: The car will probably cost about $16,000.

20. The following steps would be included in hypothesis testing:

81

a) State the null and alternative hypotheses b) Set the significance level before the research study c) Obtain the probability value using a computer program such as SPSS d) Compare the probability value to the significance level and make the statistical decision 21. A

confidence interval is a range of numbers inferred from the sample that has a certain probability of including the population parameter over the long run.

22. Standard

error is the standard deviation of a sampling distribution.

23. An

effect size indicator is a statistical measure of the strength of a relationship.

24. The following can be viewed as an effect size indicator: a) r-squared b) r c) Eta-squared d) Omega-squared 25. When

the researcher rejects a true null hypothesis, a Type I error occurs.

26. A

post hoc test is a follow-up test to the analysis of variance when there are three or more groups.

82

27. Inferential

statistics uses the laws of probability to make inferences and draw statistical conclusions about populations based on sample data.

28. One-way

analysis of variance is a statistical test used to compare 2 or more group means.

29. A

statistical test used to determine whether a correlation coefficient is statistically significant is called the t-test for correlation coefficients.

30. The

cutoff the researcher uses to decide whether to reject the null hypothesis is call the significance level or alpha level.

31. A

99 percent confidence interval will be the widest (i.e., the least precise) for a particular data set that includes exactly 500 cases.

32. As a sample size goes up, the following tends to happen to 95% confidence intervals: a) They become more precise b) They become more narrow 33. Type

II error is the failure to reject a false null hypothesis.

34. According to the logic of hypothesis testing, the following statements are true:

83

a) When the null hypothesis is true, it should not be rejected b) When the null hypothesis is false, it should be rejected 35. What is the key question in the field of statistical estimation? Based on my random sample, what is my estimate of the population parameter? 36. Assuming innocence until “proven” guilty, a Type I error occurs when an innocent person is found guilty. 37. Sampling

error is the difference between a sample statistic and the corresponding population parameter.

38. The

“equals” sign (=) is included when conducting null hypothesis testing.

39. A

Type I error is also known as a false positive.

40. A

Type II error is also known as a false negative.

41. If a finding is statistically significant one must also interpret the data, calculate an effect size indicator, and make an assessment of practical significance.