Dissertation Defense 042803

DIGITAL EQUITY IN EDUCATION AND STATE-LEVEL EDUCATION TECHNOLOGY POLICIES: A Multi-Level Analysis Jonathan D. Becker Doctoral Candidate Teachers College, Columbia University April 30, 2003

INTRODUCTION • Equity in Education • Education Technology • States as Education Policy Makers • DIGITAL EQUITY IN EDUCATION AS A MULTI-LEVEL ORGANIZATIONAL PHENOMENON

RESEARCH QUESTIONS • Are education technology resources (computer access and computer use) distributed differentially across different student and school demographic categories? Or, is there digital equity in education across the country? • What is being done by the states that might contribute to the distribution of education technology resources, and to what effect?

EDUCATION TECHNOLOGY AS A RESOURCE •COMPUTER ACCESS – Quantity • How much hardware? • How much Internet connectivity? – Quality • Newness vs. Obsolescence • “Thickness” of Internet pipes

•COMPUTER USE – Quantity • How often? • For how long? – Quality • How educationally beneficial?

SOURCE: Attewell, P. (2001). The First and Second Digital Divides. Sociology of Education, 74(3), 252-59.

EDUCATIONAL EQUITY: A DEFINITION If we could secure the ideal of educational equity, without cost to any other ideals, then we would have secured a statistically describable social condition within which there is: A random distribution of resources, attainment, and educational achievement in respect to variables irrelevant to educational justice together with a predictable distribution in respect to variables relevant to educational justice. SOURCE: Thomas F. Green. (1982). Excellence, Equity and Equality. In Shulman and Sykes (ed.) Handbook of Teaching and Policy, pp. 318-41.

EDUCATIONAL EQUITY: A DEFINITION A random distribution of…

RESOURCES

ATTAINMENT ACHIEVEMENT

with respect RACE to… SEX SES GEOGRAPHY

AND a predictable distribution RESOURCES ATTAINMENT ACHIEVEMENT of… with respect CHOICE to… ABILITY VIRTUE

DEFINING DIGITAL EQUITY IN EDUCATION • Digital equity in schools is the application of Green’s (1982) definition of educational equity to Atewell’s (2001) notion of two digital divides (access and use) in education technology. • Digital equity in schools is a statistically describable condition whereby access to technology is randomly distributed between schools according to educationally irrelevant school variables (e.g. racial composition and urbanicity) and the use of education technology is randomly distributed within schools according to educationally irrelevant student variables (e.g. sex, race, SES and geography). • Assumption: those distributions are random even after controlling for relevant distributive variables (e.g. choice, virtue, etc.)

STATES AND ED. TECH. POLICY • For a host of reasons, the balance of power in our multi-layered, fragmented governance structure over education has shifted to the states. • Education technology policy is now a particularly centralized domain of educational policy increasingly within the purview of state education agencies. • Furthermore, from the federal government down to the states (and, ultimately, to districts and schools), the major goal of early days education technology policy was the equitable distribution of technology resources; i.e. digital equity in education.

ANALYTIC FRAMEWORK: Operationalizing Digital Equity in Education As a Multilevel Organizational Phenomenon

DATA SOURCES • NAEP (2000) State Mathematics Assessment Dataset – Nationwide indicators of student performance and a cross-sectional survey of conditions and practices – The data for these analyses come from the 2000 State NAEP 4th grade mathematics database • The Milken Exchange on Educational Technology’s (MEET) State-by-State Educational Technology Policy Survey – MEET surveyed and interviewed the 50 state education technology directors (or their representatives) in the summer of 1998 – A state-by-state profile of information on state education technology policies.

MULTILEVEL MODELING (HLM) • The survey items from the NAEP data collection process include items completed by an administrator in each of the schools in the sample. By aggregating certain data to the school level, a school-level dataset is created. • Linking the student-level data, the newly created school-level dataset, and the MEET state-level data, a nested data structure (students within schools within states) is created. • Additionally, for the state NAEP, a multi-stage sampling design is used (a stratified sample of schools followed by a random sample of students within the schools). • Therefore, multilevel statistical modeling procedures are appropriate, and the sampling framework necessitates that, for school-level data, a school weight must be calculated and included in the analyses.

MULTILEVEL MODEL SPECIFICATION: Further Operationalization of Digital Equity in Education As a Multilevel Organizational Phenomenon

DEPENDENT VARIABLES •COMPUTER USE •“When you do mathematics in school, how often do you use a computer?” – Almost every day – Once or twice a week – Once or twice a month – Never or hardly ever

See table 13, p. 100

•COMPUTER ACCESS

•“Are computers available all the time in the classrooms?” •“Are computers available in a computer lab?” •“Are computers available to the classrooms when needed?” •Combined into a single school-level measure of computer access points

See table 17, p. 104

INDEPENDENT VARIABLES • Student-Level – Sex – Race (3 categories, see table 19, p. 106) – SES (Eligibility for free or reduced-price lunch, see table 20, p. 109)

• School-Level – Percent African-American (see table 21, p. 109)

– Percent Latina/o (see table 21, p. 109) – Urbanicity (3 categories, see table 22, p. 109)

– Computer Access

INDEPENDENT VARIABLES (cont’d)

• State-Level – Technology-related credentialing requirements (3 variables, see table A1, p. 218)

– Student standards for technology (3 variables, see table A2, p. 219)

– Professional development for education technology (2 variables, see table A3, p. 220)

– Education technology funding (3 variables, see tables A4-A7, pp. 22124).

HLM AND NON-LINEAR ANALYSES

• The two dependent variables are ordered, categorical variables. • BUT, the standard HLM assumes that the dependent variable and the residuals at each level are normally distributed. • Fortunately, within the HLM software, it is possible to specify a nonlinear analysis for ordinal outcomes (but see footnote 25, p. 111). • For multi-category, ordinal data, the levelone sampling model is a cumulative probability model and the link function is a logit link function. The structural model assumes “proportional odds.”

FINAL MODEL SPECIFICATIONS • Computer Access = 2 levels (schools within states); Computer Use = 3 levels (students within schools within states) • The fully unconditional, random-intercept only model – Partitions the variance in the outcome by the different levels of analysis

• Basic models with random intercepts and fixed slopes – One-way ANCOVA with random effects – “Step-up” strategy – All predictors are fixed and grand-mean centered

• Basic models with random intercepts and random slopes where appropriate

RESULTS: Digital Equity as a Multilevel Organizational Phenomenon and State Effects

COMPUTER ACCESS ( p. 157) • Random Effects (Intercept only) – The variance component of the fully unconditional model (0.156) yields an ICC of .045, meaning that 4.5% of the variance in computer access is between-state variance. – The variance component does not change much between the three models. – None of the slopes of the schoollevel predictors varies randomly between states.

COMPUTER ACCESS ( p. 157) • Fixed Effects – Two school-level predictors prove statistically significant (the same as the ordinal logistic regression). The likelihood of increased computer access: • Decreases for schools with higher percentages of African-American students (-.719) • Decreases for schools in rural areas (-.317)

– State funding variables do not show any relationship to school-level computer access.

COMPUTER USE: RANDOM-INTERCEPT, FIXED SLOPES (p. 162)

• Random Effects – In a three level HGLM, there are two sets of random effects (level 3 and level+level 2). – The level 3 variance component of the fully unconditional model (0.024) yields an ICC of 0.007, meaning that less than one percent of the variance in computer use is between-state variance. – The level 1 + level 2 variance component ranges from 0.599 and 0.577, meaning that the proportion of the variance that is between the schools is right around 15%. Thus, most of the variance in computer use exists within schools.

COMPUTER USE: RANDOM-INTERCEPT, FIXED SLOPES (p. 161)

• Fixed Effects: Level-1 – All four level-1 predictors demonstrate statistically significant relationships to the dependent variable, but only two have practically significant effects. • Latina/o students are more likely to fall into a higher use category than other students (0.126). • African-American students are more likely to fall into a higher use category than other students (0.288).

COMPUTER USE: RANDOM-INTERCEPT, FIXED SLOPES (p. 161)

• Fixed Effects: Level-2 – Two level-2 predictors demonstrate statistically significant relationships to the average (unit-specific) level-1 intercept: • The higher the percentage of AfricanAmerican students in a school, the more likely that a given student in that school will report a higher use category. • Students in schools where there are computers available all the time in the classrooms (as opposed to or in addition to a lab) are more likely to use computers more frequently.

COMPUTER USE: RANDOM-INTERCEPT, FIXED SLOPES (p. 161) • Fixed Effects: Level-3 – Given that the level-3 variance component is so small, the estimates of level-3 fixed effects need to be interpreted with caution. Also, because there are only 40 level-3 units, parsimony is tantamount. – Three level-3 covariates demonstrate significant relationships to the average level-2 intercept: • In states where student technology standards are integrated into the subject area standards, students are likely to report lower levels of computer use. • In states where pre-service teachers have to meet technology-related requirements in order to receive their initial credential, students are likely to report higher levels of computer use. • In states where earmarked state funds for technology are distributed as competitive grants, students are likely to report higher levels of computer use.

COMPUTER USE: RANDOM-INTERCEPT, RANDOM SLOPES (p. 172)

• Random Effects – The effects of all four studentlevel predictors varies randomly across schools (i.e. the magnitude of the effect of race, sex or SES is different in different schools). – The proportion of variance attributable to the states is still less than 1%.

COMPUTER USE: RANDOM-INTERCEPT, RANDOM SLOPES (p. 172)

• Fixed Effects (slopes as outcomes) – Having computers available in the classrooms increases the likelihood of more frequent computer use for girls and for students who are eligible for free or reduced-price lunch. – Students who are eligible for free or reduced-price lunch have an increased likelihood of more frequent computer use in urban schools and in schools with greater concentrations of AfricanAmerican students. – African-American and Latina/o students have an increased likelihood of more frequent computer use in schools with greater concentrations of students of their own race.

COMPUTER USE: RANDOM-INTERCEPT, RANDOM SLOPES (p. 172)

• Fixed Effects – Average computer use is still lower in schools in rural areas. – Having computers available in a lab setting increases the likelihood of more frequent computer use (NOTE switch from fixed effects, fixed slopes models) – The state effects are similar to those reported for the fixed effects, fixed slopes models.

CONCLUSIONS • DIGITAL EQUITY – Computer Access • There is an increased probability of lower levels of computer access in rural schools and in schools with higher percentages of AfricanAmerican students.

– Computer Use • There are differences in the likelihood of computer use by race, sex and SES of the students, but those effects vary from school-to-school.

CONCLUSIONS • STATE EFFECTS – States have taken dramatically different approaches to education technology, but no more than 5% of the variance in computer access can be attributed to state factors, and less than 1% of the variance in computer-use is state-to-state variation. – While no state-level predictors are related to computer access at the school level, three state-level predictors consistently demonstrate a significant relationship to student-level computer use (not access): • Integrated technology standards (-) • Pre-service teacher tech. requirements (+) • Earmarked funds distributed competitively (+)

DISCUSSION