Reading Intervention 2

Attributes of Effective and Efficient Kindergarten Reading Intervention: An Examination of Instructional Time and Design Specificity Deborah C. Simmons, Edward J. Kame’enui, Beth Harn, Michael D. Coyne, Mike Stoolmiller, Lana Edwards Santoro, Sylvia B. Smith, Carrie Thomas Beck, and Noah K. Kaufman

Abstract A randomized experimental design with three levels of intervention was used to compare the effects of beginning reading interventions on early phonemic, decoding, and spelling outcomes of 96 kindergartners identified as at risk for reading difficulty. The three instructional interventions varied systematically along two dimensions—time and design of instruction specificity—and consisted of (a) 30 min with high design specificity (30/H), (b) 15 min with high design specificity plus 15 min of non-code-based instruction (15/H+15), and (c) a commercial comparison condition that reflected 30 min of moderate design specificity instruction (30/M). With the exception of the second 15 min of the 15/H+15 condition, all instruction focused on phonemic, alphabetic, and orthographic skills and strategies. Students were randomly assigned to one of the three interventions and received 108 thirty-minute sessions of small-group instruction as a supplement to their typical half-day kindergarten experience. Planned comparisons indicated findings of statistical and practical significance that varied according to measure and students’ entry-level performance. The results are discussed in terms of the pedagogical precision needed to design and provide effective and efficient instruction for students who are most at risk.

F

indings of a recent longitudinal study sponsored by the National Center for Educational Statistics (NCES) underscored the important academic foundation laid in kindergarten, particularly for children with identified risks (West, Denton, & Reaney, 2001). Researchers found that although the majority of the 22,000 kindergarten children recognized letter names and beginning sounds in words, only half demonstrated proficiency with letter–sound relationships at the end of words or the ability to read words by sight. These foundational skills relate to later reading success (Francis, Shaywitz, Stuebing, Shaywitz, & Fletcher, 1996; Stage, Abbott, Jenkins, & Berninger, 2003; Wagner & Torgesen, 1987) and are recognized ex-

pectations of kindergarten children, as reflected in numerous state standards and national reports. Although a convergence of research provides substantive guidance on how to prevent delays in foundational reading skills (National Reading Panel, 2000; National Research Council, 1998; Shaywitz, 2003), questions remain about the design attributes of interventions necessary to close the gap for children who enter kindergarten at risk for later reading difficulty.

Content of Prevention A preponderance of evidence suggests that children who struggle with early reading lack facility with the phonoJOURNAL OF LEARNING DISABILITIES VOLUME 40, NUMBER 4, JULY/AUGUST 2007, PAGES 331–347

logical structure of the English language (Vellutino et al., 1996; Wagner & Torgesen, 1987; Wagner, Torgesen, & Rashotte, 1994). Specifically, they lack sensitivity to the phonemes in words, and they struggle with the alphabetic principle or the ability to decode unfamiliar words (Byrne & FieldingBarnsley, 1989, 1990; Lundberg, Frost, & Petersen, 1988; Stanovich, 1988; Vellutino et al., 1996). Students with reading difficulties (RD) cannot easily make the connection between the sounds of our language and their printed counterparts that represent speech (Siegel, 1989; Stanovich, 1988). Consequently, they face considerable obstacles translating print to speech and fail to develop ease and facility with word recognition (Share & Stanovich, 1995),

332 which limits their capacity for higher level cognitive processes related to comprehension and, ultimately, the word and world knowledge they gain from reading (Torgesen, 2000). Intervention research provides compelling evidence that phonemic awareness, alphabetic understanding, and decoding are teachable (e.g., Adams, 1990; O’Connor, Jenkins, & Slocum, 1995; Schneider, Roth, & Ennemoser, 2000; S. B. Smith, Simmons, & Kame’enui, 1998) and that instruction results in significant gains for most children (National Reading Panel, 2000; Shaywitz, 2003; Vellutino et al., 1996). Moreover, there is converging evidence to suggest that an emphasis on alphabetic skills and phonological awareness positively influences both phonemic awareness and word reading outcomes (e.g., Ball & Blachman, 1991; Byrne & Fielding-Barnsley, 1989, 1991; Oudeans, 2003; Schneider et al., 2000; Vandervelden & Siegel, 1997). Furthermore, the National Reading Panel’s (2000) synthesis of experimental studies identified a set of nominal attributes of instruction as being positively related to kindergarten and first-grade phonemic awareness and phonics outcomes, including (a) emphasis of a few priority phonemic awareness skills, (b) integration of letters with sounds, (c) small-group instruction, and (d) use of explicit, systematic instruction. Although the scientific knowledge base provides guidance about the content and process of prevention-oriented intervention, sufficient evidence is lacking on two instructional attributes that are likely to influence learning outcomes for students at risk for RD: instructional time and design specificity.

Instructional Time One of the most consistent educational findings is that the amount of time that children are actively engaged in tasks they can perform successfully contributes significantly to achievement (Berliner, 1978, 1990; Marzano, 2000). More than 4 decades ago, Carroll (1963) proposed a model of school learning to

JOURNAL OF LEARNING DISABILITIES

guide the solution of educational problems grounded in the economics of instruction. Carroll’s model was based on the idea that only a few critical variables influence student learning, and central to this model was the role of time or opportunity to learn. He espoused the belief that economy of learning can be realized when time spent in learning equals time needed for learning. Consistent with Carroll’s model, it is common practice to add time beyond typical allotments for children who struggle with learning to read. Over the past decade, supplemental reading instruction that increases opportunity to learn has become a common focus of educational research and practice (Linan-Thompson & HickmanDavis, 2002; O’Connor, 2000; Torgesen, 2000). Cavanaugh, Kim, Wanzek, and Vaughn (2004) reviewed 27 intervention studies to examine the effects of duration and frequency of phonological awareness intervention on the kindergarten literacy outcomes of children identified as at risk for RD. Cavanaugh et al. defined duration as the length of intervention, typically reported in weeks, and intensity as the amount of time per session, reported in minutes. Interventions varied markedly in duration and intensity. Of the studies that reported duration, five studies lasted 4 to 8 weeks, five were 10 weeks long, six were 11 to 15 weeks, and the remaining nine studies ranged in duration from 17 to 28 weeks. In terms of intensity, three of the five interventions that were implemented for less than 15 min per instructional session resulted in large effect sizes (1.0 or higher). The majority of interventions were implemented from 15 to 29 min (n = 17) and had moderate to large effect sizes. Cavanaugh et al. concluded that interventions delivered in small-group formats, either two to three times per week or daily, for 15 to 30 min, for a duration of 8 to 10 weeks, produced the greatest effects. However, none of the 27 kindergarten studies reviewed specifically manipulated time as an independent variable. Therefore, the fol-

lowing important questions remain to be investigated experimentally: How much time is necessary to develop phonological and alphabetic proficiency? Do programs that vary systematically in the amount of instructional time (e.g., 15 min vs. 30 min of codebased instruction) result in differential outcomes?

Instructional Design Specificity A second attribute that has received attention in recent studies on the prevention of RD is the importance of instructional quality—specifically, the design of instruction. Such prevention research in the early elementary grades has examined a range of design issues, including the explicitness of instruction (Foorman, Francis, Fletcher, Schatschneider, & Mehta, 1998) and the depth of instruction and practice (Torgesen et al., 2001). In a synthesis of the critical elements of reading instruction from studies involving children at risk for RD, Foorman and Torgesen (2001) concluded that although the content emphases of effective instruction (i.e., what is taught) are the same whether the focus is prevention or intervention, instructional design features, such as the level of explicitness, intensity, and scaffolding, are elements that must vary. Instructional design refers to the way information in a particular domain (e.g., social studies, science, reading, mathematics) is selected, prioritized, sequenced, organized, and scheduled for instruction within a highly orchestrated series of lessons and materials that make up a course of study (Simmons & Kame’enui, 1998). According to P. L. Smith and Ragan (1993), instructional design refers to the “systematic process of translating principles of learning and instruction into plans for instructional materials and activities” (p. 2). As Smith and Ragan pointed out, “An instructional designer is somewhat like an engineer—both plan their works based on principles

333

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

that have been successful in the past— the engineer on the laws of physics, and the designer on basic principles of instruction and learning” (p. 2). The instructional designer is concerned with developing the architectural pedagogy for the communication of symbolic information that has a high probability of preventing learner errors and misconceptions and misrules (Tennyson & Christensen, 1986). Instructional design is concerned with the intricacies of analyzing, selecting, prioritizing, sequencing, and scheduling the communication of information before it is packaged for delivery or implemented. It is the behind-the-scenes activity that appears as the sequence of objectives, schedule of tasks, components of instructional strategies, amount and kind of review, number of examples, extent of teacher direction, and support explicated in teachers’ guides and lesson plans. Instructional design is the blueprint for instruction that carries significant potential support for students who may be at risk for RD. Some blueprints are skeletal, providing little instructional specification, and others have fundamental flaws that fail to provide an adequate foundation on which to build further skills and support future learning success. For the purposes of this study, instructional design was conceptualized according to the level of specificity. The experimental interventions were designed around an instructional framework (Carnine, 1994; Kame’enui, Carnine, Dixon, Simmons, & Coyne, 2002) derived from studies of high-quality instruction for students with diverse learning needs. We considered these principles fundamental to the design of early reading instruction that responds to the acute needs of students at risk for reading failure (Simmons & Kame’enui, 1998) and included big ideas, mediated scaffolding, conspicuous strategies, strategic integration, and judicious review. Furthermore, instructional design specificity was considered along a continuum ranging from highly specified, through moderately specified, to minimally speci-

fied instruction. Highly specified instructional design consistently and explicitly addressed each of the previous design principles, as demonstrated through conspicuous instructional language and feedback, systematically integrated instructional components, judicious and sufficient review cycles, and carefully prescribed example selection and scheduling. The converging scientific knowledge base in reading and RD provides a solid foundation for preventionbased practice. Specifically, numerous studies corroborate the benefits of early intervention in phonemic awareness and phonetic decoding. Still, we lack instructional precision in terms of how much of what content and under what conditions promotes and accelerates learning (Lyon & Moats, 1997). Thus, research calls for further investigation of instructional time and the design specificity of instruction necessary to effectively and efficiently promote early reading proficiency. To date, few kindergarten studies have compared the effects of instructional design specificity along selected dimensions (e.g., explicitness, level of integration). To our knowledge, the present study is the only one that has investigated the effects of phonemic awareness and alphabetic and spelling instruction while systematically controlling instructional time and design specificity. In this study, we examined whether variations in instructional time and design specificity differentially influenced rates and levels of learning. To examine instructional time, we contrasted two instructional conditions that varied systematically in the time dedicated to phonological, alphabetic, and orthographic instruction (i.e., 30 min vs. 15 min). The first 15 min of the two conditions included identical code-emphasis instruction. The second 15 min varied, with one condition continuing a code-based emphasis and the other emphasizing vocabulary and listening comprehension through a storybook reading activity. We included the 15 min of storybook reading activities to control for total instructional time

across the two conditions. Our comparison of interest, however, was between the 30 min and the 15 min of code-based instruction. To examine design specificity, we contrasted two instructional conditions that included a similar emphasis on phonological, alphabetic, and orthographic instruction over 30 min, but varied in design specificity (i.e., a highly specified, experimental intervention and a moderately specified commercial program). Both conditions emphasized code-based instruction and incorporated principles of effective instructional design, but they varied in the specificity of task scaffolding, example selection, teacher language, review cycles, and corrective feedback (see intervention descriptions). Our objective was to evaluate instructional variables required to accelerate learning rates for children who enter kindergarten with prereading performance indicators that place them at risk for later RD.

Method Participants In September of their kindergarten year, 116 students from seven elementary schools in the Pacific Northwest were screened on the Letter Naming Fluency (LNF) and Onset Recognition Fluency (OnRF) Dynamic Indicators of Basic Early Literacy Skills (DIBELS) measures (Kaminski & Good, 1996; see description of measures to follow) and selected to participate in the study based on the following criteria: 1. They scored at or below the 25th percentile in the district on both measures (i.e., less than 11 on OnRF and less than 6 on LNF) 2. Their performance was confirmed by kindergarten teachers as being at risk for RD. Children were excluded if they had (a) severe hearing or visual acuity problems or (b) were determined by school personnel to have significantly

334 limited English proficiency. All participating kindergartners then completed the Peabody Picture Vocabulary Test– Revised (PPVT-R; Dunn & Dunn, 1981) to determine their baseline level of receptive vocabulary knowledge. Socioeconomic status (SES), race, and gender were allowed to vary consistent with the district population from which the sample was selected. All participating schools received Title I funding, and the percentage of students qualifying for free- and reducedcost lunch services ranged from 32% to 63%. In terms of overall enrollment, schools ranged from 319 to 683 students; time allocated for kindergarten in all schools was 2.5 hours per day. Participating children were primarily European American (n = 94; 83.93%) and Latino/Hispanic (n = 15; 13.39%). Two of the children were African American, and one did not specify race or ethnicity. Fifty-eight percent of the sample were boys (n = 65); the mean age for students in the fall was 5 years 7 months, with a range from 5 years 0 months to 6 years 9 months. Interventionists included 4 certified teachers and 24 educational assistants between 35 and 44 years of age. The typical interventionist had a high school education with some college coursework and an average of 5.7 years instructional experience in schools.

Experimental Design A pretest–posttest experimental design with three levels of intervention (between-groups factor) was used to compare the effects of beginning reading interventions on early reading and vocabulary outcomes. The three instructional interventions varied systematically along two dimensions: (a) the time apportioned to the content taught (e.g., phonological, alphabetic, and orthographic) and (b) the degree of design specificity provided in the content taught. The three interventions consisted of (a) 30 min of phonological, alphabetic, and orthographic code-

JOURNAL OF LEARNING DISABILITIES

based, highly specified instruction (30/H); (b) 30 min of instruction partitioned into 15 min code-based, highly specified instruction and 15 min storybook activities (i.e., vocabulary and listening comprehension) instruction (15/ H+15); and (c) a commercial comparison condition that reflected 30 min of code-based, moderately specified instruction (30/M). Using a stratified random sampling procedure, all eligible participating students within a school were randomly assigned to one of the three interventions. Due to the young age of the children and the intensity of the intervention, group size was limited to five or fewer children. Each school implemented all three instructional treatments; the total number of groups per intervention was 10 for 30/H, 9 for 15/H+15, and 11 for 30/M. The number of intervention groups per school varied depending on the number of students identified as at risk and the size of the school, with a maximum of seven and a minimum of three intervention groups per school.

Instructional Procedures and Materials From November through mid-May, students received one of three 30-min early reading interventions supplemental to their typical 2.5-hour kindergarten day. The small-group interventions, which occurred during extended kindergarten hours (i.e., either before or after the regular kindergarten instructional day), were conducted by either certified teachers or teaching assistants at the child’s school. On average, children received 108 days of supplemental, small-group intervention for a total of 54 hours over the course of the year. This instruction was designed to extend instructional opportunities to the typical kindergarten reading instruction, which included 45 to 60 min of daily instruction. Four of the schools used Open Court, one used Reading Mastery, and two schools used

a combination of commercial materials for kindergarten reading instruction. A description of the instructional conditions follows. Thirty-Minute/Highly Specified Design Intervention. The design intervention 30/H emphasized strategic and systematic instruction of phonemic awareness, alphabetic understanding, letter writing, and spelling and consisted of two 15-min components delivered consecutively in daily 30min lessons. In the first half of 30/H, instruction established and reinforced the phonological skills of (a) first and last sound isolation, (b) sound blending, and (c) sound segmentation. Furthermore, the intervention emphasized the acquisition and application of fundamental alphabetic skills and strategies of (a) letter-name identification, (b) letter-sound identification, (c) letter-sound blending to read consonant-vowel-consonant (CVC) words, (d) selected irregular word reading, and (e) sentence reading of controlled text. The second 15 min reinforced previously taught phonological awareness and alphabetic skills and extended these skills through instruction in handwriting (e.g., letter dictation and formation), integrated phonological and alphabetic tasks, and spelling. Handwriting instruction did not focus on the physical production of letters but on the connection between sounds and orthography. The spelling component began with tracing and writing previously taught letters and progressed to writing initial and final sounds in words, and then to the systematic sequential analysis and synthesis of all sounds and letters in CVC and CVCC (consonant–vowel–consonant–consonant) words. During the last 3 weeks, instruction and practice focused on reading controlled text of three- to five-word sentences. All lessons were original and did not use components of commercially available materials. The highly specified design lessons were purposefully constructed

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

around the following principles of instructional design: big ideas, scaffolding, conspicuous strategies, strategic integration, and review (Simmons & Kame’enui, 1998). The scope and sequence of skills and strategies was restricted to those most essential for early reading success. For example, rather than teach the full continuum of phonological awareness skills, only sound isolation, blending, and segmentation were emphasized. Simpler tasks within a dimension of early reading (e.g., phonological awareness) were introduced prior to more difficult tasks (e.g., first sound isolation before final sound). Tasks prerequisite to more difficult tasks were introduced and taught for a sufficient period of time to develop proficiency. Example selection and sequence were carefully controlled. For example, the sequence of letter sounds was carefully prescribed to minimize potential confusion, only the short sounds of vowels were introduced, and pictures for phonemic awareness tasks were selected to allow students to work with familiar information and highly salient first sounds (e.g., /t/, /s/, /c/). All highly specified lessons provided detailed scripting to ensure that clear and consistent information was communicated and to reduce variability in implementation. When introducing a new skill, the teacher modeled the information several times using consistent wording. Furthermore, skills were carefully integrated to enhance learning. Concurrent skills involving phonemic awareness and phonemic decoding were integrated when appropriate (e.g., substituting the letter tile for /s/ in the first position of the word sun), resulting in higher-order applications. Highly specified lessons concentrated on teaching a prerequisite skill (e.g., phonemic isolation of first sound) and then strategically integrating concurrent or corequisite skills. Moreover, lesson components were related by tasks that connected phonemic awareness, reading, and spelling. All new in-

formation (e.g., letter sounds, irregular words) was introduced for a minimum of 3 days and reviewed for a minimum of 12 sessions. Lessons were introduced in 6-day cycles. In each cycle, either two new consonant letter names/ sounds were introduced or one vowel was taught during the 6-day cycle. Scheduled instruction, review, and feedback were likewise highly specified. Each lesson that introduced new information included a specified number of instructional interactions in which the teacher modeled the information and students practiced the new skill with the teacher and then applied the skill to new, untaught discrimination or generalization tasks. Furthermore, the intervention provided teachers with explicit instructional language and procedures for correcting errors and extending practice for difficult items. A lesson excerpt illustrating activities and instructional specification is found in the Appendix. This excerpt occurs at almost the midpoint of the curriculum and represents one activity within the second 15-min period that focused on the integration of phonemic, alphabetic, and orthographic instruction. Fifteen-Minute/Highly Specified Design + Fifteen-Minute Intervention. To ensure equivalence of time among intervention groups, the intervention 15/H+15 partitioned emphasis across two areas of reading—code (i.e., phonological, alphabetic, orthographic) and comprehension (i.e., vocabulary, narrative text structure, story retell)— and consisted of two 15-min, consecutive teaching segments per day. The first 15 min included the same lessons as the 30/H intervention and focused on high-priority phonological and alphabetic skills. The second 15 min had two primary foci: (a) receptive and expressive knowledge of vocabulary that appeared in storybooks, and (b) expanded knowledge and development of story structure and story retell. The comprehension portion that was part of the second 15 min was built on the

335 research of Dickinson and Smith (1994), Senechal and Cornell (1993), and Whitehurst et al. (1994, 1999). Critical features included (a) repeated reading of stories, (b) targeted vocabulary selection and instruction, (c) dialog discussion of vocabulary and stories, (d) multiple exposures to target vocabulary, and (e) systematic review and integration of vocabulary words within and between lessons within a cycle. (For a more detailed description of the effects of the storybook instruction on students’ vocabulary and comprehension outcomes, see Coyne, Simmons, Kame’enui, & Stoolmiller, 2004). Thirty-Minute/Moderately Specified Design Intervention. The intervention 30/M was designed to partial out the independent contributions of time and design specificity. The entire 30 min of instructional time was allocated to phonological, alphabetic, and orthographic activities. However, the specificity of instructional design was considered moderate compared to the 30/H condition. The 30/M intervention was based on the Sounds and Letters component of the Open Court Reading 2000 (Adams et al., 2000) comprehensive reading program. A full implementation of Open Court Reading 2000 requires 90 or more minutes of daily instruction. Because this intervention was supplemental and time restricted, only the Sounds and Letters module was used. In this module, instruction focused on the following sections: Sounds, Letters, and Language; Phonemic Awareness; and How the Alphabet Works (e.g., integrated alphabetic and phonological awareness, writing letters, and the reading and writing workbook). In contrast to the code-emphasis component of the 30/H and 15/H+15 interventions, the 30/M intervention was broader in scope and focus. On a typical day, children listened to and recited rhymes, poems, and songs; played language and word games to apply knowledge of sounds, letters, and language; engaged in phonemic aware-

336 ness activities; learned a new letter name and sound; used new sounds in sentences; and wrote new letters in the writing workbook. In contrast to the other two interventions, the moderately specified instruction provided a more integrated approach to reading and language development. For example, when reviewing rhyming words, the teacher read poetry aloud and then asked students to identify words that rhymed. We characterized the 30/M module of Open Court as moderate in design specificity for several reasons. The scope and sequence within and between lessons was highly specific, yet broader in scope than in the other interventions. For example, the range of phonological awareness skills covered a full complement of skills from rhyming to sound substitution. Within the Sounds and Letters module, lessons included seven to eight different, yet integrated, activities. No time allocations were specified per task, leaving the pacing and emphasis to the teacher’s discretion. The sequence of letter sounds was carefully prescribed, and as in the high-specificity interventions, only short pronunciations of vowels were introduced. The explicitness of teacher language varied by activity. On selected activities, such as oral blending in phonological awareness or new sound introduction, teacher language was scripted and highly specific. On other activities, more general instructions were typical. Finally, in contrast to the high-specificity interventions, the example selection was more flexible and less specified. Teachers or students were often encouraged to brainstorm to identify words that began with target sounds or that exemplified the focus of instruction. Finally, with respect to review and feedback, the moderate-specificity intervention offered directions for when and how to introduce and practice a skill, and reminders of what key skills to review when children had difficulty. In contrast to the other two experimental interventions, the number of teacher models and practice exam-

JOURNAL OF LEARNING DISABILITIES

ples were less prescribed, and the corrective feedback was less specific.

Dependent Measures Measures were selected to address five dimensions of early language and literacy: (a) phonological awareness, (b) phonemic decoding and word reading, (c) letter-name retrieval and formation, (d) spelling, and (e) general receptive vocabulary. The large number of measures reflects the progression of skill development and the need to include measures that would not result in large floor effects in early kindergarten due to task difficulty. More sophisticated measures of phonemic decoding and phonemic awareness were administered at midyear and at the end of the year to ensure that we captured and reflected children’s growth on more complex tasks that occurred in the second half of kindergarten. To address the efficiency and effectiveness of interventions, we included fluencybased (timed) and nonfluency or accuracy-based measures. Measurement periods included (a) screening, (b) preand postintervention, and (c) postintervention-only measurement of word reading, as it was considered too difficult for beginning-of-kindergarten pretest administration. Reliability coefficients for the commercial measures are those reported in the test manuals for children in the age ranges of students in this study. Interrater reliability coefficients for the experimental measures are based on the current study. Phonological Awareness Measures Yopp-Singer Test of Phoneme Segmentation. This measure (Yopp, 1995) represented an accuracy-based test of the ability to segment whole words that was appropriate for beginning-ofkindergarten administration. It is a “standardized, individually administered test of a child’s ability to separately articulate the sounds of a spoken word in order” (Yopp, 1995, p. 21). In this untimed measure, the examiner says a word, which the child then seg-

ments into its individual phonemes in the correct order of the presented word. The final score is the number of total words and the percentage of words segmented completely of the 22 words administered. Interrater reliability on the Yopp-Singer is reported as .95, and the correlation with the Comprehensive Test of Basic Skills reading cluster in first grade is .43 (Yopp, 1995). The Yopp-Singer test was administered in October and May. DIBELS Onset Recognition Fluency (OnRF). This standardized, individually administered beginning measure of phonological awareness assesses a child’s ability to recognize and produce the initial sound in an orally presented word. In this test, the examiner presents four pictures to the child, names each picture, and then asks the child to identify (i.e., point to or say) the picture that begins with the sound the examiner produces. The child is also asked to produce orally the onset for an orally presented word that matches one of the given pictures. The examiner calculates the amount of time the child takes to identify and produce the correct sound and converts the score into the number of onsets correctly produced in 1 min. Alternateform reliability of the OnRF is .65, and test–retest reliability ranges from .65 to .90 (Kaminski & Good, 1996). Predictive validity coefficients range from .28 to .51 (Kaminski & Good, 1996). Alternate forms of the OnRF were administered monthly from September to January to assess growth in beginning phonological awareness. DIBELS Phonemic Segmentation Fluency (PSF). This standardized, individually administered test of phonological awareness assesses the ability to produce the individual phonemes in three- to four-phoneme words presented orally by the examiner. In this study, it represented a fluency-based indicator of phonemic segmentation that—because of task difficulty—is most appropriately first administered in the middle of kindergarten. Credit is awarded for each phoneme or segment of the word produced, and the number

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

of correct phonemes produced within 1 min determines the final score. This fluency-based measure has an alternate-form reliability of .88 and predictive validity coefficients ranging from .73 to .91 (Kaminski & Good, 1996). Alternate forms of the PSF were administered monthly from January to May to assess growth in phonological awareness. Phonemic Decoding and Word Reading DIBELS Nonsense Word Fluency (NWF). This is a standardized, individually administered test of letter– sound correspondence and of the ability to blend letters into pseudowords in which letters represent their most common sounds (Kaminski & Good, 1996). This is a fluency-based measure most appropriately first administered in midkindergarten. The student has 1 min to produce as many letter sounds as possible in CVC pseudowords, and the final score is the number of correct letter sounds produced. Alternateform reliability for the NWF ranges from .66 to .87, and concurrent validity with the readiness subtests of the Woodcock-Johnson Psychoeducational Test Battery ranges from .35 to .59. Alternate forms of NWF were administered monthly from January to May to assess growth in alphabetic understanding. WRMT-R Word Attack subtest. The Woodcock Reading Mastery Test– Revised (WRMT-R; Woodcock, 1987) is a standardized, individually administered test that assesses a child’s ability to read a list of nonwords (e.g., tet) presented in isolation. The subtest contains 45 words of increasing difficulty; students earn 1 point for each word read correctly, which is converted into a standard score. WRMT-R Word Identification subtest. Consisting of 100 words of increasing difficulty, this test measures a child’s skill in reading real words presented in isolation. The raw score is the number of words correctly read. Internal consistency reliability for both subtests of the WRMT-R ranges from .92 to .98 (Woodcock, 1987); no specific relia-

bilities are reported for kindergarten. Because of the potential floor effect of these subtests for kindergarten children in the fall, both subtests were administered in May at posttest only. Letter Dictation Fluency and Spelling Measures Letter dictation fluency. A measure modified from Berninger et al.’s (1997) writing dictation measure was used to assess students’ ability to retrieve and produce letters automatically. Administered pre- and postintervention, this measure assessed growth in writing letters from memory in response to a dictation of the 26 alphabet letters presented in random order. In this study, Berninger et al.’s original measure was modified to a 1-min timed administration to address both letter writing accuracy and rate of production. Student responses to letter dictation were scored according to the number of capital or lowercase letters written correctly in 1 min. Interrater scoring reliability calculated on 20% of the total pre- and postintervention letter dictation tests was 1.00. In the present study, the concurrent validity correlations of the letter dictation fluency measure with the WRMT-R Word Attack and Word Identification measures were .60 and .51, respectively. Spelling. The ability to translate spoken language to print was assessed pre- and postintervention through a spelling measure originally developed by Tangel and Blachman (1992, 1995). Examiners verbally dictated a target spelling word, used the target word in a sentence, and asked the student to spell the word the best that he or she could. The original Tangel-Blachman spelling measure included 6 and 10 words, respectively, for kindergarten and Grade 1. To increase variability in scores and to minimize kindergarten children’s frustration, the original 10word Grade 1 measure was reduced to 8 words by eliminating the two most difficult words. Word difficulty in this application ranged from lap to elephant. Responses were evaluated using a partial-credit scoring system (Tangel &

337 Blachman, 1995) according to a developmental continuum. Respective concurrent validity correlations of the spelling measure with WRMT-R Word Attack and Word Identification were .65 and .62. The interscorer reliability calculated on 20% of the total pre– postintervention spelling tests was .98. Vocabulary Measure Receptive knowledge of general vocabulary. The PPVT-R was administered pre- and postintervention to assess any differential effects of instructional condition on generalized vocabulary knowledge. On this test, the child is presented with four pictures and asked to point to the one that corresponds to a word spoken by the examiner. One point is awarded for each correct response and then converted into standard scores. Test–retest reliability for the PPVT-R is reported as .77. Treatment Integrity Measure. Treatment integrity was evaluated by direct assessment methods (Gresham, MacMillan, Beebe-Frankenberger, & Bocian, 2000) by research team members at eight points during the year: twice per month during the first 2 months of intervention, and once every 3 weeks for the remaining duration. The integrity checks focused on (a) implementation accuracy of components and (b) time spent in instruction. Critical components of each intervention were identified and operationalized, and a quantitative rating list was created for the respective interventions. After each observation, the checklist was reviewed with the interventionist, and specific suggestions were provided to improve lesson implementation. Treatment integrity was evaluated by observing complete instructional sessions and documenting the presence or absence of each critical component. Two points were assigned if the critical component was always demonstrated during the observation, 1 point was given for a component that was observed most of the time, and no

338

JOURNAL OF LEARNING DISABILITIES

points were assigned if a component was not observed. The final component integrity score was calculated by tallying the observed components and dividing by the total possible component score. Total time for each intervention component and total instructional time were also documented for each observation. Prior to conducting integrity checks, observers established an interobserver reliability of .85 or higher.

Results Data Analysis Strategy Planned contrasts were first conducted to evaluate the statistical significance of the effects of instructional time and design specificity. The first contrast evaluated the effect of instructional time when design specificity was controlled by comparing the performance of the 30/H code-based intervention with the 15/H+15 intervention. The second contrast evaluated the effect of design specificity when time was held constant through the 30/H vs. 30/M comparison. The third comparison examined the combination of instructional time and design specificity, contrasting the performance of children in the 15/ H+15 condition with that of the 30/M intervention. In this comparison, we examined the effect and efficiency of a 15-min code component with high specificity versus a 30-min intervention with moderate specificity. For measures collected both preand postintervention, ANCOVA was used to assess posttest group differences, using preintervention scores as the covariate to minimize the potential influence of differences in baseline scores. For NWF, WRMT-R Word Attack, and Word Identification— measures not given in the fall—fall letter naming fluency was used as the pretest covariate. For PSF—also a measure not given in the fall—OnRF was used as the pretest covariate. First, homogeneity of regression across groups was tested. If no differences were detected, the group-by-

pretest interaction terms were dropped from the model and the pooled poston-pre regression for all groups was used to adjust mean posttest differences among the groups. When poston-pre regressions were significantly different, mean comparisons were conducted among groups at selected points in the prescore distribution. For all models, residual diagnostics were carefully scrutinized to assess the adequacy of background statistical assumptions (linearity of regression, normality, and heteroscedasticity of residuals) and to guard against undue distortions due to outliers or points of high influence. Given the planned contrasts, we adjusted alpha levels to minimize Type I errors (Keppel, 1991) using a per comparison alpha of .016 for all research questions. To assess practical significance and facilitate interpretation of findings, effect size indices are provided for all salient analyses in the form of Cohen’s d.

Demographic and Screening Comparability Analyses We first examined whether intervention groups differed on demographic characteristics or on the screening and pretest measures administered in the fall. Due to the length of the intervention, attrition rates were assessed to evaluate the potential effect of students who began but did not complete the intervention. Contingency table analyses showed that attrition rates were comparable across the three intervention groups, χ2(2, N = 112) = 2.91. Analysis of student demographic variables involving intervention completers indicated that ethnicity, gender, age, and attendance did not differ significantly across groups. Finally, analysis of teacher and educational assistant demographics and instructional factors indicated no statistically significant relation among teacher experience, age, highest educational degree, and intervention group. A multivariate analysis of variance (MANOVA) conducted on the combined onset recognition, letter

naming, letter dictation, Yopp-Singer, and PPVT-R measures indicated that intervention groups were not significantly different at pretest, F(12, 166) = 1.47, p > .05. However, the univariate F tests for Yopp-Singer and letter dictation fluency measures were statistically significant, F(2, 87) = 4.01, p < .05, and F(2, 87) = 3.56, p < .05, respectively. Table 1 reports pretest and posttest descriptive statistics for all dependent variables by intervention group. Scores for the PPVT-R, WRMT-R Word Attack and Word Identification are reported in standard scores; all other scores are raw scores.

Statistical and Practical Effects Analyses Phonological Awareness. For the Yopp-Singer Test of Phoneme Segmentation, the regression model indicated no substantial differences in post-on-pre regressions across groups and a significant effect of the pooled pretest on posttest, t(90) = 4.57, p < .001. Planned t tests assessing respective main effects indicated no statistically significant differences for any of the comparisons at the p = .016 level. Tests of practical significance indicated small to moderate effect sizes (see Table 2 for effect sizes by contrast and measure). For the DIBELS OnRF measure, administered in September and January, the results of an ANCOVA indicated no differences in post-on-pre regression across groups, a nonsignificant pooled poston-pre regression, and no statistically significant differences for any of the comparisons. Effect sizes of the respective contrasts were small to moderate, with the strongest effect favoring the 30/H over the 15/H+15 condition, ES = .47. Because the DIBELS PSF measure assessed advanced phonemic awareness skills, the fall OnRF score was used as the covariate. ANCOVA results indicated no differences in poston-pre regression across groups, a nonsignificant pooled post-on-pre regression, and no statistically significant differences among the groups, although the design specificity contrast favoring

339

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

TABLE 1 Pretest and Adjusted Posttest Scores on All Measures by Intervention Group 30/Ha Measures

15/H+15b

30/Mc

Pre

Post

Pre

Post

Pre

Post

LNF M SD

2.78 2.83

38.44 16.93

2.53 2.39

28.59 15.29

2.13 1.74

33.53 17.42

OnRF M SD

7.16 4.02

26.71 11.13

6.68 3.49

21.47 11.02

5.63 3.36

20.19 11.37

Yopp-Singer M SD

9.45 7.14

16.22 5.28

5.30 5.87

16.27 5.21

6.07 6.68

13.87 7.18

PSF M SD

NA

47.46 14.37

NA

40.86 17.69

NA

36.26 17.27

NWF M SD

NA

38.79 17.53

NA

29.11 17.62

NA

25.00 13.38

WRMT-R Word Attack M SD

NA

111.34 10.17

NA

106.88 9.95

NA

103.03 12.78

WRMT-R Word Identification M SD

NA

106.91 9.98

NA

101.47 9.68

NA

98.43 14.40

LDF M SD

4.19 2.51

11.03 4.65

3.18 2.29

7.85 2.75

2.71 1.96

7.69 3.35

TB Spelling M SD

12.28 6.18

44.03 9.83

12.27 5.08

35.77 13.15

12.40 6.67

34.38 13.96

PPVT-R M SD

89.94 13.91

96.23 10.52

92.64 13.63

93.38 15.30

91.770 12.14

94.13 11.06

Note. 30/H = 30-min/high-specificity design intervention; 15/H+15 = 15-min/high-specificity design and 15-min storybook intervention; 30/M = 30-min/moderatespecificity design intervention; LNF = Letter Naming Fluency, from the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Kaminski & Good, 1996); OnRF = Onset Recognition Fluency, from the DIBELS; Yopp-Singer = Yopp-Singer Test of Phoneme Segmentation (Yopp, 1995); PSF = Phonemic Segmentation Fluency, from the DIBELS; NWF = Nonsense Word Fluency, from the DIBELS; WRMT-R = Woodcock Reading Mastery Test–Revised (Woodcock, 1987); LDF = Letter Dictation Fluency (based on Berninger et al., 1997); TB Spelling = Spelling test (Tangel & Blachman, 1992, 1995); PPVT-R = Peabody Picture Vocabulary Test–Revised (Dunn & Dunn, 1981). LNF was used as a screening measure but not as an outcome measure. The posttest for OnRF was administered in January. For Yopp-Singer, the 30/H versus 15/H+15 contrast was significant at p = .013; For LDF, the 30/H versus 30/M contrast was significant at p = .0142. No other contrasts for any of the fall measures were significant at p < .016. an = 32. bn = 34. cn = 30.

the 30/H over the 30/M condition approached significance, t(90) = 2.35, p = .021, ES = .59. Phonemic Decoding and Word Reading. Fall letter naming fluency was used as the pretest covariate for phonemic decoding, as measured by the DIBELS NWF and WRMT-R Word Attack tests, and word reading, as

measured by the WRMT-R Word Identification task. ANCOVA results for NWF indicated no differences in poston-pre regression across groups and a statistically significant pooled post-onpre regression, t(92) = 2.76, p = .007. The 30/H group did better than the 15/H+15 group, although the difference was marginally statistically significant, t(92) = 2.39, p < .018, ES = .60.

The 30/H group did substantially better than the 30/M group, t(92) = 3.21, p < .002, ES = .82. The 15/H+15 vs. 30/M contrast was nonsignificant. Nonetheless, end-of-year scores indicated that the mean performance of all groups was above 25, which represents the 40th percentile on the systemwide database. The mean performance of students in the high-specificity design

340

JOURNAL OF LEARNING DISABILITIES

TABLE 2 Magnitude of Intervention Effects on Pre/Post and Post-Only Measures by Contrast Pre/Post and post-only measures Yopp-Singer M

30/H vs. 15+15/Ha

30/H vs. 15+15/Mb

15+15/H vs. 30/Mc

−0.20

0.29

0.49

OnRF M

0.47

0.36

−0.10

PSF M

0.36

0.59

0.23

NWF M

0.60

0.82*

0.23

1.30*, 1.07*, 0.85*, 0.62*

1.40*, 1.17*, 0.95*, 0.72*

0.10

1.28*, 1.02*, 0.76*

1.28*, 1.02*, 0.76*

.00

0.74*

0.73*

−0.01

2.28*, 1.46*, 0.80*

0.86*

−1.42*, −0.59, 0.06

0.32

0.01

WRMT-R Word Attack M WRMT-R Word Identification M LDF M Spelling M PPVT-R M

0.33

Note. Effect sizes are reported as Cohen’s d based on adjusted prescore/postmean differences. 30/H = 30-min/high-specificity design intervention; 15/H+15 = 15min/high-specificity design and 15-min storybook intervention; 30/M = 30-min/moderate-specificity design intervention; OnRF = Onset Recognition Fluency, from the DIBELS; Yopp-Singer = Yopp-Singer Test of Phoneme Segmentation (Yopp, 1995); PSF = Phonemic Segmentation Fluency, from the DIBELS; NWF = Nonsense Word Fluency, from the DIBELS; WRMT-R = Woodcock Reading Mastery Test–Revised (Woodcock, 1987) Word Attack at LNF = 0, 1, 2, and 3 and Word Identification at LNF = 0, 1, and 2; LDF = Letter Dictation Fluency (based on Berninger et al., 1997); TB Spelling = Spelling test at pre-Spelling Scores 3, 8 and 12 (Tangel & Blachman, 1992, 1995); PPVT-R = Peabody Picture Vocabulary Test–Revised (Dunn & Dunn, 1981). aTime. bSpecificity. cTime and Specificity. *p < .016 level.

group was 38, corresponding to the 67th percentile in a database of 39,000 kindergarten children (Good, Wallin, Simmons, Kame’enui, & Kaminski, 2002). For the WRMT-R Word Attack measure, the slope for the 30/H group was significantly flatter than that for the 30/M group, t(90) = −2.82, p = .006, and marginally significantly flatter than that for the 15/H+15 group, t(90) = −1.74, p = .08, but the 30/M and 15/H+15 slope difference was not statistically significant. To “unpack” the interaction of differential slopes by group, we re-estimated the model, including only the significant interaction term, and conducted adjusted mean comparisons for WRMT-R Word Attack scores among groups at selected

points in the fall LNF distribution. For students who began the year with an LNF score of 3 or below, students in the 30/H group scored significantly higher than students in the other groups (t values ranged from 2.5 to 4.75). For students who began the year with an LNF score between 4 and 6, the groups were not significantly different. As a matter of practical significance, the 30-min/high–design specificity intervention produced very strong effects for students who entered the intervention with the lowest LNF scores, with effect sizes ranging from 0.62 to 1.40, compared to the 15/H+15 and 30/M interventions. For the 15/H+15 and 30/M groups, there was no significant interaction of entry-level LNF score and intervention condition.

For the WRMT-R Word Identification measure, the slope for the 30/H group was significantly flatter than that for the 30/M group, t(90) = −2.29, p = .024, and significantly flatter than that for the 15/H+15 group, t(90) = −2.49, p = .014, but the 30/M and 15/ H+15 slope difference was not significant. We re-estimated the model including only the significant interaction term and conducted adjusted mean comparisons for WRMT-R Word Identification scores among groups at selected points in the fall LNF distribution. For students with fall LNF scores of 2 and below, the 30/H group scored significantly higher than the other two groups (t values ranged from 2.95 to 3.81), and for students with fall LNF scores of 3 to 6, the groups were not

341

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

significantly different. Effect sizes ranged from 0.76 to 1.28 compared to the 15/H+15 and 30/M interventions, respectively. The difference between 15/H+15 and 30/M groups did not depend on fall LNF and was not significant. Letter Dictation Fluency and Spelling. For the letter dictation fluency measure, analyses using the pretest as covariate indicated no differences in post-on-pre regression across groups and a significant pooled poston-pre regression, t(89) = 4.17, p < .001. The 30/H group did better than the 15/H+15 group, t(89) = 2.95, p = .004, ES = .74, and better than the 30/M group, t(89) = 2.70, p = .008, ES = .73. The 15/H+15 vs. 30/M contrast was not significant. For the modified Tangel and Blachman (1992, 1995) spelling measure, ANCOVA results indicated that the 15/H+15 group had a significantly steeper post-on-pre regression than the other two groups, which did not significantly differ on post-on-pre regressions. The respecified model indicated that the 30/H group performed significantly higher than the 30/M group, t(90) = 3.36, p = .001, ES = .86. Students who received the 15/H+15 intervention compared to the 30/H intervention and who scored below the 50th percentile (i.e., < 12 correctly spelled letters in words in 1 min) on the spelling pretest performed significantly lower at posttest. The same group differences for scores of 12 or more on the pretest were not significant. Students who received the 15/H+15 intervention compared to the 30/M intervention and who scored in the 75th percentile or higher performed significantly higher at the posttest. In contrast, for students who scored at the eighth percentile or lower, the 30/M group had higher posttest scores than the 15/H+15 group. Group differences involving the 15/H+15 group are thus complicated and should be interpreted with caution. Residual diagnostics indicate that three students with very low pre- and posttest scores in the

15/H+15 group may have unduly influenced the post-on-pre regression, making it unusually steep and less replicable than the test statistics and p values would indicate. Receptive Vocabulary. Post-onpre regressions were similar across groups, and the pooled post-on-pre regression was highly significant, t(90) = 7.41, p < .001. ANCOVAs conducted on the three contrasts indicated no statistically significant adjusted mean differences among groups.

Treatment Integrity Analysis Integrity data consisted of the average percentage of each interventionist’s compliance with the directions for delivering the components of their respective interventions, as determined by the research staff’s direct observation. Results of total score analyses indicated a relatively high level of treatment integrity for each instructional intervention (30/H = 87.66%; 15/H+15 = 92.45%; 30/M = 90.00%), with no statistically significant differences across groups, χ2(2, N = 178) = 0.78, p = .68. An issue in intervention research is the influence of the interventionist on outcomes. We examined the proportion of total variance due to interventionist and found this factor to be nonsignificant for all outcomes.

Discussion and Instructional Implications Two experimental interventions and one commercial program module were varied systematically to examine their relative effects on the early phonemic awareness, phonemic decoding, word reading, spelling, and letter dictation performance of kindergarten children who performed in the bottom quartile of their cohort. The following discussion of findings is organized by the instructional contrasts of time, design specificity, and the combination of time and design specificity to examine intervention efficacy and efficiency.

Instructional Time The critical window of opportunity in kindergarten to address early reading risks, coupled with the fixed amount of instructional time in a kindergarten school day, calls for interventions that efficiently accelerate learning rates for students to attain reading proficiency on time. “On time” in this study refers to the end of kindergarten, when the foundational skills of advanced phonological awareness (i.e., segmentation) and beginning word reading have been found to be predictive of later academic success (National Research Council, 1998). Prior kindergarten intervention studies have highlighted the importance of instructional time, and the results of these studies seem to suggest that bands of time ranging from 15 to 30 min may produce the most robust effects (Cavanaugh et al., 2004). Nonetheless, no previous kindergarten study has experimentally manipulated instructional time and unpacked the relative benefits of time for specific reading outcomes (e.g., phonemic awareness, word identification) for specific students. To examine instructional time, we compared 15 min of highly specified instruction with 30 min of highly specified instruction. Our results support the following conditional implications: 15 min of highly specified supplemental, small-group instruction that emphasized phonemic awareness and phonemic decoding was • Comparable to 30 min of highly specified instruction in increasing at-risk kindergarten students’ phonemic awareness proficiency in initial sound isolation and phonemic segmentation. • Significantly less effective than 30 min of highly specified instruction in increasing levels of (a) automatic letter retrieval and production for all students in the intervention group; and (b) phonemic decoding, word reading, and spelling proficiency for students who entered kindergarten most at risk based on

342 early alphabetic measures (i.e., letter naming fluency, developmental spelling). On initial sound isolation fluency and untimed and timed phonemic segmentation measures, the 15/H+15 and 30-min/highly specified conditions were not statistically different. In other words, students demonstrated similar growth on phonemic awareness skills with 15 min of phonemic and codeemphasis instruction as with 30 min of instruction with comparable emphases. This finding should be considered in the context of our growing understanding of the importance of developing fluency across early reading skills for later reading proficiency (Torgesen, 2000). However, the results of untimed phonemic decoding, word reading, letter dictation fluency, and spelling fluency analyses revealed statistically and practically meaningful differences between the 30/H and 15/H+15 conditions. On the WRMT-R Word Attack, Word Identification, and spelling measures, analyses revealed that the effects of intervention covaried significantly with the child’s entry-level alphabetic skills. For students who entered kindergarten with the lowest skills (i.e., naming 3 or fewer letter names per minute or spelling fewer than 13 letters correctly in a list of words), 30 min of highly specified intervention produced significantly greater effects than 15 min of highly specified intervention. For students who entered kindergarten with higher alphabetic skills, 30 min of highly specified intervention produced no more reliable differences on word attack or word identification measures than 15 min of highly specified instruction. In other words, for alphabetic skills, the 30-min intervention was differentially more effective for the lowest performing students. The explanation for this pattern of findings is intuitively transparent. The second 15 min of instruction served to reinforce and extend students’ knowledge of and fluency with phoneme– grapheme correspondences both in reading and spelling, and this addi-

JOURNAL OF LEARNING DISABILITIES

tional instruction and practice translated into significant and meaningful differences in student performance, especially for the students most at risk. Relative to this sample, children most at risk, or with entry-level LNF scores of 3 or less, represented 71% of the sample. Therefore, this finding pertains to the majority of students.

Instructional Design Specificity The second focus of this study examined the effect of design specificity, a variable that encompasses the explicitness of instruction, prioritization of tasks, and selection and schedule of information and examples. Prior panel reports and research syntheses (Foorman & Torgesen, 2001; National Reading Panel, 2000) have called for instruction that is explicit, systematic, and scaffolded. Yet few kindergarten studies have compared the benefits of interventions that vary systematically in their level of instructional design specificity. To examine the effect of instructional design specificity, we compared 30 min of highly specified instruction with 30 min of moderately specified instruction. Conditional instructional implications are summarized as follows: 30 min of highly specified design of supplemental, smallgroup instruction that emphasized phonemic awareness and phonemic decoding was • Comparable to 30 min of moderately specified instruction in increasing at-risk kindergarten students’ phonemic awareness proficiency in initial sound isolation and phonemic segmentation. • Significantly more effective than 30 min of moderately specified design of instruction in increasing levels of fluent phonemic decoding, spelling fluency, and automatic retrieval and production of handwritten letters for all at-risk students. • Significantly more effective than 30 min of a moderately specified design of instruction in increasing levels of word attack and word identi-

fication for students who entered kindergarten most at risk based on early alphabetic measures (i.e., letter naming fluency). With the exception of measures of phonemic awareness, these findings support the benefits of a highly specified versus a moderately specified design of instruction for children who have been identified as most at risk in the fall of kindergarten. Specifically, on the NWF DIBELS measure and the letter dictation measure, reliable differences were found favoring a highly specified design of instruction over a moderately specified design of instruction for all students. Similar to the time contrast, a significant interaction emerged between children’s fall alphabetic skills and end-of-kindergarten performance on WRMT-R Word Attack and Word Identification, with a highly specified design of instruction being differentially more effective for students entering kindergarten with the lowest alphabetic skills (i.e., naming 3 or fewer letter names per minute or spelling fewer than 13 letters correctly in a list of words). Again, this finding applies to approximately 70% of students in the study. The results of the design specificity contrast (30/H vs. 30/M) underscore the potential importance of carefully designed instruction for kindergarten children with the lowest entry-level skills. Moreover, the findings indicated that not all explicit instruction is equally effective. As Foorman et al. (1998) hypothesized, instruction that varies in degree of explicitness differentially affects student learning. Specifically, the current findings suggest that highly specified and strategically designed instruction can produce meaningful differences in the alphabetic and orthographic skills of children who perform in the bottom 25% of their kindergarten cohort. We argue that design specificity should be construed along a continuum, and— for children at the greatest risk—the findings suggest that the higher the design specificity, the greater the effect.

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

Naturally, further research is needed to examine the replicability of these findings.

Instructional Time and Design Specificity The final contrast investigated the combined effect of time and design specificity. Our objective in this contrast was to examine whether a highly specified design of instruction implemented for half the instructional time could equal or exceed the effects of 30 min of a moderately specified design of instruction. Of particular interest was whether the instructional efficiencies gained through highly specified instruction could, in effect, “buy time” for other reading skills and strategies, such as vocabulary and comprehension. Trends across phonological, alphabetic, and orthographic findings indicated only two significant end-ofyear differences between the 15/H+15 and 30/M conditions, and both were for the same outcome—spelling. First, students with high entrylevel scores on spelling had better endof-year scores when they received the 15/H+15 intervention rather than the 30/M intervention. Second, students with the lowest entry-level scores on spelling had better end-of-year scores when they received the 30/M intervention rather than the 15/H+15 intervention. However, even these findings may be ambiguous due to the effects of outliers. Given that posttest performance was adjusted using pretest measures, we have confidence that the absence of group differences on the majority of measures was not attributable to differential entry-level performance. This pattern of findings points to the potential of instructional design specificity in maximizing instructional time.

Contributions and Limitations In this study, our primary interest was to unpack the dimensions and archi-

tecture of two instructional variables— instructional time and instructional design specificity. We experimentally manipulated these variables systematically across three interventions and examined the relative effects of each on the early phonemic, reading, and spelling performance of kindergarten students at risk for RD. In summary, our findings are consistent with prior kindergarten intervention research documenting the substantial benefits of kindergarten intervention that targets alphabetic skills for children who are identified as at risk for later RD (O’Connor, 2000; Oudeans, 2003; Schneider et al., 2000). Moreover, our findings serve to extend current knowledge about the importance of instructional time and design of instruction specificity, especially for students who begin kindergarten most at risk. That said, further research is needed to maximally control for curriculum variables that may influence outcomes. Design specificity encompasses many dimensions (e.g., scope and sequence, scaffolding, example selection), and in this study, we used the code component of a commercial curriculum to represent the moderately specified condition. For external validity purposes, the choice of an existing program makes sense. To maximize internal validity, a tighter comparison would hold all dimensions of the curriculum constant except design features. A plausible limitation of the study was the absence of a no-treatment control group. Our decision to treat all students was purposeful, as the trends of prior research provide strong evidence of the benefits of early intervention over typical practice (e.g., Cavanaugh et al., 2004; Torgesen et al., 1999). To some degree, the 30/M intervention served as a comparison group that employed a moderately specified design for phonemic and alphabetic instruction derived from the core reading program delivered in small groups. Yet future researchers may want to compare the efficacy of time and design specificity under practice-as-usual conditions. Future researchers may also

343 want to investigate the relationship between general classroom reading practices and students’ response to supplemental kindergarten reading interventions. The majority of kindergarten classrooms in this study implemented either Open Court or Reading Mastery as the core reading program. The instructional approach of both these programs was well aligned with the supplemental interventions. Treatment effects may be different in schools where classroom reading practices are less aligned with supplemental intervention provided to students at risk for RD. The findings of this study suggest that the effects of instructional time appear to matter most for alphabetic and orthographic skills and for students who are most at risk. Fifteen minutes of a highly specified phonological/ alphabetic design of instruction produced comparable results to 30 min of a highly specified design of instruction on timed and untimed isolation and segmentation tasks. In comparison, most outcomes of early reading and spelling measures that required students to translate letters into sounds, read high-frequency words, or translate speech into print indicated a differential response to instructional time based on initial skill status. Even though all participants were identified as at risk, students who began kindergarten with the lowest levels of alphabetic skills benefited more from 30 min of intervention than students who entered with higher-level skills. A similar pattern of findings emerged through the comparison of a design of instruction that was moderately versus highly specified. Overall main effects were found favoring a highly specified design instruction on the majority of alphabetic and orthographic measures and statistically significant interactions, signifying the importance of carefully designed instruction for students who enter with the lowest letter naming fluency scores. For those outcomes where we did not find significant interactions, the lack of significance may have been due to lower

344 reliability of the outcome measures. Our work that is in preparation suggests that when corrections are made using latent variable models for differential reliability across outcomes, the interaction with initial skill generalizes to all alphabetic outcome measures. In sum, we conclude that both instructional time and the instructional design specificity contribute to significant and meaningful differences in kindergarten performance and matter most on tasks involving alphabetic and orthographic knowledge. For the lowest performing students, deliberate use of both time and design specificity was determined to be even more critical for accelerating learning. For phonemic awareness measures (less advanced or non–fluency based), 15 min of highly specified instruction was sufficient. However, to develop the skills most predictive of later reading proficiency, students who were the furthest behind responded best to 30 min of highly specified design instruction. We must note a possible limitation when interpreting study findings. Early identification of children inherently builds in the risk of false positives (Vaughn & Fuchs, 2003); that is, identifying children as at risk who do not actually require preventive instruction. In this study, we identified children in early fall using measures of letter naming and phonological awareness fluency (National Research Council, 1998; Torgesen et al., 1999). Our decision to identify and intervene early is accompanied by the risk that we may have provided intervention to children for whom it was not necessary. Nonetheless, early identification and intervention deserve further investigation to determine more precise profiles of children who require prevention-based instruction. In addition to providing preliminary guidance on the parameters of instruction, important questions remain. For example, in a 30-min intervention, what portion of time should be dedicated to phonemic, alphabetic, and orthographic emphasis, respectively?

JOURNAL OF LEARNING DISABILITIES

Can we compress 30-min interventions into high-intensity 15-min lessons by conducting further research on the components that are the most active ingredients in the lesson and that require a highly specified design of instruction? In conclusion, this study provides important findings on how to accelerate learning effects and rates in young children. Given a finite amount of time and resources, emerging evidence suggests how specificity in the design of instruction and the differential use of instructional time affect early reading outcomes. In an era of heightened reading expectations and societal demands of all children being able to read by Grade 3 (No Child Left Behind; U.S. Department of Education, 2002), the positive effects of instructional precision provide preliminary evidence on how to enhance the effectiveness and efficiency of prevention efforts (National Reading Panel, 2000). Primary prevention, by design, focuses on the implementation of interventions that reduce risk and obviate the need for further intervention. Findings suggest that optimal early reading growth and prevention of RD requires more than starting early; it requires a diligent commitment to designing instruction and using instructional time strategically to accelerate learning during a defined period of time to close the phonemic, alphabetic, and orthographic differences that children bring to the kindergarten door. ABOUT THE AUTHORS

Deborah C. Simmons, PhD, is a professor of special education in the Department of Educational Psychology at Texas A&M University. Her research interests include early intervention, instructional design, and reading disabilities. Edward J. Kame’enui, PhD, is professor of special education at the College of Education at the University of Oregon. His research focuses on early literacy, schoolwide reading improvement, the design of high-quality educational tools, and design of instruction. Beth Harn, PhD, is a visiting professor and research associate in the Department of Special Educa-

tion at the University of Oregon. Her research interests include early reading instructional design and intervention, linking assessment to intervention, and system-level approaches to academic difficulties. Michael D. Coyne, PhD, is an assistant professor of special education in the Department of Educational Psychology at the Neag School of Education, University of Connecticut. His current research focuses on early reading and vocabulary interventions for young students at risk for reading disabilities. Mike Stoolmiller, PhD, is a research associate at the University of Oregon, College of Education. His research interests include statistics and methodology, the development of reading proficiency, and problem behavior. Lana Edwards Santoro, PhD, is a research associate with the Pacific Institutes for Research. Her research interests include learning disabilities and reading instruction in the areas of spelling, writing, comprehension, and vocabulary. Sylvia B. Smith, PhD, of Pacific Institutes for Research focuses her research on early literacy materials development and review, early literacy and language development, measures development, and schoolwide reading model implementation. Carrie Thomas Beck, PhD, codirects the Oregon Reading First Center at the University of Oregon. Her work focuses on improving the reading outcomes for K–3 students in 50 elementary schools throughout the state of Oregon. Noah K. Kaufman, PhD, is an independently practicing licensed psychologist in Las Cruces, NM, who specializes in neuropsychological and forensic assessment, including the assessment of learning disabilities. Address: Deborah C. Simmons, Texas A&M University, MS 4225, College Station, TX 77843-4225. e-mail: dsimmons@ tamu.edu. AUTHORS’ NOTE

1. Research funding was provided in part by the United States Department of Education, Office of Special Education Programs, Grant H324C980156. The opinions expressed in this article do not necessarily reflect the position of the U.S. Department of Education. 2. We express sincere thanks to the admininstrators, teachers, and students of the Bethel and Springfield school districts for their participation in this research. REFERENCES

Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press.

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

Adams, M. J., Bereiter, C., Brown, A., Campione, J., Carruthers, I., Case, R., et al. (2000). Open court reading. Columbus, OH: SRA. Ball, E. W., & Blachman, B. A. (1991). Does phoneme awareness training in kindergarten make a difference in early word recognition and developmental spelling? Reading Research Quarterly, 26, 49–66. Berliner, D. (1978). Changing academic learning time: Clinical interventions in four classrooms. San Francisco: Far West Laboratory for Educational Research and Development. Berliner, D. (1990). The nature of time in schools. New York: Teachers College Press. Berninger, V. W., Vaughan, K. B., Abbott, R. D., Abbott, S. P., Rogan, L. W., Brooks, A., et al. (1997). Treatment of handwriting problems in beginning writers: Transfer from handwriting to composition. Journal of Educational Psychology, 89, 652–666. Byrne, B., & Fielding-Barnsley, R. (1989). Phonemic awareness and letter knowledge in the child’s acquisition of the alphabetic principle. Journal of Educational Psychology, 81, 313–321. Byrne, B., & Fielding-Barnsley, R. (1990). Acquiring the alphabetic principle: A case for teaching recognition of phoneme identity. Journal of Educational Psychology, 83, 805–812. Byrne, B., & Fielding-Barnsley, R. (1991). Evaluation of a program to teach phonemic awareness to young children. Journal of Educational Psychology, 83, 451–455. Carnine, D. W. (1994). Introduction to the mini-series: Educational tools for diverse learners. School Psychology Review, 23, 341–350. Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64, 723–733. Cavanaugh, C., Kim, A., Wanzek, J., & Vaughn, S. (2004). Kindergarten reading interventions for at-risk students: Twenty years of research. Learning Disabilities: A Contemporary Journal, 2(1), 9–21. Coyne, M. D., Simmons, D. C., Kame’enui, E. J., & Stoolmiller, M. (2004). Teaching vocabulary during shared storybook readings: An examination of differential effects. Exceptionality, 12, 145–162. Dickinson, D. K., & Smith, M. W. (1994). Long-term effects of preschool teachers’ book readings on low-income children’s vocabulary and story comprehension. Reading Research Quarterly, 29, 105–122. Dunn, L., & Dunn, L. (1981). Peabody picture

vocabulary test–Revised. Circle Pines, MN: American Guidance Service. Foorman, B. R., Francis, D. J., Fletcher, J. M., Schatschneider, C., & Mehta, P. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Psychology, 90, 37–55. Foorman, B. R., & Torgesen, J. (2001). Critical elements of classroom and smallgroup instruction promote reading success in all children. Learning Disabilities Research & Practice, 16, 203–212. Francis, D. J., Shaywitz, S. E., Stuebing, K. K., Shaywitz, B. A., & Fletcher, J. M. (1996). Developmental lag versus deficit models of reading disability: A longitudinal, individual growth curves analysis. Journal of Educational Psychology, 88, 3–17. Good, R. H., Wallin, J. U., Simmons, D. C., Kame’enui, E. J., & Kaminski, R. A. (2002). System-wide percentile ranks for DIBELS benchmark assessment (Tech. Rep. No. 9). Eugene: University of Oregon. Gresham, F. M., MacMillan, D. L., BeebeFrankenberger, M. E., & Bocian, K. M. (2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research & Practice, 15, 198–205. Kame’enui, E. J., Carnine, D. W., Dixon, R., Simmons, D. C., & Coyne, M. D. (2002). Effective teaching strategies that accommodate diverse learners (2nd ed.). Upper Saddle River, NJ: Prentice Hall. Kaminski, R. A., & Good, R. H., III. (1996). Toward a technology for assessing basic early literacy skills. School Psychology Review, 25, 215–227. Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Englewood Cliffs, NJ: Prentice Hall. Linan-Thompson, S., & Hickman-Davis, P. (2002). Supplemental reading instruction for students at risk for reading disabilities: Improve reading 30 minutes at a time. Learning Disabilities Research & Practice, 17, 242–251. Lundberg, I., Frost, J., & Petersen, O. P. (1988). Effects of an extensive program for stimulating phonological awareness in preschool children. Reading Research Quarterly, 23, 263–284. Lyon, G. R., & Moats, L. C. (1997). Critical conceptual and methodological considerations in reading intervention research. Journal of Learning Disabilities, 30, 578– 588.

345 Marzano, R. (2000). A new era of school reform: Going where the research takes us. Aurora, CO: Mid-Continent Research for Education and Learning. National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Bethesda, MD: National Institute of Child Health and Human Development. National Research Council. (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press. O’Connor, R. (2000). Increasing the intensity of intervention in kindergarten and first grade. Learning Disabilities Research & Practice, 15, 43–54. O’Connor, R. E., Jenkins, J. R., & Slocum, T. A. (1995). Transfer among phonological tasks in kindergarten: Essential instructional context. Journal of Educational Psychology, 87, 202–217. Oudeans, M. K. (2003). Integration of letter–sound correspondences and phonological awareness skills of blending and segmenting: A pilot study examining the effects of instructional sequence on word reading for kindergarten children with low phonological awareness. Learning Disability Quarterly, 26, 258–280. Schneider, W., Roth, E., & Ennemoser, M. (2000). Training phonological skills and letter knowledge in children at risk for dyslexia: A comparison of three kindergarten intervention programs. Journal of Educational Psychology, 92, 284–295. Senechal, M., & Cornell, E. H. (1993). Vocabulary acquisition through shared reading experiences. Reading Research Quarterly, 28, 360–374. Share, D. L., & Stanovich, K. E. (1995). Cognitive processes in early reading development: Accommodating individual differences into a model of acquisition. Issues in Education: Contributions from Educational Psychology, 1, 1–57. Shaywitz, S. (2003). Overcoming dyslexia: A new and complete science-based program for reading problems at any level. New York: Knopf. Siegel, L. S. (1989). IQ is irrelevant to the definition of learning disabilities. Journal of Learning Disabilities, 22, 469–479. Simmons, D. C., & Kame’enui, E. J. (Eds.). (1998). What reading research tells us about children with diverse learning needs: Bases and basics. Mahwah, NJ: Erlbaum.

346 Smith, P. L., & Ragan, T. L. (1993). Instructional design. New York: Merrill. Smith, S. B., Simmons, D. C., & Kame’enui, E. J. (1998). Phonological awareness: Research bases. In D. C. Simmons & E. J. Kame’enui (Eds.), What reading research tells us about children with diverse learning needs: Bases and basics (pp. 61–127). Mahwah, NJ: Erlbaum. Stage, S., Abbott, R., Jenkins, J., & Berninger, V. (2003). Predicting response to early reading intervention from verbal IQ, reading-related language abilities, attention ratings, and verbal IQ–word reading discrepancy. Journal of Learning Disabilities, 36, 24–33. Stanovich, K. E. (1988). Explaining the differences between the dyslexic and the garden-variety poor reader: The phonological-core variable-difference model. Journal of Learning Disabilities, 21, 590– 612. Tangel, D. M., & Blachman, B. A. (1992). Effect of phoneme awareness instruction on kindergarten children’s invented spelling. Journal of Reading Behavior, 24, 233– 261. Tangel, D. M., & Blachman, B. A. (1995). Effect of phoneme awareness instruction on the invented spelling of first-grade children: A one-year follow-up. Journal of Reading Behavior, 27, 153–185. Tennyson, R., & Christensen, D. L. (1986, April). Memory theory and design of intelligent learning systems. Paper presented at the meeting of the American Educational Research Association, San Francisco, CA. Torgesen, J. K. (2000). Individual differences in response to early interventions

JOURNAL OF LEARNING DISABILITIES

in reading: The lingering problem of treatment resisters. Learning Disabilities Research & Practice, 15, 55–64. Torgesen, J. K., Alexander, A. W., Wagner, R. K., Rashotte, C. A., Voeller, K. K. S., & Conway, T. (2001). Intensive remedial instruction for children with severe reading disabilities: Immediate and long-term outcomes from two instructional approaches. Journal of Learning Disabilities, 34, 33–58, 78. Torgesen, J. K., Wagner, R. K., Rashotte, C. A., Rose, E., Lindamood, P., Conway, T., et al. (1999). Preventing reading failure in young children with phonological processing disabilities: Group and individual responses to instruction. Journal of Educational Psychology, 91, 579–593. U.S. Department of Education, Office of Elementary and Secondary Education (2002). No Child Left Behind. Washington, DC. Vandervelden, M. C., & Siegel, L. S. (1997). Teaching phonological processing skills in early literacy: A developmental approach. Learning Disability Quarterly, 20, 63–81. Vaughn, S., & Fuchs, L. (2003). Redefining learning disabilities as inadequate response to instruction: The promise and potential problems. Learning Disabilities Research & Practice, 13, 137–146. Vellutino, F. R., Scanlon, D. M., Sipay, E. D., Small, S. G., Pratt, A., Chen, R., et al. (1996). Cognitive profiles of difficult-toremediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of

specific reading disability. Journal of Educational Psychology, 88, 601–638. Wagner, R., & Torgesen, J. (1987). The nature of phonological processing and its causal role in the acquisition of reading skills. Psychological Bulletin, 101, 192–212. Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (1994). Development of readingrelated phonological processing abilities: New evidence of bi-directional causality from a latent variable longitudinal study. Developmental Psychology, 30, 73–87. West, J., Denton, K., & Reaney, L. (2001). The kindergarten year (NCES 2001–023). Washington, DC: National Center for Education Statistics. Whitehurst, G. J., Epstein, J. N., Angell, A. L., Payne, A. C., Crone, D. A., & Fischel, J. E. (1994). Outcomes of an emergent literacy intervention in head start. Journal of Educational Psychology, 86, 542– 555. Whitehurst, G. J., Zevenbergen, A. A., Crone, D. A., Schultz, M. D., Velting, O. N., & Fischel, J. E. (1999). Outcomes of an emergent literacy intervention from head start through second grade. Journal of Educational Psychology, 91, 261–272. Woodcock, R. (1987). Woodcock reading mastery test—Revised. Circle Pines, MN: American Guidance Service. Yopp, H. K. (1995). A test for assessing phonemic awareness in young children. The Reading Teacher, 49, 20–28.

VOLUME 40, NUMBER 4, JULY/AUGUST 2007

347

APPENDIX

Activity 2 - Integrated Phonologic/Alphabetic Practice Session 1. Take out teacher 3 square strip and b, r, s, p, l letter squares. 2. Introduce the activity: “Today we are going to practice for Tic-Tac-Toe. We are going to say the sounds in some words slowly and then we are going to choose the letters that go with the first and last sound.” 3. Segment the word: “The first word is rib. Listen, I will say the sounds in rib slowly. /rrriiib/. Say the sounds in rib slowly with me. /rrriiib/. Now you say the sounds in rib slowly.” (First have group say the sounds together. Then have individual students say the sounds.) 4. Isolate the first sound: “I will say the sounds in rib slowly and point to a square as I say each sound. /rrriiib/. (Point to the first square). What is the first sound in rib? That’s right /rrrrr/ is the first sound in rib. 5. Identify the letter that goes with the first sound: (Lay out 2 letter squares—r and another letter) “You’re going to choose the letter that says /rrrrr/ like the /rrrrr/ in rib. (Point to the first square). Everyone think they know? (Call on an individual student to choose the letter square and place it in the first square.) Reinforce the group on the letter name and sound. “Everyone, what’s the name of the letter? And what sound does it make?” 6. Isolate the last sound: “Watch again as I say the sounds in rib slowly and point to a square as I say each sound. /rrriiib/. /b/ is the last sound in rib. (Point to the last square). What is the last sound in rib?” 7. Identify the letter that goes with the last sound: (Lay out 2 letter squares—b and another letter) “You’re going to choose the letter that says /b/ like the /b/ in rib. (Point to the last square). Everyone think they know? (Call on an individual student to choose the letter square and place it in the last square.) Reinforce the group on the letter name and sound. “Everyone, what’s the name of the letter? And what sound does it make?” 8. Practice on 2 to 4 more words as time allows. [Word Bank: beep, seal, bear, pail ] ⊗ If children make an error, model the answer, have children repeat the answer, and return to the sound/letter a second time.