Eric.ed.gov – Research Report: Impacts of the Use of Study Island Practice and Benchmarks — Reading School District, Pennsylvania

Study Island is a practice and assessment tool that provides state-standards-aligned opportunities for students to practice their skills. It features a system of continual assessments with immediate feedback to adjust instruction and learning. When educators integrate Study Island into their instructional practices, it acts as a formative, ongoing assessment tool that provides students with a platform to practice or demonstrate their knowledge of taught standards. This approach reflects the elements of formative assessments as a process for monitoring progress and adjusting instruction. Research on formative assessment and progress-monitoring practices has demonstrated positive outcomes for student achievement (Bangert-Drowns, Kulik, & Kulik, 1991; Black & Wiliam, 1998; Fuchs & Fuchs, 1986; January et al., 2018; Stecker, Lembke, & Foegen, 2008; Stiggins, 1999; Van Norman, Nelson, & Parker, 2016; Wolf, 2007). Reading School District (RSD) is a current Study Island partner located 60 miles northwest of Philadelphia and with a total enrollment of over 17,000 students. Ninety-one percent of students in the district are economically disadvantaged, and 83% are Hispanic (Pennsylvania Department of Education, n.d.-b). As a district in Pennsylvania, RSD participates in the state’s accountability system. The Pennsylvania Accountability System (PAS) holds schools and districts accountable to a range of measures, including participation rate, graduation or attendance rate, and closing the achievement gap for all students, especially historically underperforming students. As part of this accountability, the Pennsylvania System of School Assessment (PSSA) is administered annually to students in grades 3 through 8 for English language arts (ELA) and math, as well as grades 4, 8, and 11 for science. Assessment data show that RSD tends to perform at levels lower than the state average. In support of RSD’s partnership with Edmentum, this study is intended to provide a research basis for Study Island in terms of the research literature and analyses of RSD students’ level of usage and performance data within Study Island compared to their performance on the PSSA. Through a series of descriptive and statistical analyses, which include pseudo-controls through Propensity Score Matching (a process to create quasi control and treatment groups of equivalent ability), the findings in this study suggest there are discernable and statistically significant positive impacts on PSSA scores for students participating in Study Island practice and Benchmarks. Generally, implementation and use of Study Island practice and Benchmarks in RSD vary by grade and content area. In practice, students appear to be answering a moderate number of questions and spending a fair amount of time using the product over the course of the school year. Grade 5, in both ELA and math, showed especially high use in the concentration of students using practice and the intensity of their usage. Where students spend more time, answer more questions, and spread their time over active weeks, positive differences are observed. These differences are evident in significantly different math mean scale scores and proficiency classification in grades 5, 6, and 7 when comparing users to non-users and the strongest users to the weakest. Statistically significant results are also found in grade 5 ELA. In addition, when students are exposed to the Benchmarks, which are widely used in grades 3-7 ELA and math, there is a strong and significant association between scores on the Benchmarks and scores on the PSSA. These significant observations remain even after controlling for student ability, based on students’ prior-year PSSA scores. These analyses are clearly impacted by the quality and approach by which schools and teachers use Study Island practice or Benchmarks. Understanding the qualitative differences in implementation approaches, such as for grade 5 students, would be an important next step. Understanding these approaches will help guide implementations that drive evidence-based, positive outcomes for students. [The report was written by Edmentum Research.]