Note: This WestEd project was completed in 2015.
WestEd conducted a study to assess the impact of the Arts for Learning (A4L) Lessons supplemental literacy curriculum on student achievement.
The three-year evaluation was conducted as part of the Beaverton School District’s (OR) Investing in Innovation (i3) Grant. Young Audiences, Inc., Young Audiences Oregon and Southwest Washington, and the University of Washington were among the core project partners.
Research Design
A three-year, cluster-randomized trial was conducted in 32 elementary schools in grades 3-5 in the Beaverton School District in suburban Beaverton and Portland, OR.
Schools were randomly assigned to treatment or control conditions to examine the impact of the A4L curriculum on student achievement in the English language arts, as well as 21st century skills.
Randomized controlled trial designs provided the most statistically precise estimates of program effects on student achievement because treatment and control conditions are equivalent at the outset of the study, permitting causal inferences to be drawn concerning the intervention.
The evaluation consisted of both formative and summative components (described below), and employs a mixed-methods approach — collecting and analyzing quantitative and qualitative data.
Formative Evaluation
Formative evaluations help structure programs by providing data that better enable program development, improve service delivery, and/or enhance program outcomes.
This evaluation provided timely feedback, describing the implementation of A4L Lessons Units, and measuring how well the project met its key objectives based on project benchmarks.
The formative evaluation included:
- The collection, analysis, and summary of student academic achievement data (annual)
- Classroom observations to assess implementation fidelity (biannual)
- An online A4L Teacher Survey to further examine the factors influencing curriculum implementation, student engagement, classroom management; and explore potential barriers and buttresses to the implementation of arts-integration strategies (annual)
In addition, the A4L Professional Development (PD) Model was assessed using participant surveys and observations/ratings of PD sessions.
The study also incorporated annual interviews conducted with district administration and treatment school principals to gain their perspective on implementation, impacts, systemic change, and sustainability.
Summative Evaluation
The outcome evaluation focused largely on student achievement in the English language arts, with the following research questions guiding our investigation:
- What is the impact of A4L Lessons on English language arts achievement? Does increased dosage yield greater achievement gains?
- Do some subpopulations of students benefit more from A4L Lessons than others (e.g., below grade-level readers, English learners)?
- What factors contribute to the ability of teachers to implement the A4L Lessons curriculum with high fidelity?
- What aspects of the A4L professional development model are critical to effective teaching and learning?