Eric.ed.gov – Hallé SHINE on Manchester: Evaluation Report and Executive Summary

The “Hallé SHINE on Manchester” (HSoM) programme is a Saturday school educational programme designed to increase the reading and maths attainment, as well as engagement with school, of underachieving and disadvantaged pupils at Key Stage 2. Developed in collaboration between the SHINE Trust and Hallé Orchestra, the intervention provides additional school-based literacy and numeracy lessons, based on musical themes, as well as visits to Hallé rehearsals, performances and other theme-based activities. Twenty-five Saturday sessions, each lasting five hours, were planned for the intervention over the course of an academic year, delivered by qualified teachers, teaching assistants, peer mentors, and professional musicians. The evaluation consisted of two randomised controlled trials (RCTs)–a pilot trial and a main trial–and a process evaluation conducted with primary schools in the Manchester area between July 2012 and July 2015. The pilot trial involved 361 Year 5 and 6 pupils in 18 schools; the main trial involved 2,306 Year 4, 5 and 6 pupils in 38 schools. The second year involved both the pilot and main trial running concurrently. The third year involved main trial schools only. Key conclusions include: (1) There is no evidence that the “HSoM” programme had an impact on the reading attainment of children in the trial. This finding was consistent across all age groups, for pupils eligible for free school meals, and for all three years of the trials; (2) There is no evidence that the “HSoM” programme had an impact on the attainment of children in mathematics, or attitudes to reading, maths, music, and school of the children in the trial; (3) Attendance of eligible pupils was often low and considered as a barrier to successful implementation. Reasons for low attendance included pupils’ lack of availability to attend the Saturday sessions, variable parental engagement with the programme, and limited time at the beginning of the programme for schools to engage children and parents; and (4) The process evaluation revealed a positive picture of involvement and engagement for those pupils who attended the Saturday school activities. Evaluators observed good working relationships between the teachers and pupils, and positive and purposeful learning environments in lessons. All stakeholders felt pupils were making noticeable improvements in behaviour, confidence, and development of social skills.