Eric.ed.gov – Learner Response System: Evaluation Report and Executive Summary

This Learner Response System (LRS) intervention involves the use of electronic handheld devices that allow teachers and pupils to provide immediate feedback during lessons. For example, pupils can respond to a question using the device and responses are immediately visible to the teacher, or they can work through problems on the device at their own pace with answers provided as they go. The aim is to improve outcomes by increasing the speed and quality of classroom feedback. A team from Edge Hill University developed the intervention and trained teachers to deliver it to pupils in Years 5 and 6 in primary schools with higher than average proportions of children ever eligible for free school meals. The devices were to be used in at least three lessons a week for between 25 and 32 weeks each year. The intervention ran from June 2014 to June 2016. One cohort of pupils used the LRS for two school years (‘cohort B’), and one cohort for only one school year (‘cohort A’). A cluster randomised controlled trial was used to evaluate the impact of the intervention on Year 6 maths and reading outcomes. Around 6,500 pupils in 97 schools took part. The accompanying process evaluation involved classroom observations, teacher interviews, and focus groups with pupils. The project was funded as part of the EEF Digital Technology funding round in collaboration with the Nominet Trust. Key conclusions include: (1) The project found no evidence that the LRS improved Key Stage 2 results in maths and reading for either cohort; (2) The project found no evidence that the LRS improved the average scores of boys, girls or pupils who have ever been eligible for Free School Meals; (3) Classroom teachers and children were generally positive about the LRS. Teachers welcomed the ability to quickly assess pupil responses for certain types of tasks and give instant feedback. They felt that the LRS helped to engage pupils and allowed different pupils to work at their own pace; (4) Some teachers and pupils felt that the inability of the system to let children ‘show their working’ was a weakness. There was also a concern that incorrectly programmed or overspecified answers meant that pupils occasionally received negative feedback when in fact their answer was wholly or partially correct. There was some criticism from pupils and teachers that the small size of the handset made it difficult to type properly; and (5) The intervention was variably implemented, with a number of teachers not meeting the weekly targets for usage. Reasons given for this included staffing issues, lack of time, and the inability to use the system to its full potential. However, even in schools with high usage, the analysis did not find evidence of an impact.