By: Courtney Hagan, PhD
Evaluation Project Manager, Higher Ed
The Project
In partnership with the DevTech Research Group at Boston College, SEG evaluated their Education Innovation and Research (EIR) project ‘Coding as Another Language’ (CAL). The overall aim of this project was to improve computational thinking, coding skills, and language arts comprehension of early elementary school students. To achieve this, a delayed treatment design cluster RCT was implemented through randomization of 20 schools to either a control or treatment group. In the treatment group, schools implemented the Coding as Another Language curriculum using the ScratchJr platform. Teachers were provided with professional development to learn the curriculum and, throughout the school year, delivered up to 24 lessons to their students.
SEG's Role
SEG worked with project staff to design the evaluation plan according to What Works Clearinghouse standards. The evaluation plan was designed to meet standards without reservations through addressing randomization, attrition, and baseline equivalence. SEG worked with DevTech to facilitate the process of randomization and measure attrition and baseline equivalence.
The Evaluation Plan
The evaluation plan also included an outcome evaluation and implementation evaluation. Throughout the project, data were collected from both students and teachers. Student data supported the outcome evaluation, and student assessments and state test scores were used in a hierarchical linear model to determine whether participating in the CAL curriculum had an impact on computational thinking, coding skills, and language arts comprehension. Teacher data were used to support the implementation evaluation. Teachers were asked to complete surveys and participate in focus groups to determine the threshold for fidelity of implementation.
Interested in applying for an Education Innovation and Research (EIR) grant?
If you are interested in applying for the Education Innovation and Research (EIR)
Early-phase grant, and need assistance with your application, contact SEG (seg@shafferevaluation.com). SEG will collaborate with you on research design at the grant application stage at no cost to your institution in exchange for being listed in your application as the external evaluator. Learn more about us here: https://www.shafferevaluation.com/
Comments