Education Innovation and Research (EIR) Project: Coding as Another Language (2019-2023)

By: Courtney Hagan, PhD

Evaluation Project Manager, Higher Ed

The Project

In partnership with the DevTech Research Group at Boston College, SEG evaluated their Education Innovation and Research (EIR) project ‘Coding as Another Language’ (CAL). The overall aim of this project was to improve computational thinking, coding skills, and language arts comprehension of early elementary school students. To achieve this, a delayed treatment design cluster RCT was implemented through randomization of 20 schools to either a control or treatment group. In the treatment group, schools implemented the Coding as Another Language curriculum using the ScratchJr platform. Teachers were provided with professional development to learn the curriculum and, throughout the school year, delivered up to 24 lessons to their students.

SEG's Role

SEG worked with project staff to design the evaluation plan according to What Works Clearinghouse standards. The evaluation plan was designed to meet standards without reservations through addressing randomization, attrition, and baseline equivalence. SEG worked with DevTech to facilitate the process of randomization and measure attrition and baseline equivalence. 

The Evaluation Plan

The evaluation plan also included an outcome evaluation and implementation evaluation. Throughout the project, data were collected from both students and teachers. Student data supported the outcome evaluation, and student assessments and state test scores were used in a hierarchical linear model to determine whether participating in the CAL curriculum had an impact on computational thinking, coding skills, and language arts comprehension. Teacher data were used to support the implementation evaluation. Teachers were asked to complete surveys and participate in focus groups to determine the threshold for fidelity of implementation.

---

Interested in applying for an Education Innovation and Research (EIR) grant?

If you are interested in applying for the Education Innovation and Research (EIR)

Early-phase grant, and need assistance with your application, contact SEG (seg@shafferevaluation.com). SEG will collaborate with you on research design at the grant application stage at no cost to your institution in exchange for being listed in your application as the external evaluator. Learn more about us here: https://www.shafferevaluation.com/

Please note the deadline for submitting the intent to apply is June 6, 2024, and the deadline for submitting the application is July 22, 2024. The mid-phase and expansion tiers grant competitions have different deadlines. 

Solicitation Calls for the Education Innovation and Research (EIR) Program Early Phase Grant

By: Courtney Hagan, PhD

Evaluation Project Manager, Higher Ed

About the EIR Program Grant

The US Department of Education has issued an invitation for applications to the Education Innovation and Research (EIR) program. The aim of this grant program is to support the “creation, development, implementation, replication, or scalability of evidence-based innovations to improve student achievement and attainment for high-needs students.” For this particular call, the EIR program is interested in supporting projects that “propose services and activities that help students recover from the COVID-19 pandemic, accelerate learning and academic achievement, reimagine schools, and transform our education system.”

Grant Requirements

The EIR grant is broken down into three different phases: early-phase, mid-phase, and expansion. Each phase carries different expectations and requires different levels of evidence to support the application. Early-phase grant submissions need to introduce a new practice based on prior research that suggests the new practice has “promise.” Potential grantees must also submit a logic model that outlines goals, objectives, outcomes, and project components to meet the required evidence standard.

In addition to supporting evidence, early-phase grant applications require the evaluation to be either an experimental or quasi-experimental design. Either design requires meeting What Works Clearinghouse standards, ensuring that a strong research design was implemented, and therefore, allowing conclusions being drawn from how the implementation impacts outcomes. The designs offered in the grant application should provide a statistically significant effect of the intervention on student outcomes.

---

Interested in applying?

If you are interested in applying for the EIR Early-phase grant, and need assistance with your application, contact SEG (seg@shafferevaluation.com). Most recently, SEG collaborated with Boston College on their EIR Early-phase grant, which tested the efficacy of their Coding as Another Language curriculum. SEG will collaborate with you on research design at the grant application stage at no cost to your institution in exchange for being listed in your application as the external evaluator. Learn more about us here: https://www.shafferevaluation.com/

Please note the deadline for submitting the intent to apply is June 6, 2024, and the deadline for submitting the application is July 22, 2024. The mid-phase and expansion tiers have different deadlines. 

Measuring Quality in Afterschool Programs

By: Stacy Ashworth, PhD

Evaluation Project Manager, K–12

Afterschool Programs

Shaffer Evaluation Group recently began working with two Nita M. Lowey 21st Century Community Learning Centers projects in Maryland: one in Harford County Public Schools (HCPS) and one in Prince George’s County Public Schools (PGCPS). As part of both projects, students are provided intervention through afterschool programming. SEG was contracted to provide external evaluation services, which have included determining how to best measure the quality of the afterschool programs.

PGCPS elementary students planting sunflowers in recycled planters during an afterschool program rotation.

Program Quality Assessments

The Forum for Youth Investment’s David P. Weikart Center for Youth Program Quality has developed a series of Program Quality Assessments to measure best practices for ensuring programs are safe, supportive, interactive, and engaging. Tools are available for programs serving children in grades K–6, youth in grades K–12, social-emotional learning, and summer learning. These assessments can be used by program staff as part of a self-assessment process or by a trained, reliable rater, such as Shaffer Evaluation Group. SEG is conducting observations at each project site and completing the assessment to provide feedback to project staff and support continuous improvement.

---

Would you like some support?

Who couldn't use more support? If you are interested in measuring the quality of your afterschool programs and are in need of additional support, contact SEG (seg@shafferevaluation.com). We will work with you to learn more about your project and needs and determine together how we can best support your program's growth. Learn more about us here: https://www.shafferevaluation.com/