Valuable Lessons in Evaluation for Educational Institutions

Stacy Ashworth, Ph.D., Evaluation Project Manager

Engaging Stakeholders Early

One of the most valuable steps in any evaluation is collaborating with stakeholders who are directly involved in program implementation or data collection. While project directors often oversee planning, they may not be the ones executing the work or gathering data. By involving them early, project directors and evaluators gain insight into what’s feasible and what may need adjustment.

These leaders, embedded in the daily operations of their schools or departments, can identify who has the capacity to support implementation and data efforts—insights that are often missed without their input. Early engagement fosters a sense of ownership and accountability among stakeholders, which can enhance the evaluation process.

Being Strategic with Data Collection

More data isn’t always better. One of the most important lessons in evaluation is the value of being intentional about what data to collect—and why. Project directors should focus on gathering data that directly supports understanding of implementation progress. It’s essential to be mindful of the burden placed on both data collectors and respondents, such as survey participants.

A helpful starting point is to review what data are already being collected. Leveraging existing sources can reduce duplication, streamline efforts, and ensure that new data collection is purposeful and manageable. This strategic approach not only saves time but also enhances the quality of the evaluation.

Prioritizing Ongoing Communication

Consistent communication is essential for a successful evaluation. Project directors should regularly connect with those responsible for implementation to understand how things are progressing. This helps identify emerging needs and ensures staff have the time and resources to carry out their roles effectively.

These check-ins—whether through recurring meetings or informal in-person visits—build trust and foster a collaborative environment. Equally important is maintaining open lines of communication between the evaluator and the project director. Regular meetings not only provide valuable context about implementation but also create space for reflection, adaptation, and strategic decision-making.

Building Relationships for Effective Evaluation

Evaluation is as much about relationships and strategy as it is about data. Across diverse projects and grant programs, three lessons consistently stand out: the importance of stakeholder collaboration, the need for intentional and manageable data collection, and the value of ongoing communication.

When these elements are prioritized, evaluations become more responsive, grounded, and impactful—ultimately supporting programs in achieving their goals more effectively. A strong relationship between evaluators and stakeholders can lead to richer insights and a more nuanced understanding of program dynamics.

Conclusion: The Path Forward

As we continue to navigate the complexities of evaluation, these lessons remain at the forefront of our practice. By engaging stakeholders early, being strategic with data collection, and prioritizing ongoing communication, we can enhance the effectiveness of our evaluations.

This approach not only benefits the evaluation process but also strengthens the programs we aim to support. Together, we can make a bigger difference for underserved communities both in the US and globally.

Interested in working with Shaffer Evaluation Group? Contact us today for a free 30-minute consultation: seg@shafferevaluation.com.

Sustainability Planning for Federal Grants: From Closeout to Continuity

Patricia Moore Shaffer, Ph.D., Principal & Lead Evaluator

Sustainability planning for federal grants is a disciplined process. It ensures a project’s goals, principles, and activities endure after the award period ends. Federal funders increasingly prioritize long-term impact. Proposals and performance reports should demonstrate a credible plan for lasting success. In this article, I will share guidance on sustainability planning to support your federal grant.

Start Early

Treat sustainability as a design constraint from day one. Don’t wait until the final quarter to address it. Build time, roles, and budget for sustainability work into your management plan. Revisit this plan at every reporting cycle to ensure you stay on track.

Use a Sustainability Assessment

At Shaffer Evaluation Group, we guide clients using the Program Sustainability Assessment Tool (PSAT). The PSAT helps teams score and prioritize capacity across eight domains: Environmental Support, Funding Stability, Partnerships, Organizational Capacity, Program Evaluation, Program Adaptation, Communications, and Strategic Planning. Conducting a quick PSAT baseline early in Year 1, a midpoint check, and a final assessment creates a clear trajectory of capacity-building for your organization. This also provides evidence of progress for funders.

Build a Sustainability Plan

Your sustainability plan should be developed early in your grant term. Here are the core components a sustainability plan for a federally funded project should include, distilled from cross-agency best practices:

• Purpose & Outcomes: A concise statement of your project's value proposition, intended outcomes, and beneficiaries, aligned with a logic model you will continue to monitor and adjust post-award.

• Core Program Model & Operating Protocols: A description of your core program model and what must be maintained (including workflows) so the project can persist with quality after grant closeout.

• Governance & Ownership: Named post-award owner(s) and ongoing “champions” to anchor the work.

• Funding Strategy (Braided Financing): A targeted mix of reallocated (internal) funding and external sources, with milestones, responsible leads, and cultivation plans (e.g., philanthropy, fee-for-service).

• Partnerships & MOUs: Roles, cost-shares, and data-sharing agreements with agencies, nonprofits, and community partners that are essential to sustain activities.

• Workforce Continuity & Knowledge Transfer: Staffing map, cross-training to avoid single points of failure, onboarding materials, and a schedule for refresher trainings.

• Communications & Stakeholder Engagement: A plan to keep sponsors, end-users, and other stakeholders informed and invested.

• Data, Evaluation & Learning: Post-grant metrics, feedback loops, and routines for using evidence in decisions.

• Policy & Institutionalization: The specific policies, procedures, and standards that will embed the work into normal operations.

• Risk & Scenario Planning: Anticipated risks (funding gaps, turnover, vendor change), trigger points, contingencies, and “minimum viable operations” to protect continuity.

• Technology & Infrastructure: Post-award requirements for platforms, equipment, licensing, accessibility/privacy, and support so services can run at a steady state.

  
  

Operationalize It

Translate the plan into a 12–24 month roadmap. Include owners, milestones, and a lightweight dashboard. Budget explicitly for sustainability tasks, such as grant writing time, partnership stewardship, and training refreshers.

How We Help

Shaffer Evaluation Group uses the PSAT to run a rapid diagnostic. We co-facilitate strategy sessions with leadership and partners. Together, we co-develop a sustainability roadmap that balances quick wins (policy changes, SOPs, cross-training) with longer-horizon plays (braided funding, institutionalization). A thoughtful sustainability plan signals stewardship, strengthens proposals, and, most importantly, keeps effective work alive for the communities it was designed to serve.

For more information, contact us today at seg@shafferevaluation.com.

Key Considerations for Evaluating Career and Technical Education (CTE) Programs

Stacy Ashworth, Ph.D., Evaluation Project Manager

Shaffer Evaluation Group partners with a wide range of Career and Technical Education (CTE) programs supported by various grant initiatives, including the Department of Defense Education Activity (DoDEA), the Native American Career and Technical Education Program (NACTEP), and the National Science Foundation (NSF). Our goal is to be the go-to partner for educational organizations, helping them secure funding, improve their programs with solid evidence, and ultimately make a bigger difference for underserved communities both in the US and globally.

Designing an Effective Evaluation Strategy

When designing an evaluation strategy for CTE programs, consider the following two key factors:

Capture the Full Range of Program Impact

Outcome measures should reflect both short-term achievements and long-term impacts. Traditional metrics like completion rates, graduation rates, and post-program plans are essential, but they only tell part of the story. To gain a deeper understanding of program effectiveness, collect data throughout the program’s duration. This might include:

• Surveys that track changes in participants’ beliefs, attitudes, and dispositions.

• Measures of academic performance and engagement.

• Insights into future educational or career plans.

• Enrollment in additional training, educational programs, or courses.

  
  

These broader indicators provide a more complete picture of how the program shapes participants’ trajectories. By focusing on these aspects, we can better understand the transformative effects of CTE programs.

Account for Attrition in Your Evaluation Design

Attrition is a reality in many CTE programs, so it’s crucial to build evaluation measures that reflect progress, even when participants don’t complete the full program. Consider tracking the following:

• Milestones toward program completion.

• Course-level completion rates (e.g., percentage of students who finish individual courses).

• Endorsement or certification attainment.

  
  

These metrics can demonstrate early wins and incremental progress, offering valuable insights into program effectiveness even when full completion isn’t achieved. Understanding these dynamics allows us to refine our approaches and enhance program delivery.

The Importance of Continuous Improvement

In the realm of CTE, continuous improvement is essential. Evaluators should not only focus on what works but also on what can be improved. Regularly revisiting evaluation strategies ensures that they remain relevant and effective. This iterative process allows programs to adapt to changing needs and contexts.

Engaging Stakeholders in the Evaluation Process

Engaging stakeholders is vital for a successful evaluation. Involve educators, students, and community members in discussions about evaluation goals and methods. Their insights can provide valuable context and help tailor evaluations to meet specific needs. This collaborative approach fosters a sense of ownership and commitment to the evaluation process.

Utilizing Technology for Enhanced Data Collection

Advancements in technology offer new opportunities for data collection and analysis. Online surveys, mobile applications, and data management systems can streamline the evaluation process. By leveraging these tools, we can gather more comprehensive data and analyze it more efficiently. This not only saves time but also enhances the quality of insights derived from the evaluation.

Final Thoughts about Evaluating Career and Technical Education Programs

Effective evaluation of CTE programs requires a thoughtful and flexible approach that captures both the breadth and depth of program impact. By incorporating diverse outcome measures and planning for participant attrition, evaluators can build a more accurate and resilient picture of program success. Whether you're working with DoDEA, NACTEP, NSF, or other grant-funded initiatives, these strategies help ensure your evaluation reflects the real-world progress and potential of your CTE efforts.

Shaffer Evaluation Group is committed to helping programs tell their story through meaningful data. With the right tools and insights, your evaluation can do more than meet reporting requirements—it can illuminate the transformative power of CTE. Contact us today for a free 30-minute consultation.