Beyond the Numbers: Storytelling's Place in Holistic Evaluation

When most people think about evaluation, they think about numbers. Evaluators track performance indicators, compare benchmarks, and analyze changes over time. We measure outcomes, calculate percentages of improvement, and assess whether programs are meeting their goals. These metrics are important, but they rarely tell the whole story.

Complex programs, especially those in education, the arts, community development, and other human-centered fields, require a broader lens. Program outcomes are rarely straightforward. They unfold across multiple levels and often extend far beyond what can be captured through assessments, surveys, or Likert-scale data alone. To understand the full scope of a program's impact, evaluators must engage in ongoing conversation with project teams, stakeholders, and participants. A holistic evaluation requires more than measuring change. It requires understanding the experiences, relationships, and contexts that shape that change.

This is where storytelling becomes an important part of the evaluation process.

Storytelling as Evidence

Every participant enters a program with a unique set of experiences, goals, perspectives, strengths, and challenges. Those differences shape how individuals engage with a program and how they experience its benefits. At the same time, implementation is influenced by changing circumstances, organizational priorities, community needs, and countless other factors that cannot always be reduced to a number.

As evaluators, our responsibility is not only to determine whether a program worked, but also to understand how it worked, for whom it worked, and under what conditions. Quantitative data can tell us that change occurred. Stories help us understand why.

Consider a student who persists through a difficult semester because a success coach helped them navigate financial aid, connect with campus resources, and build confidence in their academic abilities. The student's persistence may appear in retention data, but the pathway that led to that outcome remains largely invisible without qualitative inquiry. Interviews, observations, focus groups, and participant reflections provide critical context for understanding how and why outcomes occur.

Stories do not simply add color to evaluation findings. They provide evidence about participant experiences, implementation processes, and program impacts that may otherwise go unnoticed. They help evaluators understand both individual experiences and broader patterns across a project.

At Shaffer Evaluation Group, we have increasingly incorporated storytelling into our work as we evaluate complex and multifaceted initiatives. One current national arts program, for example, defines success in ways that are highly individualized and difficult to quantify through traditional measures alone. To capture the full scope of the program's impact, we are drawing upon observations, interviews, participant testimony, images, and narrative accounts alongside more traditional data sources. Together, these methods help create a richer picture of outcomes that range from personal growth and artistic exploration to community engagement and nationwide network-building.

Humanizing Data

Storytelling also plays an important role in how evaluation findings are shared and used.

Program leaders, funders, and community stakeholders often rely on evaluation results to make important decisions. While charts, dashboards, and summary statistics are valuable tools, stories can make findings more accessible and meaningful.

A participant quote can bring a trend to life. A narrative describing implementation challenges can provide context for unexpected outcomes. Stories help stakeholders connect data to lived experiences and understand the significance behind the numbers.

This does not mean replacing evidence with anecdotes. Rather, it means recognizing that meaningful evidence can take many forms. When quantitative and qualitative findings are considered together, they provide a more complete understanding of a program's effectiveness and impact.

Program success is, in many ways, deeply human. Behind every outcome is a collection of experiences, relationships, decisions, and circumstances that influence how change occurs. By integrating storytelling into evaluation practice, evaluators can illuminate pathways of change, elevate participant voices, and capture impacts that might otherwise remain unseen. The result is an evaluation that not only measures outcomes but also helps tell the full story behind them.

If your institution is preparing to implement a nuanced, human-centered project and are looking for an evaluation partner to understand and capture complex impacts, please reach out to us at seg@shafferevaluation.com.

Career Services: From Career Center to Career Pathways Infrastructure

In higher education, Career Services is often treated as a place students go when they are ready to write a resume, search for internships, or prepare for interviews. However, that model is too late, too optional, and too disconnected from how students actually make academic and career decisions. Students begin forming ideas about work, income, identity, and opportunity long before they reach graduation. If Career Services only appears at the end of that process, institutions miss one of the strongest opportunities to improve student persistence, completion, and employment outcomes.

In a recent study we conducted with a large community college serving an urban population, students described career decision-making as a complex process shaped by interests, academic strengths, prior work experience, family expectations, job availability, earning potential, and cultural background. Survey results reinforced this complexity: students rated their own strengths and academic abilities as highly influential, but they also placed strong weight on job availability and earning potential. In other words, students were not choosing programs in a vacuum. They were trying to make sense of who they were, what they could afford, what their families expected, and what the labor market might offer.

The same study also showed that career information reaches students through multiple channels. Students did not rely only on formal Career Services. They turned to faculty, advisors, college websites, family members, peers, and prior work experiences. Faculty were especially important because students often saw them as credible contacts: people who could explain what a job actually involved, what employers expected, what coursework mattered, and what pathways were realistic. If Career Services is confined to one office, it cannot fully support the way students actually navigate the career decision-making process.

A more promising model is to treat Career Services as institutional infrastructure. In this model, central career planning and advising services remain important, but they are not the only way career development happens. Instead, career planning should be embedded across the student journey through predictable touchpoints with Career Services, faculty and staff referrals, employment-based learning opportunities, and student support services. Students should encounter career development repeatedly, not only when they are preparing to leave.

This is where a Career Services journey model becomes useful. A journey model defines how students move from early exploration to informed decision-making, employer-connected learning, skill development, and employment or continued education. It asks: What should students understand in their first year? When should they connect their academic program planning to labor-market information? How and when will they encounter internships, work-based learning, or applied projects? Who helps them interpret employer expectations? How are career conversations documented and followed up? The point is not to add more disconnected services. The point is to make career development a routine part of academic progression.

The model also requires a broader view of who provides career guidance. Faculty, advisors, coaches, program staff, and student support personnel often serve as the first point of contact when students ask whether a program is worth the investment, whether a career is realistic, or what they can do with a credential. Institutions should not expect these staff to become career counselors, but they can equip them with career road maps, referral protocols, labor-market summaries, and clear guidance on when to connect students with Career Services. This turns informal advice into a more consistent institutional system.

The journal approach to Career Services is especially important for applied, technical, and workforce-oriented programs. Students in these pathways may need specialized advising that connects program requirements, credentials, labor-market information, employer expectations, work-based learning, and next-step employment or continued education. Traditional advising models can overemphasize transfer pathways and underemphasize immediate workforce goals. Specialized technical/workforce-oriented career advising protocols can close that gap by creating common intake, referral, advising, and follow-up procedures providing students with the Career Services they need for success.

Employer partnerships are another essential part of Career Services infrastructure. In a stronger model, employers are not engaged only at the point of hiring. They help students understand fields of work, clarify skill expectations, test career interests, build professional networks, and see how academic programs connect to real jobs. Employers can serve as mentors, classroom speakers, mock interviewers, project sponsors, internship and apprenticeship hosts, advisory committee members, curriculum informants, and partners in identifying emerging skills. They can also help faculty and Career Services staff interpret labor-market changes in practical terms: what entry-level roles look like, which credentials matter, what experiences strengthen employability, and how students can move from a first job to career advancement.

The larger lesson is that Career Services should not be viewed as a peripheral student support. It is a student success strategy, a workforce strategy, and an institutional capacity strategy. When Career Services is embedded across the student journey, students are more likely to see the purpose of their academic choices. When faculty and staff have tools to support career conversations, students receive more consistent guidance. When employers are engaged with faculty, staff, and students, more students gain access to sector expertise and applied learning and employment opportunities. When labor-market information is translated into advising, students can make better-informed decisions.

Evaluating for Sustainability: Evaluation of Title III Strengthening Institutions Grants

Many colleges and universities approach Title III grants as a way to launch new programs, improve student services, or modernize campus systems. Those goals matter. But the strongest Title III projects do something larger: they help institutions build lasting capacity.

That is why evaluation should be viewed as more than a reporting requirement. A well-designed evaluation helps institutions determine whether project changes are actually becoming part of the institution’s long-term operations.

In our work evaluating Title III grants, one issue consistently emerges: sustainability cannot be treated as a final-year discussion. Institutions that wait until Year 4 or Year 5 to think about sustainability are often too late.

The most successful projects begin planning for institutionalization at the start of the grant.

This distinction matters because many Title III initiatives begin as “special projects.” They may depend on temporary staff positions, grant-funded technology, or a small group of highly committed faculty and administrators. Early implementation may look successful, but if the work is not integrated into institutional structures, it can disappear once funding ends.

We have seen this challenge across many types of Title III activities, including advising reform, developmental education redesign, online learning expansion, student success initiatives, and workforce-focused program development.

For example, a college may hire grant-funded advisors who significantly improve student support services. But the long-term question is whether the institution eventually adopts those positions within its regular operating budget and advising structure. Similarly, faculty may redesign courses or pilot new instructional practices during the grant period, but those improvements are far more likely to continue when they become embedded into ongoing faculty development systems and academic planning processes. There is where evaluation becomes important.

Strong Title III evaluation goes beyond tracking outputs such as the number of workshops held, students served, or courses redesigned. Those metrics are necessary, but they are not enough. Evaluation should also examine whether the institution is building the systems, leadership support, policies, and internal ownership needed to sustain the work over time.

Federal guidance and emerging higher education evaluation literature increasingly emphasize this broader view of institutional change. Current discussions around sustainability in higher education focus on issues such as organizational readiness, cross-department collaboration, leadership alignment, continuous improvement, and institutional ownership of innovation.

In practice, this means evaluators should pay close attention to institutional context. Leadership turnover, staffing shortages, enrollment pressures, and reorganizations can all affect implementation. Even strong projects can lose momentum if they are not connected to institutional priorities and systems.

This is one reason qualitative methods are so valuable in Title III evaluation. Interviews, focus groups, implementation reviews, and sustainability assessments often reveal important issues before they appear in performance data. Faculty and staff may identify gaps in communication, uncertainty about long-term funding, or concerns about workload and institutional support. These conversations can help institutions make mid-course corrections while there is still time to strengthen the project.

Another important lesson is that sustainability is rarely tied to a single activity. Instead, it develops through repeated integration into institutional practice. Projects become sustainable when faculty, staff, and administrators begin to view them not as grant activities, but as part of how the institution operates.

Ultimately, the best Title III evaluations do more than document compliance. They help colleges and universities understand whether they are creating meaningful institutional change. When evaluation is approached this way, it becomes a practical tool for decision-making, strategic planning, and long-term improvement and not simply an annual reporting exercise.

Shaffer Evaluation Group has conducted project evaluations of Title III Strengthening Institutions grants since 2014. Our team has also provided grant writing support for Title III grant applications. If your institution is applying for a Title III grant and looking for an evaluation partner, please reach out to us at seg@shafferevaluation.com.