Skip over global navigation links
U.S. Department of Health and Human Services

Importance of Evaluation to the HPP Program

What is evaluation?

 

Evaluation is the process of collecting, analyzing and ultimately using data to raise awareness, provide surveillance, or make decisions. Program evaluation entails collecting and analyzing data to make decisions about a program or aspects of a program. Program evaluation staff systematically collect and analyze data to determine how well a program is working and why the program is performing at its current levels.

 

Why do we conduct evaluations?

 

There are two primary reasons to conduct evaluations:  to demonstrate accountability to stakeholders (including funders), and to facilitate internal program improvement (also referred to as organizational learning).

 

The U.S. Congress, federal oversight agencies, state and local legislatures, and taxpayers alike expect to know the concrete results of HPP/ASPR investments and whether the nation is better prepared to respond to health and medical emergencies.  Should available HPP funds continue to decrease, the need to articulate HPP successes and impacts will become more urgent. With the data gathered through program evaluation, HCCs can respond to requests for information from various stakeholders and provide evidence that ASPR is using HPP investments as intended to achieve desired outcomes.

 

Improving program performance is just as important as accountability. Program evaluation can help monitor progress and establish benchmarks in key areas which allow awardees and HCCs to assess measure improvement over time. Evaluation that seeks to improve program performance tends to focus on collecting data that can teach organizations about their strengths, weaknesses, and critical chokepoints impeding optimal results.

 

To evaluate a program, it is helpful to understand the connections between program resources, activities, and goals. Logic modeling is one way to display these connections. Logic models identify and propose relationships between and among program resources, activities, outputs, and outcomes. Figure 1 provides a sample logic model, followed by definitions of its components.

 

Sample Logic Model

The sample logic model is divided into six boxes with double sided arrows on each side:  inputs, activities, outputs, short term outcomes, intermediate outcomes, long term outcomes
 
 

Definitions of Logic Model Components

 
Component Definition
​Inputs ​Resources that are required to support the program, including staff and volunteers, funding, facilities, and equipment.
​Activities Actions that use or involve program inputs.
​Outputs ​Products and services produced by program activities.
​Outcomes ​Changes or benefits resulting from program activities and outputs. Outcomes can be intended or unintended, positive or negative, and are often divided into short term, intermediate, and long-term timeframes.
 

How are data used?

Measurement data can facilitate internal program improvement and demonstrate accountability.

Improvement measures are designed to provide data to Awardees, HCCs, and ASPR staff to identify strengths, weaknesses, and areas of improvement, along with opportunities for training and technical assistance. ASPR will use measurement data to facilitate internal program improvement and learning. Most HPP measures have an improvement component.

Accountability measures are collected in compliance with specific federal requirements, statutes or initiatives such as the Pandemic and All-Hazards Preparedness Act (PAHPA), the Government Performance and Results Act (GPRA), and the Healthy People 2020 Initiative. Program staff often report data from these measures to requesting agencies and other entities such as the U.S. Department of Health and Human Services, the White House Office of Management and Budget (OMB), and others. ASPR will use data from accountability measures to provide evidence to the federal entities that the HPP Awardees adhere to funding requirements and demonstrate effectiveness in public health preparedness.

How are data verified?

HPP is currently developing a Site-Visit Tool that Field Project Officers (FPOs) will use to systematically verify healthcare coalition development, capability adherence, and program measure completion.  The HPP Site‐Visit Tool will provide:

  • Guidance to awardees for planning, training, exercise, and resource development
  • Methods to monitor continuous Awardee progress from year to year
  • A means to validate the reported BP2 Program Measure indicators and HCCDA factors

The Site-Visit Tool will assist both the HPP FPO and Awardee to identify needs, prioritize resources, and track progress over time.

For more information on this and other HPP Program Evaluation topics, please refer to the HPP Measure Manual: Implementation Guidance for the HPP Program Measures.  SHARPER welcomes your feedback. Please submit your questions and comments via the email at SHARPER@hhs.gov.

SHARPER: Science, Healthcare, Preparedness, Evaluation and Research

  • This page last reviewed: June 13, 2016