Evidence Generation — Process and Outcome Measures

evgenmeasurements

The Evidence Generation initiative assists affiliated agencies with measurement challenges. The EvGen measurement matrix identifies gaps in each agency’s data collection efforts and encourages them to distinguish between process and outcome measures. Both types of measurement, of course, are important for evaluation research.

Process Measures are used to monitor program implementation. They are often collected from the moment of program entry, while programming is underway, and at program completion. Process measures are essential to document who is being served and what services they are receiving. If a program is not being implemented as intended it cannot be expected to affect later measures of outcomes. Process measures monitor the amount and quality of a specific type of activity and output. The most common type of process measure is a counting system that keeps track of how much of something is being administered. For example, if an organization attempts to increase school engagement by doing school visits, recording the number of school visits can be one measure of program delivery. Other types of process measures are more complex because they require more than a tallying system. Examples are the characteristics, behaviors, attitudes, opinions, and beliefs of individuals enrolled in a program.

Outcome Measures are used to assess the short and long-term results of a program’s activity or set of activities. Outcome measures examine whether participants are improving as a result of the program or due to other outside factors. In the justice field, the most traditional process measures are often negative outcomes, including recidivism. Process measures can also be positive, such as program completion, school graduation, employment, etc. The EvGen initiative fills out the measurement matrix in collaboration with staff from each affiliated agency based on our understanding of the agency’s goals and objectives as represented by the theory of change and logic model.

One useful way to distinguish process and outcome measures is to consider the basic questions each tries to answer (paraphrasing Peter Drucker):

  • Outcome measures are used to answer the question:
    “Are we doing the right things?”
  • Process measures are used to answer the question:
    “Are we doing things right?”

Most people are initially interested in the first question, which is another way of asking whether programs and practices are having their intended effects. But that question cannot be answered in full without knowing the answer to the second question — are programs and practices being delivered as intended? The EvGen measurement matrix reviews each affiliated agency’s ability to generate both outcome and process measures.

Developing a Measurement Plan