What makes a good performance measure?

Answering questions to guide future work

A good performance measure always starts with a question. Think about the decisions that need to be made in the coming years and what questions should be answered to make those decisions. Where is the gap between what’s currently happening and what we desire to happen? How can it be measured, and what strategies could we implement to address the root cause of the gap? This aligns closely with the strategic planning process; many of the questions will inform your strategic plan, flow from it, or both. Below are some sample questions to start the conversation:

Are we reaching everyone who needs our services?

• Does everyone who needs us know about us?
  • How many people are we reaching with our outreach, and who/where are they? How does that compare to who could benefit from our services?
• Is everyone who needs us eligible?
  • Are our eligibility requirements furthering disparities in who benefits from our work, or helping resolve them?
• Do we have capacity to serve everyone who needs us?
  • Where are the bottlenecks in our work?
  • Which resources could we be using more efficiently to broaden our impact?
• Are there barriers to accessing us, such as physical, time, technology, culture or knowledge constraints?
  • How much effort do people have to put in to reach us?
  • How long do people have to wait to receive our services?
  • Are there disparities in who these barriers impact the most?
  • Are all audiences able to find and use our website, forms, resources to access the resources they need regardless of disability?

Is our service having the impact on lives we intend?

• How does this service relate to the broader social problem we want to solve?
• Does this work overlap, compete or conflict with other policies and programs?
• Are outcomes improving for those we serve? How much?
• Are the impacts focused on the populations where the need/burden is greatest?
• Is our service harming anyone directly or indirectly?
   

Are we doing our work in a sustainable way for the effectiveness of our organization in the future?

• Do our staff and partners have a sense of belonging that allows them to bring their whole selves into our work?
• Are we attracting and retaining the people we need to do our best work?
• Do our staff and partners feel they are able to impact their work?
• Are we using our spending in a way that promotes justice and economic wellbeing in the communities we serve?
  • Are our purchasing and contracting procedures fair to the people we do business with?
    • How quickly are we paying contractors for their work?
    • How much time are we giving businesses to develop proposals when we have opportunities?
    • Are we spending with women- and minority-owned businesses proportionate to the business community?
• Are we meeting our obligations to Washingtonians for transparency and privacy?
• Are we managing organizational risk to protect our customers from foreseeable crises?
   

Characteristics that make performance measures helpful

Below are some of the characteristics that make a helpful performance measure. With limited resources, we must focus our attention on the measures that are most likely to support good decisions.

• Specific measures are well-defined and focused. Having a clear understanding of the “who, what, where, when, which and why” of the question you are trying to answer will help create a measure that is specific enough to truly answer it.
• Relevant measures matter to your audience and clearly relate to the work being measured.
• Actionable measures help connect specific actions or events to strategic goals and are used to make decisions, regardless of results.
• Understandable measures are clear, concise and easy for a non-specialist to comprehend. This applies to the terms we use in titles and descriptions, and technical aspects of the measure such as the scale used in charts or selection of performance targets. Making extra effort at the beginning to define a measure that is understandable will save time, effort, and frustration when we are sharing the measure in the future.
• Timely measures have information available frequently enough to have value in making decisions and assessing accountability. Think about the timing of the decisions you’ll be making – will this measure support decisions for annual or biennial budget requests, quarterly progress meetings on your strategic plan, monthly team meetings, etc.?
• Reliable measures have data that is verifiable, free from bias and an accurate representation of what it is intended to be. Data should be gathered in the same way from person to person, and a standard process for data collection helps ensure that what is gathered and when it’s gathered stays consistent no matter who is obtaining it.
• Cost effective measures are feasible and worth the time and effort to collect, record, display and analyze the data.
• Inclusive measures provide accurate representation for communities by incorporating varying perspectives and ensure we are measuring what matters to all Washingtonians impacted by our work, not just the loudest voices or those with the most power.
• Equitable measures provide the insights we need to act in addressing injustice, inequity or oppression in our work.
• Accessible measures ensure that performance data is accessible and understandable by all members of the public as well as employees.
• Sustainable measures are those that can be measured repeatedly over time. Although it can be very difficult and time consuming to get structured data for certain types of information, improving internal processes in your agency to support obtaining these measurements can be done if deemed important enough, and those improvement efforts themselves can also be measured.

Influence

One other consideration to think about when deciding on a set of performance measures is influence. Influence relates to an agency’s ability to change or affect a measure. In general, the closer we get to the outcome we are trying to achieve, the less influence we have over what we are measuring. As a result, it can be tempting to collect only measures we can directly influence. However, that can lead to incomplete information, short-sighted decisions and losing focus on our ultimate goals.

Tracking a range of measures, from those we have the most influence on to those that measure the outcomes we’re trying to achieve, will give a full picture of the actions we are taking and the results they are producing. We must be clear in how we communicate these measures, so we can set appropriate expectations in advance.

Targets and baselines

Whenever it is feasible, measures should include information about their baseline and trends, including forecasts (predictions based on past and present data) if they are available. Baselines are particularly helpful when we’re discussing measures that relate to an upcoming policy or program change. When we seek funding requests, for instance, showing the expected measure without new funding and what could change with it will support informed conversations about resources and priorities.

Once the change is implemented, we will be able to go back and see if it had the result we expected. The purpose is not to determine whether we are trying hard enough or whether everyone is doing their job. When a target isn’t reached, it leads to crucial conversations about barriers and resources. Did we lack information about how quickly to expect results, did we find a barrier we didn’t anticipate that we’ll need to navigate, or have our priorities shifted since the measures were created? Whatever the answer is, the conversation is critical.

Accurate forecasting is particularly helpful for measures that are difficult to influence. If an indicator is going up and we ultimately need it to go down, first we will need to slow the rate of increase. During the time we are slowing the increase, it can be difficult to communicate progress because the numbers are still going up. But by including a baseline trend from the beginning, we can show that the measure is lower than it would be without our changes, and that we are closer to the result we want. Or, if we don’t see a difference from our baseline, that will tell us the plan needs to change.

Targets can be helpful in some circumstances. For instance, there may be minimum standards that must be met due to legal or funding requirements, or benchmarking based on other states might give us a good idea of what level of service we can fairly expect. If they exist, those standards should be explicit. But adding a target arbitrarily for the sake of having a target provides no benefit, makes honest conversations harder, and erodes trust by creating unrealistic expectations. This is where influence is a key factor: the less influence an agency has over a measure, the less likely a target is to be helpful, and the more likely a baseline trend will be informative.