Assessing status - How is your population doing on the most important of these measures?

The product of this step is an assessment tool.  You can use it to support arguments for taking action.  It can also offer a place to start when looking for chances to promote real change.  In population accountability terms, success means a better result than was predicted using a baseline.

Section contents:

• Graph the indicator baseline
• Tell the story behind the baseline for each indicator
• Calculate the cost of bad results

Graph the indicator baseline and forecast:

A baseline is a multi-year display of data with two parts: (a) an historical part which shows where the community has been, and (b) a forecast part that shows where the community will be if nothing canges.

The historical part of the baseline is fairly straight-forward, while the forecast is difficult.  The forecast is the more important part.

If you have a forecast, you can use your baseline to:

1. Ask, is this forecast acceptable?
2. Define success as doing better than the forecast.

Forecasts should reflect the consensus view of key partners about where the population is heading.

Tips

1. The most important part of forecasts is that they are credible.
2. Do not manipulate forecasts to set up an easy definition of success or overstate the importance of a proposed program or change in policy
3. Don't let statistical experts dominate the process.

Tips:

1. The most important part of forecasts is that they are credible.
2. Do not manipulate forecasts to set up an easy definition of success or to overstate the importance of a proposed program or change in policy.
3. Expert advice is very helpful for the technical work of forecasting; you may find this kind of help from university or state program staff.  Although a forecast must reflect historic trends, there are other influences too.  Your partners may have key information that will help your forecast be more accurate.

Baseline Example: No leisure exercise

This graph was created using AK BRFSS data for the years 2000 to 2006, open cells for the years 2007 to 2010, and the add trendline option in Excel, which is a right click option if a data line is selected.  There are five types of trendlines available, and a linear trend is used here.  Excel has another function, FORECAST, which might also be helpful.

If everything continued relatively unchanged, this trend suggests that about one-fourth of Alaska’s women would report that they did not exercise in 2010.  The Population Accountability question is whether that result is acceptable.

Forecasts are rooted in the past, and the period of time used can greatly change the prediction.  For instance, if only the years 2001 to 2003 had been used, the trendline would have a downward direction.  If the time period was limited to 2005 and 2006, it would have been flat.

Tell the story behind the baseline for each indicator:

Include in these stories all of the influences identified when you develop your forecast.  These are the indicator’s causes and the forces that affect its status.

Here is a series of questions that might be helpful in thinking about what a forecast means if the community does not take action:

• Will the trend continue in the same direction?  WHY?
• Will the trend continue in its current direction faster, slower or about the same?  WHY?
• Will the trend flatten out?  When will it flatten out and at what level?  WHY?
• Will the trend change direction?  When?  What will happen after?  WHY?

If not already identified, factors that might influence a forecast include changes in:

• Your target population (Is it getting older or younger?  Is the gender balance changing?  Is the racial mix changing?), or
• The economy (Is the balance betwen the subsistence and cash economies shifting?  Is the amount of disposable income changing?)

The behind-the-baselines stories need to be short, focused, and easy to understand.  Most indicators have many information resources that you could use for background and ideas.  Start an information and research agenda if you find a gap in the information you need for a story.

As you work on your behind-the-baselines stories, you might talk with as many people with different points of view and as many parts of your community as you can.  You might consider many behind-the-baselines opinions and ideas, and not everyone need to agree.  You might find people from your target population particularly helpful, as they often know a lot about the factors that influence your baselines and actions that would help you succeed.

If you identify a behind-the-baselines cause that seems important but lacks enough information, you could add the need for information to your information and research agenda.  This is a list of projects that could help your population accountability process move forward.  It is a method for keeping your focus on what can be done without losing track of what is needed for a complete picture.  Your information and research agenda may include a needs assessment to help understand specific factors and help identify actions to address them.

Calculate the cost of bad results

The product of this component is an estimate of how much your community would spend if your forecast is accurate, and how much would be saved if you succeed.  Because this is a dollars-and-cents point of view, it can be very powerful.  Your business community might be helpful in this analysis.

The analysis involves collecting the budget information for the longest period of time possible:

1. First identify the expenditures in your community (local, state, federal, public and private) that would go down if your prevention activities were successful.  What expentitures are needed only because your population has not achieved the desired state of well-being?
   • Include whole programs whenever possible.
2. Then list current expenditures in your community (local, state, federal, public and private) that are intended to reduce (A).
   • Look in the program budgets associated with (A) for items that are explicitly devoted to reducing long term cost.
   • Add any expenditures for whole programs that are devoted to reducing long-term need for remediation (including prevention programs).
   For example, recently published studies suggest that ovarian cancer is now less common than it was because fewer women are using hormone replacement therapy to cope with menopause symptoms.  In this case, (A) might include medical treatment costs, lost productivity associated with time spent in treatment, disability and death of women with ovarian cancer, and public and health care provider ovarian cancer education programs.  (B) might include the ovarian cancer screening efforts that are part of larger ovarian cancer treatment programs, and the public and health care provider education programs.
3. Then construct a graph showing the trends for total spending on (A) and (B).

If you think about the cost of bad results from the point of view of your entire community and you think broadly, you will be able to include very diverse costs (e.g., the bad result for not achieving "all women are healthy" includes both hospital treatment costs and lost wages).

Also, if you start with the total cost of bad results, you can avoid trying to allocate a budget item between the causes of bad results (e.g., How much of the cost of lost wages was caused by obesity?  How much of the cost of a hospital was caused by tobacco use?).