Outcome-based evaluation is a powerful tool that can help you tailor your programs to the needs of your community, report the difference you make, and improve library service. Outcome-based programs are designed with specific outcomes in mind, which helps you reflect on the reasons you offer summer programs, the type of summer programs you plan, and the results you want to achieve with your programs. In addition, outcome-based programs include an evaluation component which enables you to capture data about the impact of your work.
Public library summer programs make a difference. They help children and teens retain and enhance their reading skills during the summer. They help adults model reading activity for youth. They provide learning and enrichment opportunities, free of charge, that extend the reading experience and keep kids engaged while school is out. And they provide a haven and a community for readers.
But how do we share the impact of our work? How do we extend our programs effectively to underserved communities? And how do we ensure that our programs remain relevant? Outcome-based programming and evaluation are effective methods for accomplishing all of this and more.
- show meaningful results that demonstrate the value and impact of summer reading;
- are relevant to the community;
- are attractive to funders;
- build capacity among staff;
- are cost-effective;
- contribute to improved management decision-making.
Outcome-based evaluation is the process of identifying desired outcomes from a particular program or service and then conducting some kind of data collection to determine if those outcomes have been achieved.
Building on the output data that most librarians collect about their summer programs, outcome measures answer the question: “so what?” What changes in attitude, knowledge, skills, or status happened to your patrons as a result of taking part in the summer program? Outcome evaluation is most effective when it is part of a systematic planning process. Therefore, California’s Summer @ Your Library outcome- and outreach-based initiative is based on the four-step process that is outlined in the book, Dynamic Youth Services through Outcome-Based Planning and Evaluation (American Library Association, 2006), by Eliza T. Dresang, Melissa Gross, and Leslie Edmonds Holt:
- Gathering information.
- Determining outcomes.
- Developing programs and services.
- Conducting evaluations.
Each step builds on the step before. First you gather information from and about your users that will help you determine what their needs might be for a particular service. Based on what you learn, you decide the particular need the library can fulfill and express that need in terms of one or more outcomes as defined above. The third step is to plan and implement a program or service designed to achieve the desired outcome for your patrons. Finally, you conduct an evaluation study to find out if you really did achieve the outcomes.
It takes more thought, time, and effort to determine outcomes than it does to document outputs. Among the outputs of summer reading programs are numbers of people who sign up, who attend programs, the number of books read during the summer (or the weight of all the books read in the summer). It is really just a matter of counting.
Outcome measures are a little trickier because you are trying to figure out what change happened to an individual as a result of some service or program or intervention provided by the library. Usually the only way you can know this is to ask the person to tell you. Therefore, the typical methods used to collect the data that yields outcome measures are those used by social scientists: pre- and post-tests, surveys, interviews, and focus groups.
Libraries often include a budget for evaluation in large grant-funded projects. This allows them to bring in an outside consultant with special skills in this area. Increasingly, however, libraries are learning how to do it themselves and applying simple outcome measures to their ongoing services and programs. California’s Summer @ Your Library outcome and outreach initiative can help you do this.