Contact Name
Barbara Dietsch
562-985-9488

Contact Email
bdietsc@WestEd.org

Dennis McBride, Ph.D.
The Washington Institute
University of Washington

The Washington State Department of Health (DOH) has funded 11 Adolescent Pregnancy Prevention projects since 1994. Funding, authorized by state legislative action, was provided through a competitive process that was open to all health departments, schools, family planning agencies, churches, and youth organizations across the state. The 11 "Community Projects," as they are called, received funding ranging from $40,000 to $50,000 per year.

The teen population served by most of these projects is at "high risk" of premature sexual activity and its related consequences. They include youths referred by school counselors, family planning clinics, and other sources that have contact with potentially high-risk youth. Because of the high risk, the projects are more focused on the individual than are most curriculum based models. While they use an education and skill-building approach similar to popular curriculum based models (e.g., Postponing Sexual Involvement, Sex Can Wait, and Reducing the Risk), they add a "client-centered" component that combines education with individualized services including counseling, mentorship and/or advocacy.

In 1995, the DOH contracted with the Washington Institute, a research and training institute affiliated with the University of Washington, to conduct the evaluation. The DOH, the evaluators, and legislators were aware that a lack of strong evaluations had resulted in limited knowledge about effective teen pregnancy prevention approaches. Therefore, projects were required to develop, with the assistance of the evaluation team, rigorous evaluation designs that would employ random assignment (preferred) or matched comparisons (minimum). Of the 11 funded projects, 9 have randomized designs.

Establishing rigorous evaluation protocols in adolescent pregnancy prevention projects, or for any social and health service programs for that matter, is a difficult process. Nonetheless, with perseverance and cooperation among all stakeholders, we were able to do this. Certainly, there were barriers to overcome, but once we did overcome them and the evaluation process became integrated into everyday project operations, things progressed smoothly. The process of implementing these strong designs and the rewards realized from them has been a highlight of this evaluation project.

The first year was difficult. Project staffs were not accustomed to doing evaluation. Not only were they not used to identifying treatment versus control clients, assisting in data collection activities, or documenting service delivery, they were also required to obtain active parental consent for those clients under 14 to participate in the evaluation. Obtaining this consent, especially from high-risk families, was challenging. A second problem was that some staff, as well as some stakeholders (e.g., school principals, Advisory Board member), viewed the evaluation protocol as ethically questionable in cases where, for comparison purposes, services were withheld from certain clients, or different services were provided to separate groups. Other barriers encountered included sites' ability to attract and maintain clients, project staff turnover, and community resistance.

To overcome these barriers, the evaluation team, stakeholders (which includes project personnel), and the funding agent maintain a close working relationship. To foster good working relationships, the evaluators and DOH personnel conduct statewide and regional workshops, ongoing site visits, continuous phone contact, and careful assessment and suggested solutions for problems as they occur. The evaluators and the DOH hold regular meetings to discuss each project's progress and troubleshoot problems. Gradually, solutions were found.

Perhaps the most important factor contributing to the success in implementing random designs is that we have been successful in relating the value of having such designs to project staff and other stakeholders. Instead of emphasizing whether projects are successful or not, having a strong design enables us to better understand how the program is working so that needed modifications can be made to maximize the effect of services on clients. Simply stated, strong designs give better information for program decisions than do weaker designs. Stakeholders have become more comfortable and supportive of these designs and evaluation activities in general, as they discover how information feedback is useful to the development of stronger interventions.

To date, we have data on 1086 treatment and 869 control clients. Hypotheses being tested include positive changes in sexual behavior intent, improvements in attitudes and values concerning postponing sexual intercourse, teen-parent communications about sexuality, educational aspirations, substance use and abuse, and other factors concerning assets and risks. These are for younger clients, 10 - 13 years of age. For older clients (14 - 18), hypotheses concerning sexual behavior and contraceptive use and intent are added.

It has taken three years to get a solid test of these hypotheses. While one project consistently shows positive differences between treatment and control groups and some isolated effects occur in other projects, the vast majority of hypotheses are not supported, showing little or no effects of the interventions across most of the projects. It is tempting to conclude that these projects are not effective and cut funding. However, cutting funding is at least premature and would probably be a mistake. Instead, we have opted to begin modifying the project interventions.

The reasons to maintain the projects are compelling. The currently funded programs have strong evaluation components integrated into their projects. The measures are highly reliable and appear to be valid. Focus groups indicate that project assumptions and orientation appear to fit well with the populations served. Each project has had success in overcoming barriers to implementation, attracting and keeping clients, and gaining acceptance in the communities in which they operate. Finally, sites are using evaluation reports to modify and improve their programs. Hence, we have the processes in place to detect improvements if, and when, they occur.

Modifications of interventions began last year. For instance, we are increasing the average service dose to clients and, for higher risk clients, increasing the intensity of service as well. The amount and intensity of service needed by our clients are not yet known but are high on our list to discover. We will continue evaluating these projects and monitoring their progress until we discover the most promising strategies for addressing the difficult issues surrounding sexual behavior that are impacting our youth.