Assessment, Strategic Planning, and Evaluation of SWIs

Organizing a statewide college alcohol and other drug (AOD) prevention initiative can be a daunting task, not simply because of the large scale of the project. There is no single formula or approach to a statewide initiative, and the research literature on campus prevention is still too limited to yield definitive answers to the question of “what works.” This is especially true of statewide initiatives, where only a few states have mounted a comprehensive evaluation of how initiative activities created change at the state, campus and community, or individual student behavior levels.1 Statewide initiative leaders frequently ask, “What programs should be implemented? What combination of efforts will be most cost-effective and likely to yield results? What do campus and community coalitions need in order to succeed?”

Research from community settings suggests a set of promising college AOD prevention strategies with potential for effectiveness (Task Force on College Drinking, 2002). Despite the lack of “proven programs” specific to college campuses, statewide initiatives are organized to encourage campuses to adopt an environmental approach to AOD prevention based on the community research. Given the diverse settings, structures, and cultures found in different states and on different campuses, statewide initiative leaders need to design an initiative that is targeted to their particular circumstances and needs. This task is best accomplished using a systematic planning and evaluation process.

It is important to remember that statewide initiatives work on two levels: stimulating work at the campus-community level and making policy and systems changes at the state level. Therefore, the planning, goal setting, and evaluation process needs to happen at both campus and state levels.

In the first steps at the state level, statewide initiative leaders build relationships, gather resources, identify and involve key players, and develop an infrastructure – funding and staff – to implement the initiative. At the next stage, leaders collect data about the problems and needs at the state and local level, assess the capacity of local teams to implement environmental strategies, and determine the need for training for campus and community teams. This assessment informs the creation of a strategic plan that describes the initiative activities, outlines how the activities are expected to produce change at the local level, and includes a plan for evaluating the initiative. Typically, the central focus of a statewide initiative strategic plan at this stage focuses on building the capacity of campus and community teams, through training and technical assistance, to produce local strategic plans, build effective coalitions, and implement environmental strategies. In other words, at the same time that initiative leaders are building a state-level initiative, they are supporting similar progress at the campus and community level.

This is the crux of how statewide initiatives affect student AOD use: the expectation is that this sequence of activities at the state and local levels will lead to implementation of environmental strategies, leading in turn to changes in the environment that supports high-risk AOD use and its consequences. Then, changes in the environment will produce decreases in individual student AOD use and the adverse consequences of such use.
A good statewide initiative strategic planning process is driven by:

• an understanding of the problems that are occurring at both state and campus-and-community levels,
• specifies what program activities will be undertaken,
• draws from theory and best practices,
• develops consensus about how to move forward, and
• includes a plan for evaluation.

Simply put, statewide initiative efforts are most effective when the leadership knows what outcomes they want to achieve and how they expect what they are doing will lead to those outcomes.

The following five steps are a planning and evaluation process that can be used to plan interventions for statewide initiatives. While these steps are presented as a linear process, in practice statewide initiative leaders will often rework earlier steps as more information is gathered and assessed. In addition, planning is most effective when it is a collaborative effort, allowing key stakeholders to contribute information and ideas for program design, implementation, and evaluation. The best mechanism for statewide initiative planning is to gather information about problems, needs, resources, and long-range goals from a wide variety of stakeholders and then charge a smaller planning group with design of the plan itself. The planning group provides continuous feedback to the larger number of constituents on the progress of the plan and submits draft plans to the larger group to gain support. While experience has shown that statewide initiative planning can move forward quite effectively in this way, there are other possible ways to develop the partnerships and alliances necessary to build support for a strategic plan.

The strategic planning steps include:

1. Conduct a problem analysis/needs assessment
2. Establish long-range goals
3. Consult science, theory, and assumptions
4. Create a strategic plan
5. Implement program activities and evaluate whether goals were achieved