Data Collection and Management

In order to determine whether an alcohol, other drug, and violence (AODV) prevention program or policy is effective, program staff and evaluators collect and manage data during each step of planning and implementation, and after the program is over. Different kinds of data are useful at different stages. For instance, process measures describe how the program was implemented, while outcome measures describe the effects of program efforts on AOD use and violent incidents. All of these measures can help administrators strengthen, improve, and showcase the program.

Once data are collected, program staff and evaluators analyze them to tell whether the program was implemented as designed and to determine if the activities resulted in the intended changes in AODV outcomes. If the data show the program is effective, administrators may want to broaden its scope or seek additional funding. If an approach appears to be failing, data analysis can show where things went wrong–for instance was there a problem with planning or with implementation? Can the program be refined, or is a brand-new approach needed?

Data collection and management are important steps to finding out what works and why in campus and community AODV prevention.

What Should You Measure?

Collecting data about AODV may at first seem overwhelming. Data need to indicate what the problems are, what programming has been implemented, and what was accomplished. At each stage, your evaluator can provide important resources to help in this process.

For example, in the needs assessment stage before program implementation, evaluators may review:

• Survey data about current levels of AOD use and violent incidents on campus and in the surrounding community in order to determine the AODV needs and issues that are unique to that environment
• Observational assessments of specific campus or community environments to determine contexts in which problems occur
• Incident reports from campus and community sources to determine magnitude of problems, as well as where and when the problems occur

These data sources can assist you in determining which specific AODV outcomes should be the focus of efforts. In addition, they can help you develop a logic model to outline the processes by which problems occur and the mechanisms through which these problems can be reduced.

During implementation, evaluators collect process measures such as:

• Utilized resources (staff, volunteers, funds) to see how they compare with what was initially planned
• Activity levels and the number of people or groups served by a program to determine if it reached the intended population

After implementation, evaluators analyze outcome measures including:

• Changes in behavior, both intermediate (e.g., wait staff refusing to serve underage patrons) and long term (fewer students driving while intoxicated)
• Environmental outcomes, such as observations of rowdy behavior by students at bars, parties, or on the street

The Center has compiled additional Sample Indicators for Measuring Environmental Change Related to College AODV Prevention.

Human Subjects Protections

It is important to conduct all data collection efforts in a manner that respects and protects the study participants. IHEs and funders may have specific requirements that prohibit data collection unless specific procedures are followed. Prior to collecting data, determine whether your institution has an institutional review board (IRB) that governs human subject protections. Nearly all universities have an IRB; state and community agencies may have their own IRB or may have an arrangement with another institution. In addition to following IRB guidelines, review general considerations for protection of human subjects and ensure that the study protocol is designed in accordance with these guidelines.