Types of Data Collection

Different types of data collection assess programs and policies in various ways. Alcohol, other drug, and violence (AODV) prevention programs focused on environmental change can measure student behaviors and consequences related to AODV, but meaningful change on this level can take many years. Therefore, it is also important to measure the environment itself. Doing so identifies and raises awareness of environmental factors that contribute to AODV problems; it also documents intermediate outcomes of environmental strategies.

Surveys

The most frequently used method of assessing student behaviors is administering surveys to randomly drawn samples of students. Surveys can also measure changes in the environment. Options for surveys run the gamut from simple pencil-and-paper surveys to Web-based surveys that can save time and money. Both methods produce similar results, so you should feel free to choose a method that best fits your resources, and–in order to get the best response rate–that will appeal to students on your campus.

You can use existing surveys, such as the Core Institute’s alcohol and other drug surveys, or make up your own questions. When using an existing instrument, make sure that the survey is appropriate for your population and information is available regarding reliability and validity. If you design your own survey, try to obtain the assistance of someone experienced in instrument construction and be sure to pilot test your survey.

Survey samples are intended to represent the population of interest. Generally, larger samples are preferable. In most cases, a 10 percent sample is sufficient, although actual sample sizes will vary depending on the population.

When conducting surveys, also consider response rates. As response rates decrease, the likelihood of a biased sample increases. A response rate of 70 percent is considered ideal with response rates of 50 percent to 70 percent generally considered adequate. Additionally, you should always compare the characteristics of your sample with that of the population of interest to determine if there are differences between the two that might bias results.

Secondary Data

Written records, such as those from campus and community police, health services, and hospitals can assist with measuring environmental-level change. Campus prevention professionals can collaborate with these constituencies to be sure their record keeping yields quality information that is useful for needs assessment, data collection, and evaluation.

For instance, does your health services department record data on how many students come in with alcohol-related problems? Does campus housing keep track of vandalism in campus housing? Data like these are useful for measuring shorter-term environmental change, and longer-term behavior change among students.

Focus Groups

A focus group is a quick and inexpensive way to gather in-depth information from members of your target audience. Focus groups with students, faculty, and community members can help assess both behavioral and environmental level change. They usually produce data that are more in-depth than that of surveys.

Focus groups should be kept to 10–12 participants, and, in order to produce reliable data, you have to have a moderator who is trained in conducting focus groups. Participants should feel they are in a safe place, and that their responses will remain confidential. Focus groups can be run during a class, if you have a cooperative professor, or can be set up independently on campus (which requires time for recruitment).

Any data collection method has flaws. Student surveys, for example, may underestimate problems due to participants providing socially desirable responses, rather than truthful ones. A good data collection plan will include multiple measurement techniques to increase reliability. This process, called triangulation, allows you to consider whether student reports are consistent with police reports and focus group results.

Knowing what data you want to collect and when can help you identify the most appropriate type of data collection. Most evaluation plans include several different methods of data collection to yield meaningful results.

Observations

Field observations are another good way to assess student behavior. Measures may include numbers of students attending alcohol-free events or numbers of students seen drinking in public places on campus. Structured field observations can be used to measure blood alcohol concentration. Field observations also generally provide reliable measurement of the environment. Environmental field observations may include rowdy incidents by students at bars or parties.

For More Information

The Center's publication, Methods for Assessing College Student Use of Alcohol and Other Drugs, provides detailed information on using data collection to understand the extent of AOD use on campus and in the community.