Methods of Data Collection

Regardless of the method of research, data collection will be necessary. The method of data collection selected will primarily depend on the type of information the researcher needs for their study; however, other factors, such as time, resources, and even ethical considerations can influence the selection of a data collection method. All of these factors need to be considered when selecting a data collection method because each method has unique strengths and weaknesses. We will discuss the uses and assessment of the most common data collection methods: observation, surveys, archival data, and tests.

Sampling

Science offers a systematic way to make comparisons and guard against bias. For instance, one technique used to avoid sampling bias is to select participants for a study in a random way. This means using a technique to ensure that all members have an equal chance of being selected. Simple random sampling may involve using a set of random numbers as a guide in determining who is to be selected. For example, if we have a list of 400 people and wish to randomly select a smaller group or sample to be studied, we use a list of random numbers and select the case that corresponds with that number (Case 39, 3, 217, etc.). This is preferable to asking only those individuals with whom we are familiar to participate in a study; if we conveniently chose only people we know, we know nothing about those who had no opportunity to be selected. There are many more elaborate techniques that can be used to obtain samples that represent the composition of the population we are studying. But even though a randomly selected representative sample is preferable, it is not always used because of costs and other limitations. As a consumer of research, however, you should know how the sample was obtained and keep this in mind when interpreting results. It is possible that what was found was limited to that sample or similar individuals and not generalizable to everyone else.

The observational method involves the watching and recording of a specific behavior of participants. In general, observational studies have the strength of allowing the researcher to see for themselves how people behave. However, observations may require more time and resources than other data collection methods, often resulting in smaller samples of participants. Researchers may spend significant time waiting to observe a behavior, or the behavior may never occur during observation. It is important to remember that people tend to change their behavior when they know they are being watched (known as the Hawthorne effect).

Observations may be done in a naturalist setting to reduce the likelihood of the Hawthorne effect. During naturalistic observations, the participants are in their natural environment and are usually unaware that they are being observed. For example, observing students participating in their class would be a naturalist observation. The downside of a naturalistic setting is that the researcher doesn’t have control over the environment. Imagine that the researcher goes to the classroom to observe those students, and there is a substitute teacher. The change in instructor that day could impact student behavior and skew the data.

If controlling the environment is a concern, a laboratory setting may be a better choice. In the laboratory environment, the researcher can manage confounding factors or distractions that might impact the participants’ behavior. Of course, there are expenses associated with maintaining a laboratory setting, increasing the cost of the study, that would not be associated with naturalist observations. And, again, the Hawthorne effect may impact behavior.

Surveys are familiar to most people because they are so widely used. This method enhances accessibility to subjects because they can be conducted in person, over the phone, through the mail, or online, and are commonly used by researchers to gather information on many variables in a relatively short period of time.

Most surveys involve asking a standard set of questions to a group of participants. In a highly structured survey, subjects are forced to choose from a response set such as “strongly disagree, disagree, undecided, agree, strongly agree”; or “0, 1-5, 6-10, etc.”  One of the benefits of having forced-choice items is that each response is coded so that the results can be quickly entered and analyzed using statistical software. While this type of survey typically yields surface information on a wide variety of factors, they may not allow for an in-depth understanding of human behavior.

Of course, surveys can be designed in a number of ways. Some surveys ask open-ended questions, allowing each participant to devise their own response, allowing for a variety of answers. This variety may provide deeper insight into the subject than forced-choice questions, but makes comparing answers challenging. Imagine a survey question that asked participants to report how they are feeling today. If there were 100 participants, there could be 100 different answers, which is more challenging and takes more time to code and analyze.

Surveys are useful in examining stated values, attitudes, opinions, and reporting on practices. However, they are based on self-report, and this can limit accuracy. For a variety of reasons, people may not provide honest or complete answers. Participants may be concerned with projecting a particular image through their responses, they may be uncomfortable answering the questions, inaccurately assess their behavior, or they may lack awareness of the behavior being assessed. So, while surveys can provide a lot of information for many participants quickly and easily, self-reporting may not be as accurate as other methods.

Video 2.6 Methods of Data Collection explains various means for gathering data for quantitative and qualitative research.