Lecture Notes: Experiments


Experiments provide the only method by which researchers can specify cause and effect. Experiments are set up with minimum of two groups. One group, called the control group, is not given the experimental treatment, i.e., some procedure or substance. The other group, called the experimental group, is given the experimental treatment. Everything else about the two groups is as similar as possible. More than one treatment can be tested at the same time by adding additional experimental and control groups.

Here is a quick example. I want to test whether or not brushing teeth three times a day with toothpaste is better than just brushing with water. How could I answer that question? To answer that question experimentally, I would set up two groups. One, the experimental group, would brush with toothpaste three times a day. The other, the control group, would also brush three times a day, but with water. What else must I do to make this a good experiment? First, I need to be sure that both groups are about the same with respect to their teeth. I will exclude those with false teeth or no teeth from participation. Also, by using random assignment of participants to each group, I can take advantage of a statistical property related to sampling. Namely, if I randomly assign participants to each group and if I use large enough samples, I will usually have samples that approximate the population and thus approximate each other.

Now, I need a way to determine whether or not the toothpaste is working. I can do this by measuring some variable affected by toothpaste. In this case, I will choose the number of cavities found after six months. Then, to further make the groups equal, I will send all of them to the dentist and have their teeth checked and their existing cavities filled. At this point, my two groups are about as equal as I can make them. Let's stop and define some terms.

In the example above, the toothpaste is the independent variable or IV. The number of cavities is the the dependent variable or DV. Formally, the IV is the only variable that a researcher does not control. More practically, think of the IV as the reason for conducting the study. The DV depends on the IV. So, fewer cavities should occur in the group that brushes with toothpaste. Recall our discussion of hypotheses and hypothesis formation. The hypothesis here is that brushing with toothpaste will result in fewer cavities after six months. More generally, hypotheses can be stated as:

The IV led to some change in the DV

Now to finish the experiment. Two groups are randomly assigned from a population to either an experimental or to a control group. The experimental group will receive the IV, toothpaste, and will brush three times a day. The control group will not receive the IV, instead they will only brush with water three times a day. Both groups start with no cavities. Six months later, the number of cavities are counted. If toothpaste does work as a cavity preventative, then the experimental group should have fewer cavities. At this point, I can confidently say, "Toothpaste has been proven to be an effective dentifrice when used as part of a regular program of brushing. Research results available upon request."

Another good way to learn about experiments is to look at bad ones. Examine the following bad experiment. Suppose we have created a new drug, a sleeping pill. We call it Nox-Out©. Now, we are interested in marketing it by saying that it is better at putting people to sleep than other sleeping pills. So, we decide to conduct an experiment.

 

First, review the terminology. We already know that the control group is the group that does not get the treatment. In this example, the dose of Nox-Out© is the substance to be tested or the independent variable (IV). The control group does not get the IV. In contrast, the experimental group does get the IV. Finally, the dependent variable (DV) is how we measure the effects of the independent variable. Both groups will be measured in terms of the dependent variable. In this experiment, a good dependent variable might be hours of sleep after taking Nox-Out©. Others might be quality of sleep after taking Nox-Out©, or latency to sleep after taking Nox-Out©. It is legitimate to use more than one DV. Here, only one will be used for simplicity's sake.

Now comes the first bad experiment. Suppose the participants in the control group are all between 60 and 80 years of age, and the subjects in the experimental group are all between 20 and 30 years of age. When we measure both groups using the dependent variable of hours of sleep, we may find that the young group sleeps longer. But, did that difference in the dependent variable come from the independent variable or from the age difference? Here, we cannot tell. We failed to control for age of the participants. When two or more factors contribute to a dependent variable difference, it is called a confound. Confounds are always a problem in experimentation, and they are the main reason that the study of experimental design is important. Here is a graph of the data on sleep and age.

Here is another bad experiment. Now let's house the control group in the new Hilton Hotel, and let's house the experimental group in the El Sleazo Motel, the one with the neon light that flashes "Vacancy" all night. Now our dependent variable difference may show that Nox-Out© does not work very well. Again, the problem is one of experimental design. Participants just find it more difficult to sleep at the El Sleazo, even when they have taken Nox-Out©. This example, too, illustrates a confound. Here is a diagram of how a well-designed experiment would house participants in a hotel being used for a sleep experiment.

A final bad experiment could be one in which we give Nox-Out© to the experimental group at 11:00 a.m. and then put them to bed; we put the control group to bed at 11:00 p.m. Assume that both groups woke up between 6 and 7 a.m. Again, the dependent-variable differences will lead us to believe that Nox-Out© does not work very effectively.

The message here is pretty obvious. In good experiments the experimental and control groups should differ in the giving or not giving the independent variable. Everything else about the two groups should be the same. So, to properly test Nox-Out©, both groups should be of the same age, sex, and degree of sleepiness. Also, they should be tested in the same environment (e.g., the Hilton).

When all of these details are taken care of, and when a difference in the dependent variable exists, then the experimenter can say it was the independent variable that caused the difference. The ability to make such statements is the main benefit of the experimental method. However, just because a study is an experiment does not give it that power. There have been plenty of bad experiments like those described above. So, a sophisticated student will look beyond the dependent variable to the design and the conduct of the experiment in order to determine the validity of the conclusion drawn from the data.


Back to Main Page