Pilot Studies

A pilot study is a brief and limited version of the planned research. The goal of a pilot study is to refine the procedures of the research project. A pilot study is similar to a shakedown cruise of a new ship. In a shakedown cruise, the ship is put to sea for a short cruise to find out if all the systems work. No sailor would consider a long trip on a new ship without first checking whether the ship is seaworthy. Similarly, many research projects benefit from a pilot study to determine their seaworthiness. The primary goal of a pilot study is not to collect research data, but to check out research procedures so that adjustments can be made before the actual data are collected. A secondary goal is to determine if the planned statistical analyses work. Problems that show up in the pilot study are fixed by changing the data collection procedures or the statistical analyses.

Here are three examples of how pilot studies help reveal unanticipated problems in a research proposal. A student researcher built a maze for rats that had vertical, sliding doors that prevented rats from retracing their steps. Before she collected data, however, she conducted a pilot study with two rats. She soon discovered a major problem. She had not accounted for the rat’s tails! The problem was that the doors descended all the way to the floor and struck the rat’s tails. Rather quickly, the two rats in the pilot study refused to enter new sections of the maze. She solved the problem by installing small wooden blocks at the bottom of the slides so that the doors no longer struck the rats’ tails. A second pilot study with two more rats confirmed the success of her solution. The doors did not hit the rats’ tails and they readily entered the next section of the maze.

Here is another example. A doctoral student (English, 2002) investigating the dream content of women in the first stages of menopause wanted to pilot test her data–collection procedures. She recruited a small sample using the community snowball technique. In community snowball sampling, researchers find a few participants and then ask them to identify others who share the characteristic under study, in this case, being in the early stages of menopause. English administered her dream content questionnaire to the pilot sample. An additional variable, subjective sleep quality, was added.  She conducted her dissertation research with the new variable using a larger sample.

Another researcher’s pilot study revealed that going door–to–door to recruit participants in a college residence hall to administer a questionnaire individually took too much time. He adjusted his procedure and recruited participants at a weekly residence hall meeting and distributed the questionnaire in that group setting instead.

Stop & Think: What would have happened if these three researchers had not conducted these two pilot studies?

The student’s maze learning study might have failed entirely. Rats fearful of entering the next section of the maze are very slow learners. The doctoral student, benefited too. The adjustments she made led to a methodologically stronger procedure. The last researcher saved much time and effort by changing his original plans.

After conducting a pilot study, some final adjustments to the data collection procedure or to the apparatus are usually necessary. Just like planning for a party, the planning for data collection may go on until the very last minute. Once data collection begins, however, the same research methodology should be used for all participants. No further changes should be made.

Back to Chapter 1 Outline