Chapter 7 Basic Designs I

Modified: 2008-03-28

A CONCLUSION REQUIRES A COMPARISON

In The Know--A one-number data point is always just a start toward understanding. More numbers are necessary if you are to have a complete understanding.

IDEALIZED EXPERIMENTS

THE NONRANDOM ASSIGNMENT DESIGN

 

Table 7.8--A sampling of independent variables that are usually investigated with a nonrandom assignment design

Independent variable
Levels (Conditions)

gender

male and female

socioeconomic status

lower, middle, and upper class

culture

Chinese, Korean, Japanese

age

10-12 years, 14-16 years, 18-20 years

traumatic experience

raped, automobile accident, neither

personality

highly hostile, moderately hostile; not hostile

frontal lobe damage

yes, no

college major

natural science, humanities, other

Table 7.9 Notation for the nonrandom assignment design for two treatments

Method of assignment

Independent variable

Dependent variable

NR

T1

O

NR

T2

O

Time --------------------------------------------------------------------------------------------------------->

NR -- assignment to treatments is nonrandom

T1 and T2 -- two treatment levels of the independent variable

O -- observations that constitute the dependent variable

 

Table 7.10 The design of the exercise/tiredness experiment

Nonrandom assignment

IV

DV

Exercise score is 2 or less

Less active

POMS tiredness score

Exercise score is 3 or more

More active

POMS tiredness score

Campbell & Stanley's Pre-experimental and Experimental Designs

EXTRANEOUS VARIABLES

Stop & Think--Identify one variable that changes along with changes in levels of exercise. This variable must be one that affects tiredness. Sleep, nutrition, and weight are possibilities.

THE EXTRANEOUS VARIABLE OF SELECTION

THE RANDOM ASSIGNMENT DESIGN

 

Table 7.11 Notation for the random assignment design for two treatments

Random assignment

IV

DV

R

T1

O

R

T2

O

Time --------------------------------------------------------------------------------------------->

R -- assignment to treatments is random

T1 and T2 -- two treatment levels of the independent variable

O -- observations that constitute the dependent variable

Problems with the Random Assignment Design

THE EXTRANEOUS VARIABLE OF DIFFERENTIAL ATTRITION

also referred to as mortality or experimental mortality:

THE EXTRANEOUS VARIABLE OF DIFFUSION OF TREATMENT

When the conditions intended for only one group of participants are experienced by other groups, the experiment suffers from an extraneous variable called diffusion of treatment . Experiments that do not control for diffusion of treatment are not likely to detect any differences that exist among different levels of the IV.

Preventing Diffusion of Treatment:

  1. Conduct the experiment in a short time period
  2. Ask participants not to reveal details to others who have yet to participate
  3. Embed irrelevant questions into to the experiment so that participants cannot tell what the research question really is.

A COMPARISON OF RANDOM AND QUASI-EXPERIMENTA DESIGNS

  1. In random assignment designs, participants are assigned to groups using some random procedure.
  2. In quasi-experiments, participants are not assigned using a random procedure.
  3. Using random assignment prevents the extraneous variable of selection.

 

In The Know--When researchers refer to a manipulated IV, they generally mean that the participants were randomly assigned to conditions.

Table 7.15 Control of extraneous variables by different research designs.

Extraneous variables

Design

Selection

Differential Attrition

Diffusion of Treatment

Nonrandom assignment

Not controlled

Not controlled

Not controlled

Random assignment

Controlled

Not controlled

Not controlled

INDEPENDENT VARIABLE VALUES

The two most important elements of an experiment are the independent variable and the dependent variable.

In The Know--When one researcher wants to explain an experiment to another researcher, one of the first sentences is about the IV and DV. Values of the IV are selected by the researcher.

BETWEEN-SUBJECTS AND WITHIN-SUBJECTS DESIGNS DISTINGUISHED

BACKGROUND MATERIAL

A Conclusion Requires a Comparison

Web pages

The AFCARS Report

Report from United States Department of Health and Human Services estimates the number of children in foster care and provides vital statistics.

http://www.acf.hhs.gov/programs/cb/stats_research/afcars/tar/report10.htm

Design

Summarizes the concept of experimental design and links to several experimental and quasi-experimental designs.

http://www.socialresearchmethods.net/kb/design.htm

Developing an Evaluation Design

National Science Foundation page summarizes the research process including selecting a design.

http://www.nsf.gov/ehr/rec/evaldesign.jsp

Experimental and Non-experimental Designs

Tutorial on experimental and non-experimental designs covers designs and their strengths and weaknesses.

http://www.une.edu.au/WebStat/unit_materials/c2_research_design/design_exper_nonexper.htm

A Model Experiment

Web pages

Independent and Dependent Variables

Provides four experimental scenarios in which students must identify the independent and dependent variable and provides answers.

http://clcpages.clcillinois.edu/home/soc455/psycweb/research/variables.htm

An Actual Experiment

Web pages

Profile of Mood States (POMS)

Summarizes the Profile of Mood States (POMS) and links to research using it.

http://www2.hf.faa.gov/workbenchtools/default.aspx?rPage=Tooldetails&subCatId=10&toolID=202

Stem-and-Leaf Plots

Shows how to construct stem-and-leaf plots.

http://regentsprep.org/Regents/Math/data/stemleaf.htm

Stem-and-Leaf Plots

Tutorial on stem-and-leaf plots promotes their advantages and shows how to plot them.

http://www.statcan.ca/english/edu/power/ch8/plots.htm

The Quasi-experimental Design

Web pages

Quasi-experimental Design

Explains several quasi-experimental designs.

http://www.socialresearchmethods.net/kb/quasiexp.htm

Nonequivalent Groups Design

Describes a two-treatment quasi-experimental design and explains it.

http://www.socialresearchmethods.net/kb/quasnegd.php

Research Methods: Experimental Design

Online chapter covers pre-experimental design, quasi-experimental design, and true experimental design.

http://allpsych.com/researchmethods/experimentaldesign.html

Articles

Gribbons, B., & Herman, J. (1997). True and quasi-experimental designs, Practical Assessment, Research & Evaluation, 5. Retrieved December 28, 2006 from http://PAREonline.net/getvn.asp?v=5&n=14 .

Article discusses and differentiates between true (e.g., random assignment) designs and quasi-experimental designs.

Malhotra, Y. (1994). Role of science in knowledge creation: A philosophy of science perspective. Retrieved December 28, 2006 from http://www.kmbook.com/science.htm

Working paper discusses the the role of quasi-experiments in creating the knowledge base of science.

Extraneous Variables

Web pages

Definition of Extraneous Variables

Page defines extraneous variables.

http://www.alleydog.com/glossary/definition.cfm?term=Extraneous%20Variable

Wikipedia on Extraneous Variables

Wikipedia's page lists and defines eight extraneous variables.

http://en.wikipedia.org/wiki/Extraneous_variables

Extraneous Variables

Provides two examples of research scenarios in which students must identify extraneous variables. Page provides answers.

http://clcpages.clcillinois.edu/home/soc455/psycweb/research/Extraneous.htm

Threats to Internal Validity

Lists threats to internal validity (extraneous variables).

http://www.evaluationwiki.org/wiki/index.php/Threats_to_Internal_Validity

Wikipedia on Donald T. Campbell

Wikipedia's page on Donald T. Campbell gives a brief biography and lists selected works.

http://en.wikipedia.org/wiki/Donald_T._Campbell

Julian C. Stanley, Pioneer in Gifted Education

Obituary of Julian C. Stanley details his life and research.

http://www.jhu.edu/cty/about/stanley.html

The Extraneous Variable of Selection

Web page

Selection

Tutorial covers selection, provides example and non-example, analysis, and links to background information.

http://psych.athabascau.ca/html/Validity/selection.shtml

The Random Assignment Design

Web pages

Random Selection and Assignment

Discusses the concepts of random selection and random assignment as they relate to experimental design.

http://www.socialresearchmethods.net/kb/random.htm

Research Randomizer

Provides sets of random numbers for use in creating randomly assigned groups.

http://www.randomizer.org/

Effects of Food Assistance and Nutrition Programs on Nutrition and Health: Volume 1, Research Design

Page links to USDA online publication on research design that labels the random assignment experiment as the "gold standard" of research designs.

http://www.ers.usda.gov/publications/fanrr19-1/

What's So Special About Random Assignment?

Lists advantages of random assignment design.

http://www.srdc.org/en_about_us.asp?id=890

Randomized Experiment

Graphical depiction of process of randomly assigning participants to groups.

http://www.richmond.edu/~pli/teaching/psy200/ResearchMethods/RandomExperiment.htm

Articles

Ottenbacher, K. (1992). Impact of random assignment on study outcome: An empirical examination, Controlled Clinical Trials, 13, 50-61.

Empirical study of 60 clinical trials divided by design (random assignment vs. nonrandom) found that effect sizes did not differ by type of design.

Souraya, S. (2006). Random assignment: A systematic review, in R. R. Bootzin & P. E. McKnight (Eds.), Strengthening research methodology: Psychological measurement and evaluation (pp. 125-141).Washington, DC: American Psychological Association, 125-141.

Chapter evaluates the effectiveness of random assignment by reviewing previous research challenging the central role of random assignment in research methodology.

The Extraneous Variable of Differential Attrition

Web page

Experimental Mortality (e.g., Differential Attrition)

Tutorial covers selection, provides example and non-example, analysis, and links to background information.

http://psych.athabascau.ca/html/Validity/mortality.shtml

The Extraneous Variable of Diffusion of Treatment

Book

Girden, E. R. (1996). Evaluating research articles from start to finish, Thousand Oaks, CA: Sage Publications.

Text is designed to help students learn to critically read research articles and to look for threats to internal validity including diffusion of treatment.

A Comparison of Random and Quasi-experimental Designs

Web page

How much should random assignment research be privileged (or emphasized) over other types of research in education?

Three experts present their views on the methodological status of randomized assignment designs.

http://ed.stanford.edu/suse/news-bureau/educator/fall2004/pages/ForumQuestion.html

Independent Variable Values

Web page

Independent variable

Page displays definitions of independent variable.

http://www.answers.com/topic/independent-variable

Variables

General discussion of variables includes discussion of levels of the independent variable.

http://cnx.org/content/m10802/latest/

Between-subjects and Within-subjects Designs Distinguished

Web pages

Between vs. Within Subjects

Tutorial compares and contrasts between subjects and within subjects designs.

http://www.gseis.ucla.edu/courses/ed230a2/notes/subjects.html

What is a Between-subjects design?

Defines between subjects designs.

http://www.ergolabs.com/between_subjects_design.htm

What is a Within-subjects design?

Defines within-subjects design.

http://www.ergolabs.com/within_subjects.htm

Between Subjects Experimental Designs

Lists the various types of between subjects designs along with their advantages and disadvantages.

http://www.richmond.edu/~pli/teaching/psy200/ResearchMethods/BetweenSubjectsDesign.html

Within Subjects Experimental Designs

List the various types of within subjects designs along with their advantages and disadvantages.

http://www.richmond.edu/~pli/teaching/psy200/ResearchMethods/WithinSubjectsDesign.html

GLOSSARY

Between-Subjects Design: Any research design in which participants contribute scores to only one level of the independent variable.

Design: The part of a research plan that identifies the independent and dependent variables, tells how they are measured, and explains the procedures for collecting data.

Differential Attrition: An extraneous variable caused by a differential loss of participants from the levels of the IV in such a way as to bias the outcome.

Diffusion of Treatment: An extraneous variable that occurs when the conditions intended for only one group of participants are experienced by other groups.

Nonrandom Assignment Design: An experimental design in which participants's characteristics are used to assign levels of the independent variable.

Random Assignment: A procedure that uses chance to assign participants, procedures, or materials to groups

Random Assignment Design: An experimental design in which participants are randomly assigned to a level of the independent variable.

Replication Repeating an experiment with the same procedures or with planned changes in the procedures.

Research Plan: A description of the tasks required to complete a research project.

Selection: An extraneous variable that occurs when participants in the different levels of the IV are not equivalent even before the treatment is administered.

Within-subjects Design: A research design in which participants's scores are matched or paired on the basis of a similarity other than the IV or DV.

Back to Main Page