cumulative

D.2 Internal vs. external validity

What the task asks: Explain the difference between confidence that an intervention caused a change in behavior and confidence that the result will hold for other people, settings and behaviors.

Experimental Design4 min read5 practice questionsOfficial outline (opens in a new tab)

The big idea

Two different questions decide how much a study is worth. First: did the independent variable, and nothing else, cause the change in behavior? That is internal validity. Second: would the same result occur with other people, in other settings, with other behaviors or with variations of the procedure? That is external validity.

Internal validity comes first. Until a study has shown that the intervention produced the effect, there is nothing to generalize. A study with strong experimental control and one participant can have high internal validity and untested external validity.

Behavior analysts build internal validity through experimental design. They build external validity mainly through replication, not through large random samples.

Breaking it down

Internal validity

A study has high internal validity when its design rules out other explanations for the change in the dependent variable. Experimental control, shown through prediction, verification and replication within the study (see D.4), is the evidence. Uncontrolled variables that could account for the change are confounds; D.3 covers the common ones.

Questions to ask:

  • Was a steady baseline established before the IV was introduced?
  • Did behavior change when, and only when, the IV changed, more than once?
  • Did only one variable change at each phase change?
  • Was the IV delivered as described (treatment integrity), and was the DV measured reliably (interobserver agreement)?

External validity

External validity is the degree to which a study’s findings hold beyond its particular participants, settings, behaviors, experimenters and procedural details. Cooper et al. (2020) describe it in terms of the generality of a study’s results.

Group designs try to reach external validity by sampling many participants and estimating an average effect. Single-case researchers point out that an average does not tell you whether the effect will hold for a particular new person (Sidman, 1960). They build generality one demonstration at a time:

  • Direct replication repeats the experiment under the same conditions. With the same participant it shows the effect is reliable; with new but similar participants it shows the effect is not unique to one person.
  • Systematic replication repeats the experiment while deliberately changing something: the setting, the type of participant, the behavior or the person delivering the procedure. Each success extends the range over which the finding holds.
Participant Setting Implementer Original study Noor Clinic room Therapist Direct replication Noor Clinic room Therapist Systematic replication Noor Classroom Therapist Systematic replication Noor Clinic room Parent
Extending Noor's result. A direct replication repeats every feature of the original study. Each systematic replication deliberately changes one feature (highlighted) and keeps the rest.

The trade-off

Tight control can make a study less like everyday life. A procedure run by a researcher in a quiet room with one learner may not work the same way when a busy teacher runs it with 25 students. The answer is not to loosen control but to replicate in more natural conditions.

Other kinds of validity

Measurement validity (C.8) asks whether the data measure the behavior you care about. Social validity asks whether the goals, procedures and outcomes are acceptable and important to the people affected. Neither is the same as internal or external validity.

Seeing it in practice

ScenarioInternal validityExternal validity
A reversal design with Noor, age 6, shows hitting drops each time functional communication training is in place and rises each time it is withdrawn. One room, one therapist.High: the effect was shown and replicated.Untested: one learner, one setting, one therapist.
Dozens of published studies show functional communication training reducing problem behavior across ages, settings and functions.Depends on each study’s designStrong: many systematic replications
A school reports that bullying fell after a new program began, but the program started the same month a new principal arrived.Low: the new principal is a confound.Not worth judging until internal validity is established
A token system works for three students in one class, and the teacher then uses it successfully in two other classes.High if each demonstration was controlledGrowing through systematic replication

Common mix-ups

  • Internal validity is about cause; external validity is about reach. One asks whether the IV caused the change here. The other asks whether the change would happen elsewhere.
  • A large sample does not guarantee external validity for an individual. An average effect can hide many people who did not improve.
  • One participant does not mean no external validity. It means external validity has not been tested yet. Replication tests it.
  • Measurement validity is a different question. A study can have perfectly valid measures and still be confounded.

Exam tip: If the stem asks whether something other than the intervention could explain the change, the answer concerns internal validity. If it asks whether the result would apply to other people, settings or behaviors, it concerns external validity, and replication is usually the remedy.

Quick recap

  • Internal validity: the IV, not something else, caused the change in the DV.
  • External validity: the result holds across other participants, settings, behaviors and versions of the procedure.
  • Experimental design builds internal validity; replication builds external validity.
  • Direct replication repeats a study as it was; systematic replication varies features on purpose.

Further reading

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), chapter 10.
  • Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. Basic Books.

Key terms

Internal validity
The extent to which a study shows that the independent variable, and not some uncontrolled factor, produced the change in the dependent variable. Designs that demonstrate experimental control have high internal validity.
External validity
The degree to which a study's results hold beyond the specific participants, settings, behaviors and procedural details of that study. In single-case research it is established mainly through direct and systematic replication.
Experimental control (Control)
The highest level of scientific understanding. An experimenter manipulates an independent variable and shows that a specific change in the dependent variable reliably follows, while ruling out other explanations.
Confounding variable (Confound)
An uncontrolled variable that changes along with the independent variable and may affect the dependent variable, so that its effect cannot be separated from the effect of the independent variable.
Direct replication
Repeating an experiment under the same conditions as the original. Repeating it with the same participant shows the effect is reliable; repeating it with new but similar participants shows the effect is not unique to one person.
Systematic replication
Repeating an experiment while deliberately varying one or more of its features, such as the setting, the participants, the target behavior or the person delivering the procedure. Each success extends the range of conditions under which the finding holds.

Check yourself

5 questions on D.2. Choose an answer to see whether you're right and why.

  1. Question 1A behavior analyst uses an A-B-A-B design to show that self-monitoring increases on-task behavior for Lena, a high school junior, in her chemistry class. A colleague asks whether the procedure would work for other students in other classes. The colleague's question is about:
    Show the answer

    Answer: D. Whether a result holds for other people and settings is a question of external validity, or generality. The A-B-A-B design already speaks to internal validity, since it showed the effect and replicated it for Lena.

  2. Question 2An intervention for a client's elopement begins the same week the client's family moves to a home with a fenced yard. Elopement drops to near zero. Which kind of validity is most compromised?
    Show the answer

    Answer: C. The move is an uncontrolled change that happened at the same time as the intervention, so it offers another explanation for the drop. That threatens internal validity. External validity cannot even be judged until it is clear that the intervention caused the change.

  3. Question 3After a peer-tutoring procedure worked in a third-grade class, researchers repeat the study in a sixth-grade class with a different teacher, keeping the procedure the same. What is this, and what does it build?
    Show the answer

    Answer: D. Changing the grade and teacher on purpose while keeping the procedure makes this a systematic replication, and each success extends the range over which the finding holds. A direct replication would repeat the study under the same conditions.

  4. Question 4Why do single-case researchers rely on replication, rather than large random samples, to establish external validity?
    Show the answer

    Answer: A. A group average describes the population, and it can hide many individuals who did not respond. Replicating the effect person by person and setting by setting shows where it holds. Replication adds to experimental control within a study; it does not replace it.

  5. Question 5A study run in a quiet research room by a highly trained experimenter shows clear experimental control. A school asks whether its busy classroom teachers will get the same effect. What is the best next step?
    Show the answer

    Answer: C. High internal validity shows the effect is real under the original conditions, not that it will hold elsewhere. Testing it with the people and settings in question is a systematic replication that answers the school's question. More sessions in the same room would add nothing about generality.

Adds a step to your study record. Saved in this browser only.

Type to search all 104 guides and the glossary.