The big idea
Two different questions decide how much a study is worth. First: did the independent variable, and nothing else, cause the change in behavior? That is internal validity. Second: would the same result occur with other people, in other settings, with other behaviors or with variations of the procedure? That is external validity.
Internal validity comes first. Until a study has shown that the intervention produced the effect, there is nothing to generalize. A study with strong experimental control and one participant can have high internal validity and untested external validity.
Behavior analysts build internal validity through experimental design. They build external validity mainly through replication, not through large random samples.
Breaking it down
Internal validity
A study has high internal validity when its design rules out other explanations for the change in the dependent variable. Experimental control, shown through prediction, verification and replication within the study (see D.4), is the evidence. Uncontrolled variables that could account for the change are confounds; D.3 covers the common ones.
Questions to ask:
- Was a steady baseline established before the IV was introduced?
- Did behavior change when, and only when, the IV changed, more than once?
- Did only one variable change at each phase change?
- Was the IV delivered as described (treatment integrity), and was the DV measured reliably (interobserver agreement)?
External validity
External validity is the degree to which a study’s findings hold beyond its particular participants, settings, behaviors, experimenters and procedural details. Cooper et al. (2020) describe it in terms of the generality of a study’s results.
Group designs try to reach external validity by sampling many participants and estimating an average effect. Single-case researchers point out that an average does not tell you whether the effect will hold for a particular new person (Sidman, 1960). They build generality one demonstration at a time:
- Direct replication repeats the experiment under the same conditions. With the same participant it shows the effect is reliable; with new but similar participants it shows the effect is not unique to one person.
- Systematic replication repeats the experiment while deliberately changing something: the setting, the type of participant, the behavior or the person delivering the procedure. Each success extends the range over which the finding holds.
The trade-off
Tight control can make a study less like everyday life. A procedure run by a researcher in a quiet room with one learner may not work the same way when a busy teacher runs it with 25 students. The answer is not to loosen control but to replicate in more natural conditions.
Other kinds of validity
Measurement validity (C.8) asks whether the data measure the behavior you care about. Social validity asks whether the goals, procedures and outcomes are acceptable and important to the people affected. Neither is the same as internal or external validity.
Seeing it in practice
| Scenario | Internal validity | External validity |
|---|---|---|
| A reversal design with Noor, age 6, shows hitting drops each time functional communication training is in place and rises each time it is withdrawn. One room, one therapist. | High: the effect was shown and replicated. | Untested: one learner, one setting, one therapist. |
| Dozens of published studies show functional communication training reducing problem behavior across ages, settings and functions. | Depends on each study’s design | Strong: many systematic replications |
| A school reports that bullying fell after a new program began, but the program started the same month a new principal arrived. | Low: the new principal is a confound. | Not worth judging until internal validity is established |
| A token system works for three students in one class, and the teacher then uses it successfully in two other classes. | High if each demonstration was controlled | Growing through systematic replication |
Common mix-ups
- Internal validity is about cause; external validity is about reach. One asks whether the IV caused the change here. The other asks whether the change would happen elsewhere.
- A large sample does not guarantee external validity for an individual. An average effect can hide many people who did not improve.
- One participant does not mean no external validity. It means external validity has not been tested yet. Replication tests it.
- Measurement validity is a different question. A study can have perfectly valid measures and still be confounded.
Exam tip: If the stem asks whether something other than the intervention could explain the change, the answer concerns internal validity. If it asks whether the result would apply to other people, settings or behaviors, it concerns external validity, and replication is usually the remedy.
Quick recap
- Internal validity: the IV, not something else, caused the change in the DV.
- External validity: the result holds across other participants, settings, behaviors and versions of the procedure.
- Experimental design builds internal validity; replication builds external validity.
- Direct replication repeats a study as it was; systematic replication varies features on purpose.
Further reading
- Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), chapter 10.
- Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. Basic Books.
Key terms
- Internal validity
- The extent to which a study shows that the independent variable, and not some uncontrolled factor, produced the change in the dependent variable. Designs that demonstrate experimental control have high internal validity.
- External validity
- The degree to which a study's results hold beyond the specific participants, settings, behaviors and procedural details of that study. In single-case research it is established mainly through direct and systematic replication.
- Experimental control (Control)
- The highest level of scientific understanding. An experimenter manipulates an independent variable and shows that a specific change in the dependent variable reliably follows, while ruling out other explanations.
- Confounding variable (Confound)
- An uncontrolled variable that changes along with the independent variable and may affect the dependent variable, so that its effect cannot be separated from the effect of the independent variable.
- Direct replication
- Repeating an experiment under the same conditions as the original. Repeating it with the same participant shows the effect is reliable; repeating it with new but similar participants shows the effect is not unique to one person.
- Systematic replication
- Repeating an experiment while deliberately varying one or more of its features, such as the setting, the participants, the target behavior or the person delivering the procedure. Each success extends the range of conditions under which the finding holds.
Check yourself
5 questions on D.2. Choose an answer to see whether you're right and why.
Show the answer
Answer: D. Whether a result holds for other people and settings is a question of external validity, or generality. The A-B-A-B design already speaks to internal validity, since it showed the effect and replicated it for Lena.
Show the answer
Answer: C. The move is an uncontrolled change that happened at the same time as the intervention, so it offers another explanation for the drop. That threatens internal validity. External validity cannot even be judged until it is clear that the intervention caused the change.
Show the answer
Answer: D. Changing the grade and teacher on purpose while keeping the procedure makes this a systematic replication, and each success extends the range over which the finding holds. A direct replication would repeat the study under the same conditions.
Show the answer
Answer: A. A group average describes the population, and it can hide many individuals who did not respond. Replicating the effect person by person and setting by setting shows where it holds. Replication adds to experimental control within a study; it does not replace it.
Show the answer
Answer: C. High internal validity shows the effect is real under the original conditions, not that it will hold elsewhere. Testing it with the people and settings in question is a systematic replication that answers the school's question. More sessions in the same room would add nothing about generality.
Adds a step to your study record. Saved in this browser only.