The big idea
A threat to internal validity is any factor other than the independent variable that could explain a change in behavior. A threat left uncontrolled becomes a confound, and the study can no longer claim that the intervention caused the change.
The classic list comes from group research (Campbell & Stanley, 1963), and most of it applies to single-case work as well. Some threats matter more in single-case designs, because participants are measured many times, conditions follow one another, and one person experiences every condition.
The exam usually describes a situation and asks you to name the threat. Sort them by what changed: the world, the person, the measurement or the order of conditions.
Breaking it down
Something in the world changed: history
A history threat is an event outside the study that happens at the same time as a phase change: a new teacher, a family move, a schedule change, an illness going around the classroom. It is the most common threat in applied settings, because real life does not pause for an experiment.
Something in the person changed: maturation
Maturation is change within the participant that comes with the passage of time: growing older, developing new skills, tiring over a long session, recovering from an illness. It is a particular risk in long studies with young children.
Measuring changed the behavior: testing
A testing effect occurs when being measured repeatedly changes performance by itself. A learner may improve on a probe through practice alone, or act differently because they know they are being watched (reactivity). Single-case designs measure repeatedly, so this threat is always present. A steady baseline shows whether repeated measurement alone is producing a trend.
The measurement changed: instrumentation
An instrumentation threat comes from a change in the measurement system rather than the behavior. Observers may drift from the definition over time (observer drift, see C.8), a new observer may score differently, or observers who know the hypothesis may score in the expected direction. Interobserver agreement checks, retraining and observers who do not know the phase help.
Extreme scores drift back: regression to the mean
Regression to the mean is the tendency of unusually high or low measures to be followed by measures closer to the person’s average. If an intervention starts right after a run of unusually bad days, behavior may improve with no help from the intervention. The single-case remedy is to wait for steady-state responding before changing conditions.
Earlier conditions affect later ones: multiple-treatment interference
When one person experiences several conditions, behavior in one condition can be shaped by experience in another. This is multiple-treatment interference, often discussed as sequence or carryover effects. It is a special concern in reversal designs with more than one treatment and in alternating treatments designs (see D.7). Counterbalancing the order and making each condition clearly distinct help. Sources classify this threat differently: Campbell and Stanley (1963) listed it under external validity, because a result found after several treatments may not hold when one is given alone, while single-case texts also treat it as a confound within the study.
Threats that mostly concern group designs
- Selection: the groups differed before treatment began. Because each person in a single-case design is their own control, this threat is largely avoided.
- Attrition: participants drop out, and those who leave differ from those who stay. It still matters in single-case research if participants who did not respond are left out of the report.
- Diffusion of treatment: the intervention leaks into a condition that is supposed to be untreated. In single-case work this appears in multiple-baseline designs, for example when a teacher starts praising behavior in a setting still in baseline.
Treatment integrity belongs on this list too: if the IV is delivered poorly, the study tests something other than what it describes (see C.12).
Seeing it in practice
| Situation | Threat |
|---|---|
| Aggression drops the week intervention begins, which is also the week a classroom aide is added. | History |
| A 3-year-old’s spoken vocabulary grows steadily across a six-month study, baseline included. | Maturation |
| A student’s scores on a weekly spelling probe improve across baseline because the same words are tested each time. | Testing |
| On-task scores rise after a new observer takes over, though video shows little change. | Instrumentation |
| Intervention starts the day after the worst tantrum week of the year, and tantrums decline at once. | Regression to the mean |
| In an alternating treatments comparison, Treatment B looks better only when it follows Treatment A. | Multiple-treatment interference |
Common mix-ups
- History vs. maturation. History is an event outside the person. Maturation is a change within the person over time.
- Testing vs. instrumentation. Testing means the behavior changed because it was measured. Instrumentation means the measurement changed while the behavior may not have.
- Regression is not a treatment effect. It is a statistical tendency, controlled by not starting treatment on an extreme data point.
- Treating one AB comparison as proof. A single change from baseline to intervention leaves history, maturation and regression open. The fix is replication: an outside event is unlikely to line up with every phase change in a reversal design, or with every staggered start in a multiple baseline.
Exam tip: Ask what else changed. An outside event points to history. Growing up or tiring points to maturation. Practice or being watched points to testing. A different observer or definition points to instrumentation. Starting after an extreme point points to regression. Earlier conditions affecting later ones point to multiple-treatment interference.
Quick recap
- Threats to internal validity are alternative explanations for a change in the DV.
- History (outside events) and maturation (change within the person) are the most tested.
- Testing, instrumentation, regression and multiple-treatment interference matter especially in single-case work.
- Selection and attrition are mainly group-design concerns; diffusion of treatment matters in multiple baselines.
- Steady baselines, repeated demonstrations of effect, interobserver agreement and integrity checks are the main controls.
Further reading
- Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Rand McNally.
- Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), chapter 10.
- Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). Oxford University Press.
Key terms
- Internal validity
- The extent to which a study shows that the independent variable, and not some uncontrolled factor, produced the change in the dependent variable. Designs that demonstrate experimental control have high internal validity.
- Confounding variable (Confound)
- An uncontrolled variable that changes along with the independent variable and may affect the dependent variable, so that its effect cannot be separated from the effect of the independent variable.
- History effect (History)
- A threat to internal validity in which an event outside the study, occurring at the same time as a change in conditions, produces or contributes to the change in behavior.
- Maturation
- A threat to internal validity in which changes within the participant that come with the passage of time, such as growing older, developing new skills, tiring or recovering from illness, produce the change in behavior.
- Testing effect (Testing)
- A threat to internal validity in which repeated measurement itself changes the participant's performance, through practice or through reactivity to being observed.
- Instrumentation
- A threat to internal validity in which the measurement system changes during a study, for example through observer drift, a new observer or a revised definition, so the data change even if the behavior does not.
- Regression to the mean (Statistical regression)
- The tendency of unusually high or low measures to be followed by measures closer to the person's average. It threatens internal validity when an intervention begins right after extreme data, because improvement may follow anyway.
- Multiple-treatment interference (Sequence effects, Carryover effects)
- A confound in which a participant's behavior in one condition is influenced by experience with another condition in the same study, either through the order of conditions (sequence effects) or through the lingering effects of an earlier condition (carryover effects). It is a concern in multielement designs and in reversal designs with more than one treatment.
Check yourself
5 questions on D.3. Choose an answer to see whether you're right and why.
Show the answer
Answer: B. The tutor is an outside event that began at the same time as the plan, so it could account for the change. That is history. Maturation would be a change within Mateo that comes from the passage of time, such as growing older.
Show the answer
Answer: C. The change appears in the measurement, not in the behavior: two observers score the same sessions differently. That is an instrumentation threat. A testing effect would be a real change in the client's behavior caused by repeated measurement.
Show the answer
Answer: C. Unusually extreme measures tend to be followed by measures closer to the person's average, with or without treatment. Starting right after the worst week, with no steady baseline, leaves this explanation open. Selection concerns differences between groups, which do not apply here.
Show the answer
Answer: A. Rosa's performance under one condition depends on her experience with the other condition, which is multiple-treatment interference, also called a sequence or carryover effect. It is not regression to the mean, because the difference follows the order of conditions rather than a return from one extreme score. Counterbalancing the order of conditions helps detect and control it.
Show the answer
Answer: B. Selection threats arise when the people in different groups differed before treatment. In a single-case design, each person is compared with their own baseline, so there are no separate groups to differ. Repeated measurement, not single measurement, is a feature of these designs.
Adds a step to your study record. Saved in this browser only.