cumulative

H.7 Data-based decisions about effectiveness and modification

What the task asks: Analyze ongoing data to decide whether an intervention is working and whether to continue it, change it, fade it or end it.

Selecting and Implementing Interventions5 min read5 practice questionsOfficial outline (opens in a new tab)

The big idea

Behavior analysts collect data so they can act on them. Ongoing, graphed data show whether an intervention is producing the change it should. Standard 2.17 (Collecting and Using Data) and Standard 2.18 (Continual Evaluation of the Behavior-Change Intervention) expect you to use those data to decide about continuing, modifying or ending services, and to take corrective action when goals are not being met.

The decision is rarely just “working” or “not working.” It is a choice among continuing, modifying, fading and discontinuing, made from the evidence in the graph and the context around it.

Breaking it down

Visual analysis basics

Visual analysis is the systematic inspection of graphed data to judge whether behavior changed and by how much (see C.11 and D.6). Within and across phases, look at:

  • Level: the value around which the data cluster, often summarized by the mean or median. Is the behavior higher or lower than in baseline?
  • Trend: the direction and steepness of the data path. Is the behavior moving toward the goal?
  • Variability: how much the data bounce around. High variability makes an effect harder to see and may point to uncontrolled variables.
  • Immediacy of effect: how quickly change appears after a phase change.
  • Overlap: how many intervention data points fall within the range of baseline.

Compare the data with the goal (H.1). An aim line, drawn from the starting level to the goal level by the target date, shows whether progress is fast enough.

Decision rules

A decision rule states in advance what to do when the data show a given pattern. Examples:

  • If three consecutive data points fall below the aim line, review and change the intervention.
  • If there is no improvement after six sessions, check integrity, then modify.
  • If the mastery criterion is met, fade supports and begin maintenance probes.
0510152002468101214SessionsRule met: modifyGoalAim lineRequests per session
A decision rule at work. The dashed aim line runs from the starting level to the goal. After three consecutive points fall below it, the team changes one part of the intervention, and the data climb back to the aim line.

Rules protect against two errors: changing too soon because of normal variability, and waiting too long while a client makes no progress. The exact rule should fit the behavior, how often data are collected and what is at stake.

The four decisions

Data patternDecisionTypical actions
Steady progress toward the goalContinueKeep the procedure; keep collecting data
Little or no progress, worsening, or very high variabilityModifyCheck integrity first (H.6); then examine reinforcers, prompts, prerequisites, motivating operations and task difficulty
Goal met and stableFadeThin reinforcement, fade prompts, shift to natural contingencies, check generalization and maintenance
Goals met overall, or no benefit despite reasonable changesDiscontinue or transitionPlan discharge or transition (E.6)
Continue On track with the aim line Aim line Modify Flat, well below the aim line Aim line Fade Goal met and stable Goal Continue Early burst, then below baseline Baseline level Sessions
Data patterns and the decision each calls for. Data on the aim line: continue. Flat data well below it: modify, after checking integrity. A goal met and held: fade. A spike early in extinction that then drops below baseline: continue, because it is an expected burst.

Making good modifications

  • Rule out integrity and measurement problems first. A flat line may reflect a plan that is not being followed or data that are not reliable.
  • Analyze the error pattern. In skill teaching, look at what the errors look like: always choosing the card on the right (position bias), waiting for prompts (prompt dependence), or errors only on new targets.
  • Change one thing at a time when possible, so you can tell what made the difference.
  • Allow for expected patterns. An extinction burst early in treatment is not a reason to stop (H.4).
  • Use other evidence too: generalization probes, social validity feedback, and the client’s assent and engagement.

Seeing it in practice

ClientDataDecision
Asha, 4, learning to request with a speech deviceIndependent requests rose from 2 to 15 per session over three weeks, on track with the aim lineContinue; begin probing requests at home
Tomás, 10, learning sight wordsAccuracy flat at 40% for eight sessions; integrity 95%; most errors on words that look alikeModify: teach similar words in separate sets and add error correction
Mrs. Lee, 70, walking program at a care homeDaily steps have met the goal for four weeksFade staff prompts; schedule weekly maintenance checks
Ben, 13, aggressionAggression spiked in the first week of FCT with extinction, then dropped below baselineContinue; the spike fits an expected burst

Common mix-ups

  • Trend and level are different. Data can sit at a better level than baseline yet trend back toward it, which may signal a fading effect.
  • One bad day is not a trend. Use decision rules rather than reacting to single points.
  • Modify does not mean replace. Often a small change in the reinforcer, prompt type or task size is enough.
  • No progress is not always the intervention’s fault. Check integrity, measurement and motivating operations first.
  • Meeting a goal is not the end. Fading, generalization and maintenance come next.

Exam tip: When the data show no progress, the strongest answer usually checks procedural integrity first. If integrity is high, look for the answer that changes a specific part of the intervention rather than abandoning it.

Quick recap

  • Use graphed data to decide whether to continue, modify, fade or discontinue.
  • Inspect level, trend, variability, immediacy and overlap, and compare the data with the goal.
  • Set decision rules in advance to avoid changing too soon or too late.
  • Rule out integrity and measurement problems before modifying the procedure.
  • Change one variable at a time, and keep collecting data after each change.

Further reading

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), chapters 6 and 7.

Key terms

Visual analysis (Visual inspection)
The systematic inspection of graphed data to judge whether behavior changed meaningfully and whether the design shows that the independent variable caused the change. It considers level, trend and variability within phases, and immediacy, overlap and consistency across phases.
Level
The value on the vertical axis of a graph around which a series of data points clusters, often summarized by the mean or median. A change in level between phases is a common sign of an intervention effect.
Trend
The overall direction of a data path (increasing, decreasing or flat) along with its steepness. A baseline trend toward the improvement the intervention is expected to produce makes any effect hard to interpret.
Variability
How much the data points in a phase differ from one another or from the trend line. The more variable the data, the more measurements are needed before a confident decision can be made.
Decision rule (Data-based decision rule)
A rule set in advance that specifies what action to take when the data show a particular pattern, such as changing the intervention after a set number of data points fall below the aim line.

Check yourself

5 questions on H.7. Choose an answer to see whether you're right and why.

  1. Question 1For eight sessions, Tomás's accuracy on sight words has stayed near 40%. Integrity checks show the plan is implemented at 95%. What is the best decision?
    Show the answer

    Answer: D. Flat data with high integrity mean the plan is being delivered as written and is not working, so it needs to change. Looking at the error pattern shows what to change. Retraining staff would address a problem the integrity data have already ruled out.

  2. Question 2Data on a client's aggression show a clear downward trend across the intervention phase, but the phase mean is still close to the baseline mean. Which statement is accurate?
    Show the answer

    Answer: D. Level and trend are separate features of a data path. A downward trend means aggression is decreasing over time, even if the average for the phase is similar to baseline so far. Judging only by the mean would miss the change in direction.

  3. Question 3A team writes: 'If three consecutive data points fall below the aim line, the plan will be reviewed and changed.' This statement is an example of:
    Show the answer

    Answer: B. A decision rule states in advance what action to take when the data show a given pattern. A mastery criterion defines when a skill counts as learned, which is a different question from when to change the plan.

  4. Question 4During the first week of FCT with extinction, Ben's aggression increases sharply, then drops below baseline levels in the second week. What should the team do?
    Show the answer

    Answer: D. A short increase followed by a clear decrease is the pattern expected from an extinction burst, and the current data show improvement. Changing or dropping parts of the plan because of the early spike would ignore the trend that followed it.

  5. Question 5Kiran, 12, has met the mastery criterion for washing his hands independently at the clinic sink for two weeks. What is the most appropriate next step?
    Show the answer

    Answer: D. Meeting a criterion in one setting is the signal to fade supports and check generalization and maintenance. Keeping the same prompts and reinforcement indefinitely risks dependence, and ending data collection would hide any loss of the skill.

Adds a step to your study record. Saved in this browser only.

Type to search all 104 guides and the glossary.