cumulative

C.12 Measuring procedural integrity

What the task asks: Choose how to measure whether an intervention is carried out as written, capturing the dimensions that matter (such as accuracy and dosage) in a way the setting can support.

Measurement, Data Display, and Interpretation5 min read4 practice questionsOfficial outline (opens in a new tab)

The big idea

Procedural integrity, also called treatment integrity or procedural fidelity, is the degree to which an intervention is carried out as designed. Measuring it means collecting data on the behavior of the people delivering the intervention, not only on the client.

Without those data, results cannot be interpreted. If the client improves, you cannot be sure the plan caused it. If the client does not improve, you cannot tell whether the plan failed or was never delivered as written. Poor integrity is also a threat to internal validity (D.3). This task is about collecting integrity data you can trust; deciding what to do with them is H.6.

Breaking it down

Dimensions worth measuring

  • Accuracy (often called adherence): were the steps done as written, in the right way and at the right time? For example, delivering a token within 3 seconds of a correct response.
  • Dosage: how much of the intervention was delivered compared with the plan, in sessions, minutes, trials or opportunities. For example, 12 of 20 scheduled hours.

The two can come apart. A teacher can carry out every step perfectly whenever they run a program but run it on only two of five scheduled days. Some frameworks add quality of delivery, the client’s engagement and how clearly the intervention differs from other conditions (Dane and Schneider, 1998).

Building the measure

  1. Task-analyze the protocol into observable steps, each written as carefully as an operational definition (C.1).
  2. Make a checklist. Score each step as correct, incorrect or not applicable (no opportunity occurred).
  3. Score per opportunity for steps that repeat, such as praise after each correct response.
  4. Calculate the percentage of steps implemented correctly: correct ÷ (correct + incorrect) × 100, leaving out the not-applicable steps. With 17 correct, 3 incorrect and 2 not applicable, integrity is 17 ÷ 20 = 85%.
  5. Look at each component as well as the total. An overall 85% can hide one step that is missed every time.
  6. Calculate dosage as delivered ÷ planned × 100.

Errors of omission and commission

  • An error of omission leaves out a required step, such as forgetting to deliver a token.
  • An error of commission adds something the plan does not include, or does a step the wrong way, such as giving attention after problem behavior that is on extinction, or adding extra prompts.
The plan What the RBT did Instruction Prompt a break request Break within 3 s Task stays, no comment Instruction Nothing Break within 3 s Says "stop that" Omission Commission
Two errors from Keanu's session set against the plan. Leaving out the prompt is an error of omission; commenting after Keanu pushes materials away, which the plan rules out, is an error of commission.

Record them separately because they do different damage. Translational research on differential reinforcement has found that accidentally reinforcing problem behavior can undermine treatment more than occasionally missing reinforcers for the replacement behavior.

Making the data representative

  • Direct observation, live or by video, gives the most accurate data but costs observer time and can be reactive: people often implement better while watched.
  • Permanent products, such as completed token boards, data sheets and reinforcer logs, are cheap but capture only some steps.
  • Self-report and rating scales are easy but tend to overestimate integrity. They are indirect measures (C.2).
  • Sample fairly. Spread checks across implementers, days, times, settings and phases, including baseline. Brief unannounced or video-based checks reduce reactivity.
  • Measure repeatedly. Treatment drift, a gradual change in how the procedure is carried out, often appears weeks after training.
  • Check agreement. A second observer can score the same session to confirm that the integrity data are themselves reliable (C.8).

Seeing it in practice

A BCBA scores an RBT’s 20-minute session with Keanu, who is learning to ask for a break instead of pushing materials away.

StepOpportunitiesCorrectErrors
Presents the task instruction once1010None
Prompts a break request when Keanu stops working for 5 s431 omission
Gives a break within 3 s of a break request862 omissions
Keeps the task in place, without comment, after pushing312 commissions (removed the task once, said “stop that” once)
Total252020 ÷ 25 = 80%

The overall score is 80%, but the last step, the one most likely to keep problem behavior going, was done correctly in only one of three opportunities (33%). The plan also calls for three sessions a day, and the log shows an average of two, a dosage of 67%. Each number points to a different fix, which H.6 takes up.

0%25%50%75%100%100%Instruction75%Prompt75%Break33%Task stays80%OverallPercent correct
Keanu's session scored step by step. The overall 80% looks acceptable, but keeping the task in place after pushing was done correctly in only 1 of 3 opportunities.

In a school, Ms. Brooks rates her own use of a token system as “always.” A 20-minute direct observation shows tokens delivered on 5 of 12 opportunities, about 42%. The self-report was no substitute for observation.

Common mix-ups

  • Client data are not integrity data. Improvement does not show the plan was delivered as written.
  • Accuracy is not dosage. Perfect sessions delivered half as often are still a problem.
  • A high total can hide a failing step. Look at each component.
  • Counting not-applicable steps as correct inflates the score. Leave them out.
  • Integrity is not IOA. IOA asks whether observers agree on the data; integrity asks whether the implementer followed the plan.
  • One check is not enough. Integrity at the end of training says little about integrity months later.

Exam tip: If a stem describes poor client progress, check whether integrity was measured before concluding that the intervention failed. If it asks how to measure integrity, look for direct observation with a step-by-step checklist, scored per opportunity and sampled across implementers and sessions.

Quick recap

  • Procedural integrity is the degree to which an intervention is implemented as designed.
  • Measure accuracy (steps done correctly) and dosage (how much was delivered), by component as well as overall.
  • Percentage of steps correct = correct ÷ (correct + incorrect) × 100, leaving out not-applicable steps.
  • Record errors of omission and commission separately.
  • Direct observation sampled across people, times and phases beats self-report; repeat checks to catch treatment drift.

Further reading

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), chapter 10.
  • Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15(4), 477-492.
  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23-45.

Key terms

Procedural integrity (Treatment integrity, Treatment fidelity, Implementation fidelity)
The extent to which a procedure is carried out exactly as designed. It is usually reported as the percentage of steps implemented correctly. Low integrity makes it impossible to tell whether an intervention works, and the same idea applies to whether supervision is delivered as planned.
Dosage (Exposure)
The amount of an intervention a client actually receives compared with the amount planned, such as the number of sessions, minutes or trials delivered. It is a dimension of procedural integrity separate from how accurately each step is carried out.
Error of omission
An implementation error in which a required step of a procedure is left out.
Error of commission
An implementation error in which the implementer adds something the procedure does not include, or carries out a step incorrectly. Commission errors that deliver reinforcement for problem behavior can be especially harmful to treatment.
Treatment drift (Procedural drift)
A gradual, unplanned change in how an intervention is carried out over time, so that the procedure in use no longer matches the one designed. Repeated integrity checks are the way to detect it.

Check yourself

4 questions on C.12. Choose an answer to see whether you're right and why.

  1. Question 1An observer scores a technician's session on a 12-step checklist: 8 steps correct, 2 incorrect and 2 with no opportunity to occur. What is the integrity score?
    Show the answer

    Answer: C. Steps with no opportunity are left out, so integrity is 8 ÷ (8 + 2) × 100 = 80%. Counting them as errors gives 8 ÷ 12 ≈ 67%, and counting them as correct gives 10 ÷ 12 ≈ 83%; both distort the score. The 20% figure is the error rate, not the integrity score.

  2. Question 2Mr. Alvarez carries out every step of a reading intervention correctly whenever he runs it, but logs show he runs it on 2 of the 5 scheduled days each week. Which dimension of procedural integrity is the problem?
    Show the answer

    Answer: C. Mr. Alvarez is accurate when he runs the program, but his students receive only 2 of 5 planned sessions, or 40% of the planned dosage. Accuracy concerns how the steps are carried out, which is fine here. Interobserver agreement and social validity describe other things entirely.

  3. Question 3Under a DRA plan, Ms. Park sometimes forgets to praise her daughter's polite requests and sometimes gives a snack after screaming. How should these errors be classified?
    Show the answer

    Answer: C. Leaving out a required step, praising the request, is an error of omission. Doing something the plan does not call for, delivering a snack after screaming, is an error of commission, and it can reinforce the problem behavior. Recording the two separately points to different fixes.

  4. Question 4A BCBA supervising a classroom team has limited time. Which plan for collecting procedural integrity data is most likely to give representative results?
    Show the answer

    Answer: A. Brief, unannounced checks spread across people, days and activities sample integrity fairly and reduce reactivity. Self-ratings tend to overestimate integrity, and one announced observation of the best implementer shows only best-case performance. Client progress is not a measure of integrity at all.

Adds a step to your study record. Saved in this browser only.

Type to search all 104 guides and the glossary.