The big idea
Procedural integrity, also called treatment integrity or procedural fidelity, is the degree to which an intervention is carried out as designed. Measuring it means collecting data on the behavior of the people delivering the intervention, not only on the client.
Without those data, results cannot be interpreted. If the client improves, you cannot be sure the plan caused it. If the client does not improve, you cannot tell whether the plan failed or was never delivered as written. Poor integrity is also a threat to internal validity (D.3). This task is about collecting integrity data you can trust; deciding what to do with them is H.6.
Breaking it down
Dimensions worth measuring
- Accuracy (often called adherence): were the steps done as written, in the right way and at the right time? For example, delivering a token within 3 seconds of a correct response.
- Dosage: how much of the intervention was delivered compared with the plan, in sessions, minutes, trials or opportunities. For example, 12 of 20 scheduled hours.
The two can come apart. A teacher can carry out every step perfectly whenever they run a program but run it on only two of five scheduled days. Some frameworks add quality of delivery, the client’s engagement and how clearly the intervention differs from other conditions (Dane and Schneider, 1998).
Building the measure
- Task-analyze the protocol into observable steps, each written as carefully as an operational definition (C.1).
- Make a checklist. Score each step as correct, incorrect or not applicable (no opportunity occurred).
- Score per opportunity for steps that repeat, such as praise after each correct response.
- Calculate the percentage of steps implemented correctly: correct ÷ (correct + incorrect) × 100, leaving out the not-applicable steps. With 17 correct, 3 incorrect and 2 not applicable, integrity is 17 ÷ 20 = 85%.
- Look at each component as well as the total. An overall 85% can hide one step that is missed every time.
- Calculate dosage as delivered ÷ planned × 100.
Errors of omission and commission
- An error of omission leaves out a required step, such as forgetting to deliver a token.
- An error of commission adds something the plan does not include, or does a step the wrong way, such as giving attention after problem behavior that is on extinction, or adding extra prompts.
Record them separately because they do different damage. Translational research on differential reinforcement has found that accidentally reinforcing problem behavior can undermine treatment more than occasionally missing reinforcers for the replacement behavior.
Making the data representative
- Direct observation, live or by video, gives the most accurate data but costs observer time and can be reactive: people often implement better while watched.
- Permanent products, such as completed token boards, data sheets and reinforcer logs, are cheap but capture only some steps.
- Self-report and rating scales are easy but tend to overestimate integrity. They are indirect measures (C.2).
- Sample fairly. Spread checks across implementers, days, times, settings and phases, including baseline. Brief unannounced or video-based checks reduce reactivity.
- Measure repeatedly. Treatment drift, a gradual change in how the procedure is carried out, often appears weeks after training.
- Check agreement. A second observer can score the same session to confirm that the integrity data are themselves reliable (C.8).
Seeing it in practice
A BCBA scores an RBT’s 20-minute session with Keanu, who is learning to ask for a break instead of pushing materials away.
| Step | Opportunities | Correct | Errors |
|---|---|---|---|
| Presents the task instruction once | 10 | 10 | None |
| Prompts a break request when Keanu stops working for 5 s | 4 | 3 | 1 omission |
| Gives a break within 3 s of a break request | 8 | 6 | 2 omissions |
| Keeps the task in place, without comment, after pushing | 3 | 1 | 2 commissions (removed the task once, said “stop that” once) |
| Total | 25 | 20 | 20 ÷ 25 = 80% |
The overall score is 80%, but the last step, the one most likely to keep problem behavior going, was done correctly in only one of three opportunities (33%). The plan also calls for three sessions a day, and the log shows an average of two, a dosage of 67%. Each number points to a different fix, which H.6 takes up.
In a school, Ms. Brooks rates her own use of a token system as “always.” A 20-minute direct observation shows tokens delivered on 5 of 12 opportunities, about 42%. The self-report was no substitute for observation.
Common mix-ups
- Client data are not integrity data. Improvement does not show the plan was delivered as written.
- Accuracy is not dosage. Perfect sessions delivered half as often are still a problem.
- A high total can hide a failing step. Look at each component.
- Counting not-applicable steps as correct inflates the score. Leave them out.
- Integrity is not IOA. IOA asks whether observers agree on the data; integrity asks whether the implementer followed the plan.
- One check is not enough. Integrity at the end of training says little about integrity months later.
Exam tip: If a stem describes poor client progress, check whether integrity was measured before concluding that the intervention failed. If it asks how to measure integrity, look for direct observation with a step-by-step checklist, scored per opportunity and sampled across implementers and sessions.
Quick recap
- Procedural integrity is the degree to which an intervention is implemented as designed.
- Measure accuracy (steps done correctly) and dosage (how much was delivered), by component as well as overall.
- Percentage of steps correct = correct ÷ (correct + incorrect) × 100, leaving out not-applicable steps.
- Record errors of omission and commission separately.
- Direct observation sampled across people, times and phases beats self-report; repeat checks to catch treatment drift.
Further reading
- Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.), chapter 10.
- Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15(4), 477-492.
- Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23-45.
Key terms
- Procedural integrity (Treatment integrity, Treatment fidelity, Implementation fidelity)
- The extent to which a procedure is carried out exactly as designed. It is usually reported as the percentage of steps implemented correctly. Low integrity makes it impossible to tell whether an intervention works, and the same idea applies to whether supervision is delivered as planned.
- Dosage (Exposure)
- The amount of an intervention a client actually receives compared with the amount planned, such as the number of sessions, minutes or trials delivered. It is a dimension of procedural integrity separate from how accurately each step is carried out.
- Error of omission
- An implementation error in which a required step of a procedure is left out.
- Error of commission
- An implementation error in which the implementer adds something the procedure does not include, or carries out a step incorrectly. Commission errors that deliver reinforcement for problem behavior can be especially harmful to treatment.
- Treatment drift (Procedural drift)
- A gradual, unplanned change in how an intervention is carried out over time, so that the procedure in use no longer matches the one designed. Repeated integrity checks are the way to detect it.
Check yourself
4 questions on C.12. Choose an answer to see whether you're right and why.
Show the answer
Answer: C. Steps with no opportunity are left out, so integrity is 8 ÷ (8 + 2) × 100 = 80%. Counting them as errors gives 8 ÷ 12 ≈ 67%, and counting them as correct gives 10 ÷ 12 ≈ 83%; both distort the score. The 20% figure is the error rate, not the integrity score.
Show the answer
Answer: C. Mr. Alvarez is accurate when he runs the program, but his students receive only 2 of 5 planned sessions, or 40% of the planned dosage. Accuracy concerns how the steps are carried out, which is fine here. Interobserver agreement and social validity describe other things entirely.
Show the answer
Answer: C. Leaving out a required step, praising the request, is an error of omission. Doing something the plan does not call for, delivering a snack after screaming, is an error of commission, and it can reinforce the problem behavior. Recording the two separately points to different fixes.
Show the answer
Answer: A. Brief, unannounced checks spread across people, days and activities sample integrity fairly and reduce reactivity. Self-ratings tend to overestimate integrity, and one announced observation of the best implementer shows only best-case performance. Client progress is not a measure of integrity at all.
Adds a step to your study record. Saved in this browser only.