Evaluation of psychotherapeutic interventions.

-CRES-4 emotion recall, follow-up, and perceived change correlated highly with corresponding OQ-45 scores (CRES-4 captures improvement)--> naise!

-“CRES-4’s recall of emotional state war neg verzerrt--> die leute dachten also dass sie schlechter drauf waren als sie es in der erhebung vom oq45 eig waren

-im hier und jetzt haben sie sich im cres-4 aber besser dargestellt, als das auf grundlage der im post erhobenen oq45 daten eig sein sollte

--> gibt als ergebnis: 85% had CRES-4 perceived change scores that were higher than OQ-45 change scores 

Due to the large sample size of the study power is high to detect even small effects

Figures give a misleading impression and exaggerate results

The Consumer Reports Study is a big questionnaire study on consumer satisfaction with past professional treatment of personal problems or mental disorders

▪ The study used the simplest research design (correlational study with one retrospective assessment a long time after therapy happened). -> It has severe flaws and problems -> many alternative explanations for the results are possible -> very low internal validity

▪ It cannot be an alternative or even a supplement to RCT studies

▪ With such an approach to effectiveness we do not gain valid knowledge about whether treatments really work in the field

effectiveness-study

Examining transportability of high-density cognitive-behavioral
in vivo exposure (HDE) by comparing effectiveness and clinical
significance at:
• Post treatment
• 1 year follow up
▪ Investigating generalizability of results by examining
differences between:
• Clinics
• Therapists with different levels of experience
▪ Exploring potential predictors of outcome
▪ Comparing Dropouts with Completers

692 Patients began treatment
▪ 416 Patients from three clinics of the Christoph-Dornier Foundation (CDS) (Marburg,
Braunschweig und Dresden)
▪ https://www.christoph-dornier-stiftung.de/therapie.html
• With Pre Post data
• 67 % women
• Ø 35.6 years
• Duration of disease Ø 8.4 years
• 95% patients received already treatment for panic disorder with agoraphobia
− 19% Antidepressant medication
− 37% Anxiolytics
− Only 35% without medication 692 Patients began treatment
▪ 416 Patients from three clinics of the Christoph-Dornier Foundation (CDS) (Marburg,
Braunschweig und Dresden)
▪ https://www.christoph-dornier-stiftung.de/therapie.html
• With Pre Post data
• 67 % women
• Ø 35.6 years
• Duration of disease Ø 8.4 years
• 95% patients received already treatment for panic disorder with agoraphobia
− 19% Antidepressant medication
− 37% Anxiolytics
− Only 35% without medication

High-density cognitive-behavioral in vivo exposure (HDE)
• Confrontation with fear inducing stimuli for several hours a day
▪ Three phases:
• 1. Psychological and medical examination
− 4 to 6 sessions
• 2. Diagnostic feedback und cognitive preparation for exposure
− Patient is given 1 or 2 weeks to decide whether to participate
• 3. HDE
− Individual adjustment of treatment to patient’s needs
– In the beginning: Confrontation with most difficult situation with support
of therapist
– Later self initiated confrontation
– Duration of treatment 4 to 10 days
– Up to 12 h confrontation per day
▪ Ø 36.2 sessions (50-min) (SD = 17)

Pre
▪ Post
• 6 weeks after termination
▪ Follow up
• 1 year
• 7 years (results published in a dissertation)

Diagnostic Interview (DIPS)
▪ Beck Anxiety Inventory (BAI)
• → Anxiety symptoms
▪ Agoraphobic Cognition Questionnaire (ACQ)
• → Anxiety and agoraphobic cognitions (Heart attack)
▪ Body Sensation Questionnaire (BSQ)
• → Physical symptoms
▪ Mobility Inventory (MI)
• → Avoidance behavior
▪ Beck Depression Inventory (BDI)
• → Depressive Symptoms
▪ Symptom Checklist-90-Revised (SCL-90-R)
• → Mental health symptoms and distresses

692 Patients started treatment
• 90 Dropouts (13%) after cognitive preparation
• 59 Dropouts (8.5%) during exposition
• 104 were still in treatment, no post measures available
▪ 439 Completers (table 1, p. 379)
▪ 416 Completers with pre – post assessment (p. 376)
• N = 354 to 401 available for pre-post analyses (table 2, p = 379)
• N = 263 to 293 available for pre-follow-up analyses (table 2, p = 379)
▪ Three sig. differences between Completers and Dropouts (ANOVA and chi2-
Test):
• Dropouts had
− longer duration of disease
− higher pre treatment-BDI-Scores
− lower education

Intra group effect sizes ((pre – post) / SD pre) were large: 0.93
to 1.82, average d = 1.23
-> Sig. Improvements in all measures (p<.001)

Reliable Change (RC)
• based on: (pre score – postscore) / SD of differences -->• Ø Improvement 79%

Clinical Significance (CS)
•From dysfunctional to → functional values that are within the norm • Ø 59% Improved

Patients’ Satisfaction at follow up

Patients and therapists evaluated patient’s improvement on a 7
point scale
▪ Improvement (better or much better)
• Patients 78%
• Therapists 77%
▪ no change
• Patients 4.4%
• Therapists 5.6%
▪ deterioration
• Patients 4%
• Therapists 1.8%

Therapists tend to oversee when patients get worse

No difference between clinics
• Equal implementation
▪ No difference between less and more experienced therapists
• Less experienced therapists are effective

Treatment
− conducted in a non-university setting (+)
− payed by patients health insurance (+)
− implementation was not monitored (+)
• Patients
− referred through usual clinical routes not by experimenter (+)
− were heterogeneous in personal characteristics (+)
− But homogeneous with regard to the primary diagnosis (PDAG) (-)
• Therapists
− did not use a treatment manual (+)
− free to use a variety of procedures (+)
− not restricted to a fixed number of sessions (+)
− only 50% experienced, professional therapists with regular caseloads (-)

Pre-Post and Pre-Folluw up results show high effectiveness of HDE treatment
• Limitations: No control group, but
− spontaneous remission in Panic disorder patients with agoraphobia is unlikely
− effect sizes comparable to findings in efficacy studies -> weakens alternative explanations

Many drop outs ( 21 %) and missing data at post and even more at follow up : No intent to
treat analysis was performed
• -> Completer analysis probably overestimates effect sizes

Assessment: Standardized self reports
• independent blind assessor ratings should be included; however
− Expensive and difficult to implement in ongoing clinical setting
− Meta analysis showed that effect-sizes of clinical ratings exceeded those from self reports

Dropout analysis suggests that High-density cognitive-behavioral in vivo exposure needs to be improved

Well conducted effectiveness study.
▪ Despite limitations due to the design, conclusions can be drawn by
comparing results with those from efficacy studies:
• Such a comparison is possible, because standardized measures were used at pre-,
post- and follow-up, that were also used in RCTs
▪ HDE can be successfully transferred into clinical setting
▪ HDE can be successfully trained
▪ Results are generalizable to settings and treatment conditions in CDS clinics
▪ HDE needs to be improved for those patients who have high risk for drop out
• Depression, low education, long duration of disease

HDE=High-density cognitive-behavioral in vivo exposure

ince Eysenck’s (1952) critical review about efficacy of
PT many RCTs had been conducted:
− Many non-consistent results,
− difficult to summarize, integrate, and draw conclusions.
• Glass and Kliegl (1983, p. 28) describe the situation:
“The outcome literature had splintered and disintegrated;

--> gab also grundlegend viele srudien mit wenig qualität. also wenig guten kennwerten welche für ne andewendung von psychotherapie sprechen

ähnlich wie paarvergelcih aus statistik 1; man hat also psychotherapie disziplinen untereinander vergleichen

nur 1 von 4 sagt dass kvt besser als psychodym ist

medis psychodym. ausgeglichen

medis kv ungefähr gleich

Problem: disziplinen wurden auf grundlage von signifikanztests miteinander vergleichen; in den signiwerten steckt neben der effektstärke allerdings auch immer die anzahl der vpn drin("mit hoher anzahl vpn wird auch n kleiner effekt signi") deswegen schwierig verschied. studien mit verschiednene vpn anzahlen mittels dieses t wertes miteinander zu vergleichen

Determine state of knowledge about the effects of PT“
(Smith et al., 1980, p. 55):
• comprehensive
• objective
• unbiased

mittels anwendung einer metanalyse welche diese vorteile bringt:

Systematic method to summarize and quantify results
reported in primary studies
Provides estimates of size and robustness of
intervention-effects (or any other effect of interest)
Allows identification of moderators
Provides systematic information about the field of
research

All controlled PT-outcome studies conducted between 1900-
1977
• Any form of PT:
• Clients with emotional or behavioral problems
• Treatment should ameliorate problems of clients and was
psychological or behavioral
• Treatment was performed by psychotherapists (profession or
training)

Studies evaluating the following treatments were excluded:
• Pharmacological Treatment
• Bibliotherapy
• Sensitivity Training
• Consciousness raising groups
• Encounter groups
• Education and training programs
• Peer counseling
• Electro Shock treatment
• Occupational or recreational therapy

Publication date
• Publication Type
• Experimenter
• „Blinding“
• Diagnosis
• Hospitalization
• Intelligence
• Client-therapist similarity
• Social Status
• Assignment to groups
• Drops-outs
• Internal Validity
• Allegiance of Researcher(s)
• Treatment classification
Therapy modalities
• Setting
• Duration
• Experience of therapist
etc

Coding of outcome type
• 1 Fear or Anxiety, 2 Self esteem, 3 global adjustment, ... 13 other
▪ Evaluation of reactivity of outcome:
• Rating 1 = low to 5 = high
▪ Effect size (ES): Glass’ ∆ = specific variant of Cohen’s d
• ES = (Mtherapygroup - Mcontrol group) / SD CONTROL GROUP
• ES calculated for each available measure

Grund hierfür: glass und ned cohen; sd mit der control gruppe, da man von ausgegangen ist dass diese sd ne grundpopulation besser abbilidet als die sd der experimentalgruppe, wlche ja wilder ist da die therapue bei manchen ja angeschagen hat und bei anderen wiederum nicht

a) behavioral:

-operant-classical-behavioral

-cognitive behavioral

b) verbal

-cognitive

-dynamic

-humanistic

475 Studies were included
• 75% PTs, 25% School psychology treatments
▪ 1766 Effect Sizes
• 3.7 ES per Study
▪ Therapy Duration: 1 to 300 sessions: M=16.18 (SD=26.56)
• 2/3 less than 13 sessions
▪ Assessment: (0-300 weeks after treatment)
• 2/3 Post measures
▪ Client’s gender : 47.95% Male
▪ Client’s Age: M=22.91 Jahre (SD=9.02 Jahre)

Average ES = 0.85 PT vs. control group
▪ Average ES = 0.93 when placebo or undifferentiated
counseling were excluded
▪ 9% negative ES

ES = 0.85 is converted into r = .39
▪ Treatment condition success rate is .50 plus r/2 = .50 + .39/2 = .695
▪ Control condition success rate is .50 minus r/2 = .50 - .39/2 = .305
▪ The correlation r of 0.39 is the difference between the success rates of the treatment vs.
the control group (.695 − .305 = .39).

Cell entries in the BESD are
multiplied by 100--> mproved

erster wert= improved, 2. wert not improved, 3. wert total
Control 30.5 69.5 100
Treatment 69.5 30.5 100
100.0 100.0 200

zur erinnerung: average effect size = .85

other cognt. therapie(correcting faulty beliefs, grundlage rational emotive therapie welche per se nur .68 hatte) 2.38 SD=2.05 no. of effects 57

behavior modification .73 SD=.67 no. of effects 201

psychodynamic therapy .69 SD=.50 no. of effects 108

placebo effect .56 SD=.77 no. of effects 200

simple phobic: behavioral 1.01 cognitive 1.82 cognitive-behavioral 1.71

delinquent-felon: dynamic 1.49

depressive: behavioral 1.18

humanistic + developmental--> überall kacka bzw. wenige studien

2/3 of assessments at
post measurement
▪ 1/3: 4 weeks to 10
years
▪ High association
between time of
follow-up and ES: R =.78
▪ Effect declined during follow up

No linear relationship between ES and Duration of PT: r = -.05

fear-anxiety mit 1.12

personality traits .31 --> macht sinn weils eher schwieirig ist b5 merkmale umzukrempeln

low: .78

middle: .78

high: .88

High rating of IV:
▪ When assignment of patient
to condition was random
▪ When drop outs were less
than 15%, and equal between
groups
▪ When intent to treat analyis
was performed
Medium rating of IV
▪ Randomization but high
differential mortality ...
▪ Randomization failed
▪ Well done matching

No substantial correlations of different variables with ES, except for
reactivity (the higher reactivity of the measures the larger the ES) --> allerdings nur bei .18

verb. develop--> verb.: .51

behav. develop--> behav.: .95

behav. verb. --> behav.: .96

Meta Analysis provides a comprehensive summary of the
current state of psychotherapy research
▪ Psychotherapy is efficacious (better than waitlist or placebo)
▪ In simple comparisons:
(C)BTs had largest ESs
▪ In controlled analyses:
No reliable differences between PTs
-> Confirms „Dodo Bird“ that all therapies, no matter which
theoretical background they have, are efficacious

Effect Sizes depended on:
• Outcome-Type
− Anxiety measures were associated with big ES,
− personality and performance measures were associated with small ES
• Reactivity of outcome
− ES was higher the higher the reactivity of the measure
• Time of measurement
− ES was smaller for Follow up measures
• Experimenter allegiance
− ES was higher when experimenter was identified with PT
▪ ES did not depend on:
• Therapy duration
• Internal validity
• Patient variables

• In sum Eysenck (1978) sees major problem in primary
studies included into the meta-analysis
• “Only better-designed experiments than those in the literature
can bring us a better understanding on the points raised”
• Necessary: RCTs with
− placebo groups to study therapy-specific effects
− several therapists for each method

Takes subjective reports of therapists as source of
information
• Subjective, unvalidated, and certainly unreliable clinical
judgments

Initial differences between patients
• Patients for psychoanalysis more selective than patients for
behavior therapy
− higher intelligence, emotional resources, ego strength, etc.
-> Much more likely to improve spontaneously.