Evaluation of psychotherapeutic interventions.

Network meta-analysis is a technique for comparing three
or more interventions simultaneously in a single analysis by
combining both direct and indirect evidence across a
network of studies

--> also kein direkter vergleich (e.g. ) ipt vs. gestalt therapy sondern man schaut sich ne drittvariable an (e.g.) waitlist und schaut wie die beiden therapien jeweils ggü. der drittvariable abgeschnitten haben; darauf basierend kann man dann nen indirekten vergleich ziehen (stell dir dat dreieck vor)

Estimates of the effects of pairs of treatments that have often,
rarely, or never been directly compared in a RCT can be calculated.
As a consequence, network meta-analysis overcomes some of the
limitations of traditional meta-analysis, in which conclusions are
largely restricted to comparisons between treatments that have
been directly compared in RCTs

Re-examining the comparative efficacy of different
psychotherapeutic interventions for adult depression
▪ by combining all available information with network meta-
analysis
▪ Assessing how effect estimates are influenced by
• study quality
• sample size of studies
• clinical characteristics

All RCTs that
• compared effects of PT to a control condition
− Waitlist
− Usual care
− Placebo
• or compared effects of two PTs to another
 Patients
• Adults with a depressive disorder, or
• with elevated levels of depressive symptoms
 Psychotherapeutic interventions were defined as:
• interventions with primary focus on language based communication between a
patient and a therapist, or
• bibliotherapy supported by a therapist

▪ No language restrictions and restrictions on publication type

Combinations of psychotherapeutic interventions with
pharmacotherapy or other non-psychotherapeutic
interventions (e.g., managed care interventions and disease
management programs)
▪ Comparisons of a psychotherapeutic intervention with
pharmacotherapy or other non-psychotherapeutic
interventions
▪ Maintenance treatment and relapse prevention

63 studies with two or more psychotherapeutic interventions
162 studies with psychotherapeutic intervention(s) and control conditions
total of 15118 patients

regular dpression(94)

geriatric depression(26)

student populations(8)

postpartale depression(16)

medical patients with depression(27)

misscelaenous(27)

IPT

BA
CBT
problem solving therapy

social skills training

psychodynamic therapy

supportive counseling

control: placebo, usual care, waitlist

Concealment of allocation (randomisierte zuteilung)

Blinding of outcome assessment (randomisierte ergebnisbewertung, rater blind oder halt nicht)

Type of analysis--> hier schon inadequate wenns keine intent to treat analysis gab oder einfach random completers ausgeschlossen wurden

Depressive symptoms at post treatment were used as outcome.
• When results of post treatment measurement were not reported, the
earliest follow-up measurement was taken.
• Results from intention-to-treat analysis were preferred over results
from completer analyses.
• All standardized self-report and observer-rated instruments
measuring depressive symptoms were extracted.
• If results for more than one instrument were reported, the mean of
the effect sizes was computed.
-> only one effect size for each comparison of conditions per study

Network meta-analysis allows comparison of all conditions
evaluated in a connected network of studies, and accounts for
multiple comparisons from one study
• When there is no direct comparison available, the effect is estimated
based on the total information available in the network
− E.g. there was no study that compared waitlist with placebo.
− However, 16 studies compared CBT to placebo (d = 0.49) and 61 other
studies compared CBT to waitlist (d = 0.85).
− Combining this information, the difference between waitlist and
placebo could be estimated to be d = 0.85 – 0.49 = 0.36.
− Taking further comparisons and number of comparisons and sample
size into account, the effect between placebo and waitlist was
estimated to be: d = 0.33 (table 3)

All 7 interventions were significantly more beneficial than
▪ waitlist (d = 0.62 to 0.92).
▪ usual care (d = 0.29 to 0.59).
• Effect of social skills training was d = 0.30, but not significant,
due to the small number of studies (7)
▪ Placebo (d = 0.29 to 0.58).
• Effect of social skills training was d = 0.30, but not significant

IPT:-.92 (-1,14 -  -0,68)

BA: -.80 (-1,08 -  -0,51)

CBT: -.78  (-.91 -  -.64)

problem solving therapy: -.74 (-.97 -  -.50)

psychodynamische therapie: -0.72 (-1.02  -  -0,41)

social skilz training:  -.62  (-1.19 -  -.10--> brink zur nonsignifikanz)

supportive counseling: -.62 (-.82 -  -.40)

Most relative effects of psychotherapeutic interventions
were absent to small (range d = 0.01 to d= 0.29) and not
significant
▪ Only interpersonal therapy was significantly superior to
supportive counseling (Δd = 0.30)
• Significance: confidence intervals of IPT did not include the
mean of SUP and vice versa

No significant differences in effects sizes regarding
• Regular depressed patients vs specific population: Δd = 0.08
• Formal diagnosis vs probable depression: Δd = -0.11
• Intervention format: individual and face to face vs others: Δd =
0.07)
▪ Treatment dose
• Unexpectedly, low-dose treatments were more effective in a
stratified analysis (Δd = 0.33). However no sign. effect, when
other variables were controlled

Blinding of assessment
• Effects were smaller in studies where outcome was assessed
with self-report measures or blinded observers vs non-blind
observers (Δd = 0.38)
▪ Concealment of allocation
• Slightly smaller effects (not significant) in studies with
adequately concealed randomization vs studies with
inadequate or unclear concealment (Δd = 0.19)
▪ Type of analysis
• No sig. difference between studies with intention-to-treat
analyses vs studies with completer analysis (Δd = 0.13)

Year of publication had no effect (< 2000 vs ≥ 2000: Δd =
0.14)
▪ Size of the study was significant
• small to moderate vs large = average group size ≥ 50: Δd = 0.33
• Effects were larger the smaller the sample size was

--> spricht für nen publication bias(kleine studys mussen nen hohen effekt haben um signi zu werden große können auch nen kleinen haben; viele von den nonsignifikanten wurden wahrscheinlich ned veröffentlicht

IPT: -.73 (-1.14  -  -0.32)

BA --> gab überhaupt keine studien

CBT: -.57 (-.80 - -.35)

problem solving therapy: -.46  (-.81 - -.12)

social skills training --> gab keine studie zu

psychodynamische --> nicht signi

supportive counseling --> nicht signi

--> wenn überhaupt gabs also bei ipt, cbt und problem solving gerade mal nen moderaten effekt

Results were based on 198 RCTs
▪ Compared to waitlist, placebo produced small to moderate effects that were equal to
usual care (d = 0.33)
▪ Seven psychotherapeutic interventions had comparable effects on depressive
symptoms and achieved moderate to large effects compared to waitlist (d > 0.62).
▪ All seven psychotherapeutic interventions achieved small to moderate effects
compared to placebo or usual care (d > 0.29).
▪ Differences between treatments were small and not significant (d < 0.30).
Only interpersonal therapy was significantly superior to supportive counselling.
Comparable results have been found in the first (Smith et al., 1980) and recent meta-
analyses of psychological treatments for patients with major depression (Cuijpers et
al., 2014), anxiety disorders (Tolin, 2014), social anxiety disorders (Mayo-Wilson et al.,
2014), various disorders (Marcus et al., 2014). See also Wampold et al. (2017) for a
critical discussion.
▪ -> Under similar conditions (same patients, same duration, same assessment,
standardized and manualized application of therapy) psychological treatments do not
or only slightly differ in their efficacy to reduce symptoms

Exploration of study characteristics in the meta-analysis of Barth et al.
suggests that psychotherapeutic interventions work similarly well in different
populations of depressed patients, with different formats (individual face to
face vs others).
However, categories for the examination of study characteristics for which
effects were explored were too broad or too unspecific (e.g. format:
individual face to face vs all others = face to face group therapy, online
therapy, self help)
▪ Non-blind outcome assessors, inadequate concealing of the randomization
sequence, tend to produce larger outcome effects
-> might be due to researcher allegiance
▪ The smaller the study the larger the effects
-> might be due to publication bias
• It is much easier to publish results when they are significant. In studies with
smaller sample size effects need to be larger to become significant. Studies with
effects that are not significant do rather not get published and are therefore rather
not considered in meta-analyses

Only post assessment was considered
• No information about long term effects, which are more important
▪ Category of usual care may have merged different treatments of
different intensity. This might also be true for psychotherapy classes
• Violates assumption of network meta-analysis that treatments belonging
to on category are interchangeable -> might bias results
▪ Categories for the examination of study characteristics for which effects
were explored were too broad or too unspecific
▪ Confounding effects were only explored (e.g. size of study, blinding,
duration of treatment), but could not be adequately controlled -> might
bias effect size estimates
▪ Researcher allegiance bias was not examined
▪ Severity of depression was not considered (only formal diagnosis (71
studies) vs no formal diagnosis (127 studies)). Probably most studies
included patients with subclinical or mild depression
-> Clinically, it would be more important to know the effects of
treatments for moderate and severely depressed patients

Uncertainty about status of psychoanalytic and psychodynamic treatments
• Some evidence for efficacy of short-term psychodynamic psychotherapy (STPP) for
specific disorders
• Efficacy of long-term psychodynamic psychotherapy (LTPP) unclear
▪ For patients with complex disorders short-term PTs are insufficient
• multiple or chronic mental disorders
• patients with personality disorders
• -> greater deficits in social- and occupational functioning
▪ For such patients short term PT seems not to be sufficient
▪ -> they probably need higher dosage
▪ -> indication for Long term PT
▪ Some studies suggest that LTPP may be helpful for these patients
▪ However, no strong evidence-based support yet. No meta analysis

Is LTPP superior to other (shorter) PTs, particularly for patients
with complex mental disorders?
▪ How effective is LTPP with regard to different outcome domains
(e.g. target problems, personality- or social functioning)?
▪ What patient-, treatment-, or research- factors contribute to LTPP
outcome?

LTPP uses rather interpretive or supportive interventions
depending on patient’s needs.
• involves careful attention to therapist-patient interaction
• thoughtfully timed interpretation of transference and resistance
• Embedded in appreciation of therapist’s contribution to the two
person field
▪ Duration:
• at least 1 year or
• 50 sessions

Studies of individual PPT meeting definition
• ≥ 1 year, or ≥ 50 sessions
• Prospective studies: before-and after or follow-up assessments
• Reliable outcome measures
• Clearly described sample of patients with mental disorders
• Adult patients (≥ 18 years);
• Sufficient data to allow determination of effect sizes
• Concomitant treatments were admissible (e.g. psychopharmacological)
• RCTs and observational studies (one group pre-post (follow up) design)
• -> Author’s claim that criteria were consistent with recent meta-
analyses of PT

--> keine randomisierung also gehfahr für ne regression to the mean

Studies published between 1960 and May 2008
• MEDLINE, PsycINFO, Current Contents
• Manual searches of articles and textbooks
• Communication with authors and experts in the field
• To reduce file-drawer effect

Headges D = ((M pre treatment -M post treatment)/SD pre Treatment) * J

J=factor corrects for bias due to small sample size

Grund: nicht alle integrierten Studien hatten ne kontrollgruppe(lel) deswegen musste man darauf zurückgreifen

Between groups effect size (LTPP vs control or other PT)
= Point biserial correlation
− calculated between within group ES and dummy variable that
indicates LTPP (= 1) and control or other PT condition (= 0)

Effect sizes were computed separately for
▪ target problems
▪ general psychiatric symptoms
• broad measures of psychiatric symptoms (SCL-90)
• specific measures of non target symptoms
▪ personality functioning
▪ social functioning (Social Adjustment Scale)
▪ overall outcome (average effect across domains)
▪ assessments at termination
▪--> and follow-up; with longest follow-up period
▪ Intent to treat data preferred before completer data

23 studies were included
• 11 RCTs:
− 8 RCTs compared LTPP vs. comparative treatments:
– CBT, cognitive-analytic therapy, dialectical behavioral therapy, family therapy, supportive
therapy, short-term psychodynamic therapy, psychiatric treatment as usual
− 3 RCTs compared two variants of LTPP
• 12 observational studies
− 5 with non randomized control group
− 7 without control group
Of the 23 studies:
• 16 studies with LTPP alone without any concomitant psychotropic medication
• 7 studies, patients received concomitant psychotropic medication
• 12 studies with treatment manuals or manual-like guidelines
▪ 1053 patients with LTPP
▪ Number of sessions LTPP: M = 151 (SD= 155), Median = 74
▪ Duration of therapy LTPP: M = 95 ( SD = 59) weeks, Median = 69
▪ Follow-up period: M = 93 (SD = 65) weeks, Median = 65

In small sample studies only large effects get significant.
• Studies with significant effects more likely get published. Therefore, a negative correlation
between sample size and effect size would indicate a publication bias.
• The authors reported a non significant Spearman rank correlation (p > .30) between effect
size and sample size across studies
▪ Fail-safe N (FSN; Rosenthal)
• number of nonsignificant, unpublished, or missing studies that would need to be added to a
meta-analysis in order to change results from significance to non-significance.
• In the total sample of studies examining LTPP alone, FSN was:
− 921 for overall outcome
− 535 for target problems
− 623 for general symptoms
− 358 for social functioning
− 40 for personality functioning
▪ -> Conclusion of the authors: There was no publication bias

Within-group effect sizes were correlated with Jadad scale
(3 item judgement of study quality).
− “Was the study described as randomized?”
− “Was there a description of withdrawals and drop outs?”
− “Was the outcome assessed by blinded raters or by reliable self-
report instruments?” (modified item)
• No significant correlation (p > .28)

--> kritik: im vergleich zu anderen methoden sind die kriterien nicht sehr stark

nur in einer

No sig. difference between within-group (pre-post) ES of
LTPP and type of study (RCTs vs observational studies)
(p > .36).
▪ No sig. differences between within-group ES of 16 controlled
(including 11 RCTs and 5 non-RCT studies) and 7 uncontrolled
studies
▪ -> RCTs and observational studies were combined for further
analyses

8 studies had sufficient data for comparison of LTPP with other treatments
▪ No sig. difference between psychotropic medication and outcome in
sample of 8 comparative studies (p > .13).
-> Studies with and without psychotropic medication were combined
▪ Number of sessions:
• LTPP: M = 103 (SD = 136), median = 49
• Other treatments: M = 33 (SD = 28), median = 22
▪ Sig. correlations between within-group ES (pre-post) and treatment
condition in favor of LTPP:
• overall outcome r = .60 ** pre-post ES (0.96 vs 0.47)
• target outcome r = .48 * pre-post ES (1.16 vs 0.61)
• personality functioning r = .76 ** pre-post ES (0.90 vs 0.19)
▪ LTPP treatment effects for complex mental disorders (1 study excluded)
were even larger

--> LTPP also angeblich signi besser allerdings ned fair, weil die anderen therapien im average dtl. weniger sitzungen mit den vpn hatten

LTPP alone (16 studies) produced sig. larger within group ES
(pre-post) than LTPP combined with psychotropic medication
(7 studies):
• for target problems r = -.45*
▪ Only LTPP alone was considered for subsequent analyses of
different patient groups

Impact of 10 variables on 10 outcome variables (5 pre-post and 5
pre-follow-up variables) was explored:
• age
• sex
• diagnostic group (personality disorders, chronic or multiple mental
disorders, depressive and anxiety disorders)
• experience of therapists (years)
• specific training in the applied treatment
• use of treatment manuals (0/1)
▪ Bonferoni-adjustment (0.05/100; Alpha = .0005)
▪ No sig. correlations with outcome (p > .04)
▪ -> Critical comment: After Bonferoni-adjustment power was so low that
only very large effects could have been detected with an adequate power
(r > .72). Unfair testing

--> durch die bonderonni adjustirerung auf .0005 ist es schlicht unmöglich für nen moderator signi zu werden