A New Treatment for Depression - Really?

Chuck Ruby, Ph.D.

Pay attention to language! Here is a typical example of how language is used in mental health research in order to give the impression of medicine and disease, when in fact there is nothing medical or pathological about it. Yet, it is a linguistic smoke screen that obscures the real results.

JAMA Psychiatry recently published the results of a study that claims “whole-body hyperthermia” outperformed placebo in reducing the symptoms of depression. It sounds very clinical, medical, and based on an understanding of neuroscience. The study's lead paragraphs especially set the tone and give the impression of neuroscience at work. The researchers' conclude that hyperthermia "holds promise as a safe, rapid-acting, antidepressant modality with a prolonged therapeutic benefit." Isn't there a more direct way to say this?

If you take the time to dig into the weeds and see what this study is actually saying, it is quite commonplace. It is merely proposing that increased warmth can make people feel better. Didn’t we already know this? All the talk about brain structures and pathways activated during the process of warming is irrelevant, yet it is used to make depression sound like a matter of brain pathology and "whole-body hyperthermia" as medical treatment.

Beyond the critique of language, an examination of the data further question the value of this study.

To start off, the overall difference in depression between the "whole-body hyperthermia" group and the placebo group was minimal. At 6 weeks after treatment (the longest post-treatment assessment made), the hyperthermia group’s average score on the Hamilton Depression Rating Scale was about 12 and the placebo group’s average score was about 17. A Hamilton score of 12 is within the “mild” range and 17 is within the “moderate” range. The cutoff between mild and moderate ranges is 13-14. (For now let's ignore the issue of the 86% sensitivity and 92% specificity ratings of the Hamilton).

Added to the questionable meaning of these Hamilton score differences is the fact that the confidence intervals were quite large. For example, the 6 week difference in scores was 4.27. But the 95% confidence interval for this difference was 7.94 to .61. This means there is a 95% chance that the true difference between the groups’ scores was between 7.94 and .61, and a 5% chance the difference score was outside that range. Given that the cut off score between “mild” and “moderate” depression of the Hamilton is 13-14, claiming any practically significant difference between the hyperthermia group and placebo group is dubious.

Further, it must be remembered that the above Hamilton scores are group averages and they do not sufficiently account for the individuals’ scores. The two groups' score distributions overlap such that some in the placebo group did better than those in the hyperthermia group. At 6 weeks, the Cohen's d effect size between the groups was 1.66. At this value, about 40% of the two groups overlap. And remember, this is using the 4.27 Hamilton score difference, which is questionable given the large confidence intervals.

Finally, the placebo effect was greater for the hyperthermia group than placebo group. After the study, 94% of the hyperthermia group guessed correctly that they were in the experimental group, whereas only 71% of the placebo group thought they were receiving the experimental treatment. This would have artificially reduced the hyperthermia group's scores on the Hamilton based on the placebo effect alone.

Linguistic gymnastics are used in this and many other studies to give a medical model impression. More over, the very statistics reported ostensibly to justify a claim of superior treatment demonstrate the unconvincing nature of the results. Trivial effects, excessive confidence intervals, nomothetic washing away of individuality, and the placebo effect make this study of questionable use, other than to demonstrate that warmth can help some people feel better. How surprising is this?