Concluding installment of NIMH biomarker porn: Depression, daughters, and telomeres
Pioneer HPA-axis researcher Bernard “Barney” Carroll’s comment left no doubt about what he thought of the Molecular Psychiatry article I discussed in my last issue of Mind the Brain:
Where is the HPA axis dysregulation? It is mainly in the minds
of the authors, in service of their desired narrative. Were basal cortisol levels increased? No. Were peak cortisol levels increased? They didn’t say. Was the cortisol increment increased? Only if we accept a p value of 0.042 with no correction for multiple comparisons. Most importantly, was the termination of the stress cortisol response impaired? No, it wasn’t (Table 3). That variable is a feature of allostasis, about which co-author Wolkowitz is well informed. Termination of the stress response is a crucial component of HPA axis regulation (see PubMed #18282566), and it was no different between the two groups. So, where’s the beef? The weakness of this report tells us not only about the authors’ standards but also about the level of editorial tradecraft on display in Molecular Psychiatry. [Hyperlink added]
You also can see my response to Professor Carroll in the comments.
I transferred another comment to the blog from my Facebook wall. It gave me an opportunity to elaborate on why
we shouldn’t depend on small convenience samples to attempt to understand phenomena that must be examined in larger samples followed prospectively.
I explained
There are lots of unanswered questions about the authors’ sampling of adolescents. We don’t know what they are like when their mothers are not depressed. The young girls could also simply be reacting to environmental conditions contributing to their mother’s depression, not to their mother’s depression per se. We don’t know how representative this convenience sample is of other daughters of depressed mothers. Is it unusual or common that daughters of this age are not depressed concurrent with their mothers’ depression? What factors about the daughters, the mothers, or their circumstances determine that the mother and daughter depression does not occur at the same time? What about differences with him him dthe daughters of mothers who are prone to depression, but are not currently depressed? We need to keep in mind that most biomarkers associated with depression are state dependent, not trait dependent. And these daughters were chosen because they are not depressed…
But with no differences in cortisol response, what are we explaining anyway?
The Molecular Psychiatry article provides an excellent opportunity to learn to spot bad
From http://www.compoundchem.com/2014/04/02/a-rough-guide-to-spotting-bad-science/
science. I encourage interested readers to map what is said in that into the chart at the right.
This second installment of my two-part blog examines how the exaggerations and distortions of the article reverberate through a press release and then coverage in NIMH Director Thomas Insel’s personal blog.
The Stanford University press release headline is worthy of the trashy newspapers we find at supermarket checkouts:
Girls under stress age more rapidly, new Stanford study reveals
The press release says things that didn’t appear in the article, but echoes the distorted literature review of the article’s introduction in claiming well-established links between shortened telomeres, frequent infections in chronic disease and death that just are not there.
The girls also had telomeres that were shorter by the equivalent of six years in adults. Telomeres are caps on the ends of chromosomes. Every time a cell divides the telomeres get a little shorter. Telomere length is like a biological clock corresponding to age. Telomeres also shorten as a result of exposure to stress. Scientists have uncovered links in adults between shorter telomeres and premature death, more frequent infections and chronic diseases.
From http://news.stanford.edu/news/2014/october/telomeres-depression-girls-10-28-2014.html
And the claim of “the equivalent of six years” comes from direct quote from obtained from senior author Professor Ian Gotlib.
“It’s the equivalent in adults of six years of biological aging,” Gotlib said, but “it’s not at all clear that that makes them 18, because no one has done this measurement in children.”
Dr. Gotlib seems confused himself about what he mean by the 10 to 14-year-old girls having aged an additional six years. Does he really think that they are now 18? If so in what way? What could he possibly mean – do they look six years older than age matched controls? That would be really strange if they did.
I hope he lets us know when he figures out what he were saying, but he shouldn’t have given the statement to the Stanford press officer unless he was clear what he meant.
The press release noted that Dr. Gotlib had already moved on to intervention studies designed to prevent telomere shortening these girls.
In other studies, Gotlib and his team are examining the effectiveness of stress reduction techniques for girls. Neurofeedback and attention bias training (redirecting attention toward the positive) seem promising. Other investigators are studying techniques based on mindfulness training.
That’s a move based on speculation, if not outright science-fiction. Neurofeedback has some very preliminary evidence for effectiveness in treating current depression, but I would like to see evidence that it has any benefit for preventing depression in young persons who have never been depressed
Gotlib’s claims play right into popular fantasies about rigging people up with some sort of apparatus that changes their brain. But everything changes the brain, even reading this blog post. I don’t think that reading this blog post has any less evidence for preventing later depression than neurofeedback. Nonetheless, I’m hoping that my blogging implants a healthy dose of skepticism in readers’ brains so that they are immunized against further confusion from exposure to such press releases. For an intelligent, consumer oriented discussion of neurofeedback, see Christian Jarrett’s
Attention bias training is a curious choice. It is almost as trendy as neurofeedback, but would it work? We have the benefit of a systematic review and recent meta-analysis that suggests a lack of evidence for attention bias training in treating depression and no evidence for preventing it. If it’s ineffectual in treating depression, how could we possibly expect it to prevent depression? Evidence please!
Let’s speculate about the implications if the authors found the cortisol differences between the daughters of the depressed mothers and daughters of controls that they had hypothesized but did not find. What then could have been done for these young girls? Note that the daughters of depressed mothers were chosen because they were functioning well, not currently depressed themselves. Just because they were different from the control girls would not necessarily indicate that any cortisol variables were in the abnormal range. Cortisol levels are not like blood pressure – we cannot specify a level below which cortisol levels have to be brought down for better health and functioning.
Note also that these daughters were selected on the basis of their mothers being depressed and that could mean the daughters themselves were facing a difficult situation. We can’t make the mother-bashing assumption that their mother’s depression was inflicting stress on them. Maybe any psychobiological stress response that was evident was due to the circumstances that led to the depression of their mother. We don’t know enough to specify what levels of cortisol variables would be optimal and consistent with good coping with the situation – we let even specify what is normal. And we don’t know how the daughters would recover from any abnormalities without formal treatment when their circumstances changed.
Bottom line is that these investigators did not get the results they hypothesized. Even if they had, results would not necessarily to lead to clinical applications.
Nonetheless, the director of NIMH saw fit to single this paper out or maybe he was just picking up on the press release.
Thomas Insel’s Personal Blog: Depression, Daughters, and Telomeres.
Thomas Insel’s Director’s Blog starts by acknowledging that there are no genetic or imaging markers predicting risk for depression, but research by Stanford Psychology Professor Ian Gotlib and colleagues in Molecular Psychiatry is “worth watching.”
Insel describes Gotlib’s “longitudinal” research as following depressed mothers’ early adolescent daughters.
The young girls have not yet developed depression, but 60 percent will become depressed by the age of 18.
I can find no basis in the article for Insel’s claim that Gotlib has found 60 per cent of these girls will be depressed by age 18. The estimate seems exaggerated, particularly given the case mix of mothers of these girls. It appears that some or most of the mothers were drawn from the community. We cannot expect severe course and biological correlates of depression that we would expect from a more inpatient sample.
Searching the papers coming out of this lab, I could only find one study involving a 30 month follow-up of 22 daughters of depressed mothers in the same age range as the sample in the Molecular Psychiatry article. That’s hardly a basis for the strong claim of 60% becoming depressed by 18.
Insel embellishes the importance of differences in telomere length. He perpetuates the illusion that we can be confident that differences in telomere length suggest these girls were experiencing accelerated aging and what have high risk for disease when the girls reached middle and late age. Without the backing of data from the paper or the existing literature, Insel zeros in on
Troubling early sign of risk for premature biological aging and possibly age-related chronic diseases, such as cardiovascular disease. Investigating the cause and timing of decreased telomere length—to what extent it may result from abnormalities in stress responses or is genetically influenced, for example—will be important for understanding the relationship between cellular aging, depression, and other medical conditions.
Insel ponders how such young, healthy girls could possibly show signs of aging. According to him the answer is not clear, but it might be tied to the increased stress reactivity these girls show in performing laboratory tasks.
But as Professor Caroll noted, the study just does not much evidence of “increased stress reactivity.”
Nonetheless, Insel indicates that Gotlib’s next step is
Using neurofeedback to help these girls retrain their brain circuits and hopefully their stress responses. It will be a few years before we will know how much this intervention reduces risk for depression, but anything that prevents or slows the telomere shortening may be an early indication of success.
It’s interesting that Insel sidestepped the claim in the press release that Gotlib was trying out a cognitive behavioral intervention to affect stress reactivity. Instead he presents a fanciful notion that neural feedback will somehow retrain these girls’ brain circuits and reduce their stress response throughout their time at home and prevent them getting depressed by their mother’s depression.
Oh, if that were only so: Insel would be vindicated in his requiring for funding that researchers get down to basic mechanisms and simply bypass existing diagnoses with limited reliability, but at least some ties to patients’ verbal reports of why they are seeking treatment. In his world of science fiction, patients, or at least these young girls, which come in to have their brains retrained to forestall the telomere shortening that is threatening them not only with becoming depressed later, but with chronic diseases and middle and late life and early death.
So, let’s retrace what was said in the original Molecular Psychiatry article to what was claimed in the Stanford University press release and what was disseminated in the social media of Dr. Insel’s personal blog. Authors’ spin bad science in a peer-reviewed article. They collaborate with their university’s press relations department by providing even more exaggerated claims. And Dr. Insel’s purpose is served by simply passing them on and social media.
There’s a lot in Dr. Insel’s Personal Blog to disappoint and even outrage
- Researchers seeking guidance for funding priorites.
- Clinicians in the trenches needing to do something now to deal with the symptoms and simple misery that are being presented to them.
- Consumers looking for guidance from the Director of NIMH as to whether they should be concerned about their daughters and what they should do about it.
A lot of bad science and science fiction is being served to back up false promises about anything likely to occur in our lifetimes, if ever.
Taxpayers need to appreciate where Dr. Insel is taking funding of mental health with research. He will no longer fund grants that will explore different psychotherapeutic strategies for common mental health problems as they are currently understood – you know, diagnoses tied to what patients complain about. Instead he is offering a futuristic vision in which we no longer have to pay for primary care physicians or mental health clinicians spending time talking to patients about the problems in their lives. Rather, patients can bring in a saliva sample to assess the telomere length. They then can be rigged up to a videogame providing a social stress challenge. They will then be given neurofeedback and asked to provide another saliva sample. If the cortisol levels aren’t where they are supposed to be, they will come back and get some more neurofeedback and videogames.
But wait! We don’t even need to wait until people develop problems in their lives. We can start collecting spit samples when they are preteens and head off any problems developing in their life with neural feedback.
Presumably all this could be done by technicians who don’t need to be taught communication skills. And if the technicians are having problems, we can collect spit samples from them and maybe give them some neurofeedback.
Sure, mild to moderate depression in the community is a large and mixed grouping. The diagnostic category major depression loses some of its already limited reliability and validity when applied to this level of severity. But I still have a lot more confidence in this diagnosis than relying on some unproven notions about treating telomere length and cortisol parameters in people who do not currently complain about mental health or their circumstances. And the lamer notion that this can be done without any empathy or understanding.
It’s instructive to compare what Insel says in this blog post to what he recently said in another post.
He acknowledged some of the serious barriers to the development of valid, clinically useful biomarkers:
Patients with mental disorders show many biological abnormalities which distinguish them from normal volunteers; however, few of these have led to tests with clinical utility. Several reasons contribute to this delay: lack of a biological ‘gold standard’ definition of psychiatric illnesses; a profusion of statistically significant, but minimally differentiating, biological findings;‘approximate replications’ of these findings in a way that neither confirms nor refutes them; and a focus on comparing prototypical patients to healthy controls which generates differentiations with limited clinical applicability. Overcoming these hurdles will require a new approach. Rather than seek biomedical tests that can ‘diagnose’ DSM-defined disorders, the field should focus on identifying biologically homogenous subtypes that cut across phenotypic diagnosis—thereby sidestepping the issue of a gold standard.
All but the last sentence could have been part of a negative review of the Molecular Psychiatry article or the grant that provided funding for it. But the last sentence is the kind of nonsense that a director of NIMH can lay on the and research community and expect it to be reflected in their grant applications.
But just what was the theme of this other blog post from Dr. Insel? P-hacking and the crisis concerning results of biomedical research not being consistently reproducible.
The relentless quest for a significant “P” value is only one of the many problems with data analysis that could contribute to the reproducibility problem. Many mistakenly believe that “P” values convey information about the size of the difference between two groups. P values are actually only a way of estimating the likelihood that the difference you observe could have occurred by chance. In science, “significance” usually means a P value of less than 0.05 or 1 in 20, but this does not mean that the difference observed between two groups is functionally important. Perhaps the biggest problem is the tendency for scientists to report data that have been heavily processed rather than showing or explaining the details. This suggests one of the solutions for P-hacking and other problems in data analysis: provide the details, including what comparisons were planned prior to running the experiment.
Maybe because Insel is Director of NIMH, he doesn’t expect anybody to call him on the contradictions in what he is requesting. In the p-hacking blog post, he endorsed a call to action to address the problem of a lot of federal money being wasted on research that can’t lead to improvements in the health and well-being of the population because the research is simply unreliable and depends on “heavily processed” data for which investigators don’t provide the details. Yet in the Depression, Daughters, and Telomeres post he grabs an outrageous example of this being done and tells the research community he wants to see more of it.
