The Problem With Science is Scientists

This is a brilliant post I read from a guy who blogs as the Last Psychiatrist. I sent it to EVERY bio-psychiatrist I know who loves to talk of brain studies. I love science. I also am skeptical of every such study as it relates to psychiatry. I have been reading them for 20 yrs and they have never told me shit. As I always say: If I beat my dog every day and he gets depressed and I put him in a fMRI and his brain looks different than other dogs do I really understand his problem by looking at the fMRI. This really deserves to be read in whole. I wish I had written this and toyed with lying and said I had. ahahha hha

In a recent fMRI study, a salmon was shown a series of pictures of human faces showing various emotions: can a salmon distinguish them? and what brain regions are involved. 15 pictures, ten seconds each.

I won't bore you with the anatomy. Because of the small size of the brain, exact brain structures could not be distinguished, but something in the brain did light up. A statistically significant number of voxels, comprising an area of 81mm3 in the midline of the brain, were active (p<.0001). So can fish interpret human emotions from a picture? I have no idea. I do know, however, that that fish can't do it: it was dead.

II.

Others have discussed the hows/whys of such false positives and what can be done about them. But there are two other problems not discussed:

These researchers chose a dead fish specifically so they could discuss the issue of false positives and why multiple comparisons correction in MRI studies is important. Thus, we know these results are false positives because we know that the fish is dead. Note carefully, however, that both of the things you know are told to you by the researcher; yet you are valuing one as "truth" and the other as "artifact" based on nothing but his word.

The researchers might have been mistaken about the deadness of the fish-- thus nullifying a potentially interesting finding. Or, they could have lied.

There is no wayno way to check. You're rolling your eyes, "why would they lie about that?" or "how would they possibly make a mistake about it being dead?" and you're right, about this they wouldn't.

But what about the old studies?

If they do a study in which an anxious person shows weak activity in the amygdala-- how do you know he was anxious? How wrong about the anxiety do you have to be to invalidate the weakness of the MRI findings? Not much.

III.

The danger of the "false positive" discussion is that it is forward looking: from now on, why should tighten our significance thresholds, change the confidence intervals, controls, etc. But what about all the prior data that finds only "moderate positive correlations" using more liberal significance thresholds, that may be infected by invalid behavioral assessments--

--that because of the passage of time alone-- not better data, but time-- are now knowledge?

IV.

The biggest problem with MRI studies is that they're hard for the layman to understand. Complexity in science protects prejudice.

Understanding the Anxious Mind, in the NYT Magazine, discusses the science of temperament. Jerome Kagan studied babies, then followed them over the years. Predictably (i.e. what you'd expect the NYT to say), temperament as a baby predicted temperament as an adult, especially in the extreme cases.

They explore the case of the highly anxious "Baby 19" (defined as being distressed by novelty) who, when she was 15, was a plain looking teenager who liked writing, playing the violin, worrying and fidgeting. See? Genetics.

...[Scientists] have put the assumptions about innate temperament on firmer footing, and they have also demonstrated that some of us, like Baby 19, are... born predisposed to be anxious...

...[other scientists] all have reached similar conclusions: that babies differ according to inborn temperament...

You'll observe that those two quotes are about babies-- babies are born a certain way. No argument from me. What they do not say is that the inborn temperament is the reason they are also anxious as adults, but that's the conclusion they make every single time.

"Temperament, it turned out, tended to be stable over those five years, at least in children who started out at the extremes." Its stability is the evidence that the temperament is biological. If his haircuts are stable over the years, is that biological?

But more importantly, the kids were raised by parents. Parents don't parent in an ideal dispassionate manner, they parent in reaction to the kid in front of them. In other words, kids' temperaments alter the manner in which they are parented, and it's a good bet that the parenting fosters that same temperament. Not a word on that; it's as if it that couldn't possibly be relevant.

And why is testing a four month old's behavior evidence of an innate quality? The first four months of parenting don't count? "Kagan restricted his sample to children who were white, middle class and healthy." Oh. So now all white people are the same?

V.

But that's not real science, real science uses MRIs. If they studied it in an MRI, it must be true.

Teenagers who were in the group at low risk for anxiety showed no increase in activity in the amygdala when they looked at the face, even if they had been told to focus on their own fear. ...In the high-risk kids, even those who were apparently calm in most settings, their amygdalas lighted up more than the others' did.

"Overreactivity in the amygdala" = anxiety. But we don't really know what the amygdala does, nor how it does it, all we know is where it is. Saying something occurred in the amygdala is like saying something occurred in Ohio. "Yes, but we have some sense of what the amygdala does." And I have a sense of what Ohio does, too, it causes trouble in elections and gets its teens to kill themselves.

But at least whether or anxiety is mediated by the amygdala is worth discussing. What you can't do is take a structure that may be involved and therefore conclude that anxiety is an innate trait that is generally stable. Every time I punch someone my shoulder is overactive. Is my genetically mediated shoulder the cause of my alcoholic rages?

And overactive as compared to what? What could possible serve as a control? Seriously, think about this. Point to a guy you believe could be a control in a study measuring what are here defined as subclinical levels of anxiety.

Not every brain state sparks the same subjective experience; one person might describe a hyperaroused brain in a negative way, as feeling anxious or tense, while another might enjoy the sensation and instead uses a positive word like "alert."

None of those words mean anything. Brain state? Hyperaroused? Alert? How can anyone know that the "brain state" that two people are describing differently is the same? These words concepts are so vague that the researcher has to resort to Jungian terms in his descriptions:

The persona can be controlled, but the anima often cannot... Nathan FoxNathan Fox of the University of Maryland says that when the anima erupts in high-risk children, it often takes the form of excessive vigilance and misdirected attention. In the first of his two longitudinal studies...

If you don't even have precise words to describe what you're seeing, how in God's name can you measure it, let alone blame it on the amygdala?

VI.

But research has to start somewhere, and my problem isn't with the researchers or their study, nor do I doubt the relevance of genetics. My problem is that when theory is written up in the NYT, it becomes FACT, it becomes the default understanding. This understanding becomes part of our cultural filter. In the same way porn and Cougar TownCougar Town has assured us that women over 40 can have satisfying extra marital sex with 20 year old bicycle messengers, we know that behavior is, in large part, genetically determined.

This is how the article ends:

The predictive power of an anxiety-prone temperament, such as it is, essentially works in just one direction: not by predicting what these children will become but by predicting what they will not. In the longitudinal studies of anxiety, all you can say with confidence is that the high-reactive infants will not grow up to be exuberant, outgoing, bubbly or bold.

Think about this. Think about what the average person now understands to be true.

Still, while a Sylvia Plath almost certainly won't grow up to be a Bill Clinton, she can either grow up to be anxious and suicidal, or simply a poet.

VII.

Back to the salmon. The results were statistically significant, but the fish was dead. So we laugh. In 1620, that would have been evidence for the soul, and no one would have laughed.

The problem is the same in both cases. They are questioning the nature of the data. They should be questioning the nature of the fish.