The stuttering research literature and its transition to the mainstream have always frustrated me. I hear people making statements based on research articles or some magical consensus view. However, the research rarely proves what people interpret from its findings, and its findings are often flawed due to weak methodology. And, the consensus view either does not really exist or is a good example of how the strength of an argument is not related to a majority supporting it (but you can argue that the likelihood of the statement being true is higher). Following up the lead of the source, and the source's source, and so on, decreases the certainty on who the source is and the accuracy of the interpretation of what the source actually said. As a teenager I spent my Christmas holidays in a youth winter camp in the Swiss Alps, and in the evenings we often played an interesting game, that I am sure most of you have played, too: You sit in a circle, and the first person whispers a message to the next person, and round it goes through the circle. The first person never got his first message back.
A recent example of my frustration is the random control trial research article on the Lidcombe treatment program for stuttering children. The mainstream talk is that this article has shown that the Lidcombe program effectively cures stuttering children. (Please note that I am not actually claiming that the authors say this.) I have spoken about the stats obsessively in several posts: see here. Today, I re-visited the article and actually realised that the interpretation possible are rather limited due to their design.
They have randomly chosen two groups of stuttering kids, and only treated one group. And they found that the treatment group is more fluent; and therefore claiming success. However, the standard random control trial setup has a different purpose: see whether there are above-placebo benefits. You administer a pill to BOTH groups, and therefore BOTH groups do not know whether they receive treatment or not. In the Lidcombe study, the treatment group and their parents knew that they received treatment. The extra fluency in the treatment group could mean three things:
1) Success due to the placebo effect: believing heals. (Admittedly, even placebo effect is good if it is permanent!)
2) Success due to additional-to-placebo effect: the Lidcombe study has a treatment component that is successful. This leaves open the possibility that all other known treatments like demand & capacity would be as successful. For example, the treatment component might be ANY extra attention given by parents to their stuttering child. A study by Franken et al. compared Lidcombe to demand & capacity and find no difference in outcome. (However, I think their sample size is rather small.)
3) Success due to unique-to-Lidcombe effect. Lidcombe is more successful than other treatment approaches because of treatment component X, Y, and Z.
As you can see the study leaves all three options open, but the mainstream talk is virtually exclusively about option 3 even though this is not the only possible interpretation of the data!!!!
3 comments:
Hi Tom.
I guess people need to think that stuttering (specially on children) can be controled. We need to think it, it is safe to think it and it is disturbing to think that stuttering cannot be controled. PWS, fathers of CWS, SLPs, the general population, want to believe that there is something, some terapy, to control stuttering effectively. Thus, myth propagation spreads easily.
Regards.
Pablo.
Spain
I agree, a major driver is wishful thinking. We want a therapy that works.
Also, 99.99% have not read the article, and the rest have only read the abstract. I am even convinced that at the very best 1 or 2 of the authors actually understand the statistics.
Why were many witches burned?
1. You need someone to blame for unpredictable events.
2. When you burn them, you are relieved as you have solved an unsolvable problem arising from the unpredictable event over which you have no control and dont know how it happened.
3. Unexplained is inexplicable for many.
Good points, Tom.
(1) If you (i.e., 'one') accept that the Lidcombe RCT shows a treatment effect, then you may conclude that some (not necessarily all all) of the Lidcombe treatment is effective. But you can't conclude that this means that Lidcombe Prog is better than other therapies, because there is no similar test of other therapies. As the Lidcombe BMJ paper suggests, the most that can be concluded is that Lidcombe is better than nothing.
I think this is just restating this risks in your point 3.
(2) An additional consideration that is overlooked just about everywhere concerns what/who is being treated and assessed.
Almost all discussion of the efficacy of this programme makes the assumption that it treats what it says (or implies) it's treating: that is, it's treating the disorder that would otherwise turn into persistent stuttering.
However, obviously, we can be reasonably sure that a lot of the kids in the treatment arm would have recovered spontaneously - they're the ones with transient stuttering (TS). It is not unreasonable to suggest that these TS kids could also benefit from treatment and recover faster than they would otherwise.
If it is the case that children with transient stuttering can benefit from therapy, then if you were able to identify a group of 54 kids with transient stuttering, divide them into 2 groups and treat one group with the Lidcombe programme, while leaving the others alone, you might just get very similar results as those reported in the BMJ.
In the most pessimistic view, it could be that transient stuttering and persistent developmental stuttering have different etiologies and require different interventions and that the various therapies for children who stutter are actually only effective in speeding recovery of those with transient stuttering and not effective at all in preventing the continuation of persistent developmental stuttering.
I can almost hear you thinking 'we need brain imaging studies on kids in order to differentiate', but behavioral studies can also be revealing.
Cheers,
Robin
Post a Comment