Friday, July 18, 2008

Results are even worse than in abstract

I have now the full article on the follow-up of the Lidcombe trial. The abstract claims that 16% relapsed, but that's on recorded speech sample. But if you look at the parents' feedback:
... parent questionnaires indicated that eight (40%) children had stuttered at some time during the previous month and twelve (60%) children had not.
So actually 40% of the so-called recovered kids showed some stuttered speech recently according to their parents! How can this be reconciled with the magic treatment claim that Lidcombe makes all kids fluent voiced by some? It is not hard to imagine that some of these kids who had a fluent recording are actually dysfluent at others times and will unfortunately start stuttering again more severely or become covert stutterers.

And this does not even include the discussion of the natural recovery rate. Due to the small sample of 20 kids, the statistical fluctuation is huge and all results have a huge statistical error on them.

And calling the trial a random control trial is a bit dubious considering that
The majority of controls could not be contacted, possibly because many had not had regular contact with the clinic and so could not be traced.
And I haven't even read the article closely...

6 comments:

ac said...

It does sound as if this trial doesn't really tell us anything. Hopefully a sufficient number of studies on Lidcombe will be published for a meta-study with better resolution to be possible.

I think it's unfair of you to claim a 40% relapse rate, as they defined relapse as more than 1% of syllables stuttered, not zero stuttering. You're right that the recorded speech sample is just that, a sample, but they needed some way to quantify the level of stuttering. Different parents aren't going to answer a question about the level of stuttering they observed consistently.

Tom Weidig said...

Hi ac,

how can you do a proper meta-study if their data is so bad? Most of the kids could not even be contacted again.

> it's unfair of you to claim a 40% relapse rate.

I did not claim 40% relapse. I said that "40% of the so-called recovered kids stuttered recently according to their parents", which makes me wonder what the real relapse rate is and whether some of them who show residual stuttering are on their way to stutter again.

>> as they defined relapse as more than 1% of syllables stuttered, not zero stuttering.

It is their definition. No-one expect 100% fluency, but I am wondering how well their measure captures the real issue if parents say that the kid stuttered. After all in the Lidcombe program, they are trained to be therapists and be able to spot real stuttering.

> You're right that the recorded speech sample is just that, a sample, but they needed some way to quantify the level of stuttering.

yes, but is it the right way?

> Different parents aren't going to answer a question about the level of stuttering they observed consistently.

Yes? As I said they did the training with the child, and so should be better at observation and identifying real stuttering.

Greg said...

This one's easy Tom...It all goes back to the fallacy that if someone can do something for *some* of the time, then they must be able to do it *all* of time.

So these relapsed clients really haven't relapsed--they're just lazy.

Not my view, just citing the unspoken visceral perspective. When in doubt, blame the client.

Tom Weidig said...

Good points. They are on my list!

Norbert @ BSA said...

Tom

you quote that "parent questionnaires indicated that eight(40%) children had stuttered at some time during the previous month" which you then turn into parents that have reported that their children 'stuttered recently". I haven't read the article but can you confirm what the parents are reporting: a "bumpy" word, sentence, or phrase? Or a day, or even days, of relapsed stammering?

"magic treatment claim that Lidcombe makes all kids fluent voiced by some" - Isn't it the case that the outcome of the RCT says that treating the child is significantly more beneficial in terms of stutter-free speech than leaving the chld without intervention? I can;t quite recall the RCT article talking about making all children fluent - guaranteed, as if by magic. Isn't that a bit of a strawman?

"And calling the trial a random control trial is a bit dubious considering that

"The majority of controls could not be contacted, possibly because many had not had regular contact with the clinic and so could not be traced.""

Aren't you confusing the original RCT with the study conducted in the article you quote?

Greg - can you point me to the part of the article where it says the childre are to blame? Can one actually "cite" an "unspoken" perspective?

Tom Weidig said...

Hi Norbert,

>>> you quote that "parent questionnaires indicated that eight(40%) children had stuttered at some time during the previous month" which you then turn into parents that have reported that their children 'stuttered recently". I haven't read the article but can you confirm what the parents are reporting: a "bumpy" word, sentence, or phrase? Or a day, or even days, of relapsed stammering?

The parents said that their kids had "stuttered at some time during the previous month." The parents did not mark 1 for "no stuttering", but 2 or more.

>>> "magic treatment claim that Lidcombe makes all kids fluent voiced by some" - Isn't it the case that the outcome of the RCT says that treating the child is significantly more beneficial in terms of stutter-free speech than leaving the chld without intervention? I can;t quite recall the RCT article talking about making all children fluent - guaranteed, as if by magic. Isn't that a bit of a strawman?

I was refering to the lectures of the main Lidcombe popularisers, who repeatedly said that all kids become fluent with Lidcombe, stuttering is dying out and this message was spread everywhere. But this follow-up study confirms (despite its massive flaws) my suspicion that it is not as simple. Like in adult treatment, you need to look at the long-term effect, ANYTHING ELSE IS IRRESPONSIBLE. And we still havent excluded the natural recovery effect. And remember this authors have a severe conflict of interest because they advocate the treatment, so we can expect the real results to be worse.

>>> "And calling the trial a random control trial is a bit dubious considering that "The majority of controls could not be contacted, possibly because many had not had regular contact with the clinic and so could not be traced."" Aren't you confusing the original RCT with the study conducted in the article you quote?

A serious RCT is one that looks at the long-term effect. This is especially true because there is a natural recovery. So the RCT NEEDS to be 2-3 years to filter out the natural recovery rate effect. But of course, it completely failed to do this. They should have compared two treatment approaches rather than have one control group.