Discussing the Lidcombe RCT

Ora writes:

I've just read your September 2005 letter to the BMJ "The statistical fluctuation in the natural recovery rate between control and treatment group dilutes their results. The statistical fluctuation in the natural recovery rate between control and treatment group dilutes their results" You wrote: I find a 10% probability that a "minimum worthwhile difference between the two arms set at 1.0% syllables stuttered" occurs due to statistical fluctuation and not due to treatment effect. The standard aims for a less than 5% chance that the observed results occur by chance (usually expressed as p<.05), whereas you're observing that this trial achieves only 10% (p<.10). Is that what you're saying? If so, 90% is obviously not as powerful a result as 95%, but it's not worthless either.

I show that two control groups (i.e. I assume no treatment effect) have a worthwhile difference in 10% of all cases, because the natural recovery rate causes the groups to start at a different baseline (they have a different percentage of natural recovers by chance) (This effect gets smaller with increasing sample size). Then I argue: If with no treatment effect, I have a 10% chance of seeing an effect, then something is wrong because I should expect that in 0% of the cases there is an effect because both groups receive no treatment. So their p-value (which they claim less than p<0.01) MUST BE wrong and higher, because the t-test assumes that they are the same. So their statistics is wrong, and they must re-do their stats. What exactly the value is is not easy to compute.

Possibly instead of choosing an effect size (1.0% difference in syllables stuttered) that allowed them to claim p<.10, they should have claimed a smaller effect size (for example, 0.5% difference in syllables stuttered) that would have supported the stronger claim of p<.05.

What you are doing is shifting the criteria to get a result! See my arguments above, we need a strong signal to be sure that no other systematic effect can destroy it.

Though I don't fully understand your argument in your Sept 2005 letter, I would agree if you're merely saying that their results may not be as strong as they're suggesting. But on the other hand it seems to me that your claim that "the random control trial did not show efficacy despite their claims" is also too strong; 90% may not be 95%, but a 90% number is still strongly suggestive.

I am saying that the p-value is much higher, but I do not know how much higher.

However, there is at least one more crucial flaw in the study that I haven't talked about in the letter but I did in my talk which supports my claim that they have not convincingly shown efficacy. They only observe the kids for 9 months. Let's assume that the only thing Lidcombe does is to accelerate the recovery of those kids that would have recovered anyway, then I see a treatment effect over the short-term. Here is how it works. Lets assume at the start 100% of the kids stutter, and after 2 years only 20% of the kids stutter, those that do not recover naturally. I also assume that 40% recover naturally per year, so I have 40% first year, 40% second year, and 20% don't recover. If I run the experiment for one year. Then in the control group only 40% recover, but in the treatment group 80% will recover (remember I assume that Lidcombe makes the kids who recover naturally recover faster). And so I have a 40% higher recovery in the treatment group!!! But if I wait for one more year, the control group will have the other 40% kids recover naturally, and then there is NO difference any more between the two groups!!! So they need a longer observation period to control for this effect! Another aspect might be to look for relapses of kids...

Mark Onslow and Lidcombe at ISA

At ISA, I attended Mark Onslow's workshop on the Lidcombe program. He didnt seem to be very pleased when I told him during the workshop that his random control trial is flawed. The day before, I had given a talk (which Mark did not attend, I think) on why the random control trial did not show efficacy despite their claims. Sue Block was not too pleased, and said that random control trials are widely used in all kinds of areas. Of course, this is the typical response without addressing my arguments. Why can no-one say: Tom, I have listened to your arguments. I don't agree with your argument on slide 10, because of X and Y. That's why you are wrong. No, everyone says but RCTs are widely used, or the RCT's statistics have been computed by a statistician. Which kind of scientific debate is this??? Anyway, I met Sue Block later at the B&withnoB, and we had a more constructive debate.

Several people approached me regarding my talk or even congratulated me for standing up against Lidcombe. I just want to emphasise that I am not really against Lidcombe, just against their claim that they have shown efficacy with an RCT. I am not an ideologue who feels a strong urge to defend his/her view of the world. Here are my thoughts:

1) I have been to the workshop, and overall the program seems to make sense and I dont think it will harm the children at the very least. In fact, it does encourage good parent-child interaction, which should be beneficial anyway. So it might well prevent greater behavioural, social and psychological effects from developing.

2) What I do criticise is that Mark Onslow claims that research has shown efficacy. However, as I said before, they have proven anything. In fact, during the workshop he himself showed some long-term data that suggests that only 84% of the children recover, which is close to natural recovery.

3) My guess is that Lidcombe might well help kids to recover faster that would have recovered anyway, might well ease tension between kid and parents (the therapist acting as supper-nanny), and for the non-recovers reduce some secondary effects, BUT I doubt very much that it makes the kids fluent that would not have recovered anyway. To do this, the program is not interventionist enough and one needs to do more active speech work.

4) If I had a stuttering child and the choice of doing nothing or doing Lidcombe, I would certainly do Lidcombe with them.

5) I get the feeling Mark does not really know how to handle my statistical arguments, but his statistician should have: Mark Jones. He has never properly addressed my issues I have raised by replying to my email or my rapid response in the British Medical Journal.

6) Lidcombe might well have an effect but on a behavioural (less severe secondary effects like grimaces or strong blocks) and psychological (avoidance and shame) level, which is much more difficult (if not impossible) to capture in a trial.

ISA: Too expensive at average quality

Despite the beautiful site, I found the congress quite expensive, especially as I had to pay it all by myself. And in general I found Cavtat and Dubrovnik quite expensive for the quality offered. I spent several holidays in Italy, and I have to say that the food in Italy is much better and similar prices. The people are far more friendly and you are less treated like a walking euro/dollar symbol.

For example, the gala dinner food was not very good. First of all, they only served champagne at the reception on the top terrace, not even water or orange juice. When I asked for orange juice, the waiter said: No. Everywhere else, they would have organised something, but not here. Big Minus. The buffet food was mass production. They even managed to have factory cakes as dessert: you know the one you can buy in the supermarket next door in boxes of 100 or 200! And the fish risotto was far too salted.

Next comes my hotel room. I booked through the conference organiser's recommended travel agent, because I assumed that they would make sure that the deals are fair. So I paid 70 Euros for a very very simple room in a flat complex 5 minutes from Hotel Croatia, and with breakfast at Hotel Croatia. As I stayed one day longer, I was curious how much the room in the neighbouring bed and breakfast would cost. I paid 20 Euros!!! So in fact they made at least 40 Euros per day. This was a complete rip-off, and I think it was irresponsible of the conference organisers not to check whether the deals offered by the agents were fair!!!

Then the day trip to Montenegro cost 90 Euro. I didn't go, because I was still upset by the way they "stole" 5 times 40 Euros from me. 90 Euro per person for a day trip is ridiculous. The bus costs at most 15 Euros per person, then lunch 20 Euros, and the guide 5 Euros, so in total 40 Euros at most! Again every single person was ripped off by 50 Euros!! Again, I blame the conference organisers for not making sure the deal very fair.

I did tell one of the organisers and someone from ISA that I found the price far too high for the services offered. And I think it is especially unfair for the many visitors from poorer countries. OK, I shut up now... :-)

A beautiful site for the ISA congress

ISA is the umbrella organisation of the national stuttering associations, and has a congress every four years. The last congress was is Perth (Australia). This year's congress was in Croatia in Cavtat near Dubrovnik. The conference itself was held in Hotel Croatia which is beautifully set on a hill next to the coast. The first picture shows the hotel's terrace where conference participants sat during breaks. The great thing was that they had a free wireless network! The next picture shows the nice hotel beach. I spent two days there. There was no sand, but it was OK. I also went swimming there.
The most useful room in the hotel was the disco/bar Posseidon in the cellar. The design is all Austin Power style, and a big dance floor in the middle. It was open every day, and often people from the conference would get together there. I even managed to convince Velda Osborne, Chair of the BSA and member of the Older Generation, to join in after the gala dinner. The next day she blamed me that I had been a bad corrupting influence on her, and that I "made her" stay at the bar until 3 o'clock in the morning. When she went up to the reception to get a taxi back to her hotel, the porter said "But Madam, it is 3 o'clock in morning, there are no taxi." And then I think he drove her back...Picture are from the net. If someone has better pictures, pls let me know!

Summary of ISA

Sorry I havent posted for a week. I needed to recover from the intense ISA conference in Dubrovnik! :-)

I really should have posted real-time from the conference, but unfortunately I didnt. As soon as I get too many different inputs, my focus gets hijacked.

Here are the topics that I will talk about:

0) A beautiful site for ISA

1) The high costs of the conference

2) Tom and Lidcombe.

3) AAF Devices and their usefulness.

4) Jaik at ISA.

5) People from all around the world.

Back from ISA / Live feeds

I am back from the ISA Congress from Dubrovnik (Croatia). I had a great time, and I will write more soon.

There is/was a live feed for the ISA Congress: here. I am not sure whether there are recordings of the key note speeches. If you know more, pls let me know.

Sorry, I wanted to post this information much earlier. By the way, the live feeds were done by Manfred. (I forgot his family name). Many thanks to him. This allowed people from around the world to be part of the ISA congress.

ISA(II)

Tim Saltuklaroglu spoke about the impact of stuttering on the physiological state of the listener. He showed that the listener has a lower heart rate and more blinking. Clearly, the listener reacts. There were many questions.

At the end, Illia and Natalia Rasskazova showed their software NEODIALOG that trains effective communication by measuring intonation.

More soon, my battery is nearly empty.

ISA live (I)

Right now, I am in a talk at the ISA conference. Already after the first slide, I fundamentally disagree with the lecturer! So let me talk about the research session that I chaired this morning.

Patrick Kelly talked about his experience as a therapist for a rural area in Ireland. He has worked there for more than 10 years and sent out a questionnaire to all his past children that he treated. He reports are very high recovery from stuttering; I think 44 out of 46.

Then Jelena Tadic showed evidence that many children who stutter also have articulatory problems. However, her sample was quite small, less than 20. So I am a bit skeptical.

Then Soster claimed that the voice of people who stutter shows difference to normal speaker's voice. I am a bit skeptical, and suggested that she should do the reverse test, i.e. is it possible to identify stutterers by only listening to their voices?

Off to the ISA conference

I am off to the ISA (International Stuttering Association) conference in Dubrovnik (Croatia). I might be blogging from there, depending on the availability of Internet Access.

Too much blogging..

Too much blogging about stuttering is bad for you. Here is what happened to The Stuttering Student. :-) Funnily enough, when I met him in San Diego, he came across as rather calm and a bit shy!!!

Competition from junior stuttering brains

Oh my god, the competition is getting younger: The Kids Speak Blog.

Monkeys, Hands, and speech

Have a look at this BBC science article: here:

They analysed the way two types of apes that are closely related to humans use hand and limb gestures to communicate. The scientists found that apes used gestures more flexibly than the way they used facial and vocal expressions. They say the findings support the theory that human language developed through the use of hand gestures.

The most appealing argument for me is that for most people the right hand is the dominant hand, which is controlled by the left-side brain. And speech and language is located only in the left-side brain. This is more than a coincidence.

I am talking about this connection, because people who stutter have subtle dual tasks defficiencies with hand or finger movements: see previous post. Maybe this is all related?

CNN talks about stuttering

Having CNN's Dr Gupta talking about stuttering is of course more authoritative than myself writing 100s of posts on my blog! So I don't want to hold back on the message, after all we learn that Jerry Maguire really wants to be Donald Duck: :-)

GERALD MAGUIRE, DR., UNIV. OF CALIFORNIA, IRVINE: I remember in kindergarten, first, second grade, I would imitate cartoon voices in class when I would be called upon. Hello, my name is Donald Duck or whatever. I haven't done them for a while, but...
GUPTA: Jerry Maguire grew up to be Dr. Maguire.
MAGUIRE:I likely stuttered the rest of my life.
GUPTA: A psychiatrist and researcher at UC Irvine, Dr. Maguire is one of the world's leading experts on stuttering. He's one of a growing number who believes stuttering is not caused by a psychological weakness, but by an anatomical problem in the brain.
MAGUIRE:We're learning that stuttering is actually a miscommunication of the brain speech centers with the mouth, throat, and tongue in getting the words out.
GUPTA:To better understand, let's take a tour through the brain. First off, this is the basil ganglea (ph), which controls speech. In people who stutter, it is believed the striatum is bombarded with too much dopamine.
MAGUIRE: We found that the medication was effective in reducing stuttering in over half, in over half the patients.
GUPTA: Pagaclone doesn't cure stuttering, but it does curb it.
MAGUIRE: I believe that the future of stuttering treatment will combine therapy and medications both.
PATIENT: Well, now, I have a free flow of thought that I've never experienced before. It's kind of weird. It's like a, kind of a revelation.

Dysfunctional stop signalling??

In a previous post, I talked about stop and go signal.

I have to clarify my message. It is not an inability to stop movement, but the inability to start because of an inability to produce stop signals that decay properly in-time, which is a different statement. So if people who stutter are at their final syllable, they send a stop signal and that's why they stop. BUT were they to attempt one more syllable, the start signal would take a long time to become dominant due to an abnormally strong and slowly decaying stop signal. This could be due to a weak go signal OR a strong stop signal that stays too long in the system!!We can say a word when the go signal dominates the stop signal. Because of the dual nature of there being a competing go and a stop signal in a motor system, a dysfunctional go signal is not the only suspect.

And a more empirical argument.

Think about the blocks we are having. For me not being able to say a word, feels more like trying to drive with the hand brake on and not like being unable to start the car. I can feel that the syllable is "ready" in my brain, but something is holding back. And then suddenly this resistance vanishes. And on some days there is less resistance...

No shared environmental influence!!!!

At last, I can report of an interesting piece of research. The authors Dworzynski, Remington, Rijsdijk, Howell, and Plomin have used data on 1000s of twins to study genetic relationships in recovered and persistent stuttering: see Am J Speech Lang Pathol. 2007 May;16(2):169-78 or here. Remember twins can be monozygotic (the same DNA), or dizygotic (share roughly half the DNA like siblings do). If the monozygotic twins share a disorder more often than dyzygotic twins at statistical significance, then the disorder is genetic because the only difference between them are the fact that they share either all the genes or roughly half the genes. Though the interesting aspect about twin studies is that you keep the environment relatively fixed, i.e. twin grow up in the same environment. A second advantage is that technically speaking this is simple statistical computation, that even non-quantitative researchers should not be able to mess up. There are surely negatives but I am not sure, probably the inability to locate genes.

Building up a twin pool is hard work but very lucrative scientifically speaking and ensures divine reverence by other scientists. Every research field undertakes a pilgrimage to these pools, and after worshipping the holders of the holy pool are allowed access to study the genetic influence.

They found that

Stuttering appears to be a disorder that has high heritability and little shared environment effect in early childhood and for recovered and persistent groups of children, by age 7.

Briefer on basal ganglia

Here is a description of the basel ganglia that Rafael sent to me. Actually, it's from Per Alm and published in the proceedings for the IFA conference in Dublin 2006.

The basal ganglia circuits are organized in a direct and an indirect pathway. These two pathways are assumed to work in synergy to modulate the activity of the frontal cortex: the indirect pathway providing a diffuse inhibition of cortical activity and the direct pathway providing focused activation of the desired action. Furthermore, the two pathways are dominated by different types of dopamine receptors, D1 vs. D2, resulting in differential effects of dopamine. The cueing functions of the basal ganglia to the SMA are dependent on a clear distinction between focal activation and the surround inhibition of the cortex, in other words, a good signal-to-noise ratio. Based on this model it is easy to imagine that the cueing of the basal ganglia can be distorted in different ways. Too weak focal activation of the direct pathway would result in deficient activation of the desired action, for example difficulties in initiating speech movements. On the other hand, impaired diffuse inhibition of the cortex, provided by the indirect pathway, could result in a combination of release of unintended movements and impaired release of the intended movement. These scenarios have clear parallels with the symptoms of stuttering. Basal ganglia motor disorders are characterized by motor initiation problems, involuntary movements, and deregulated muscular tension, often with co-contraction of antagonistic muscles.

Do we fail to stop rather than to go?

Several researchers describe stuttering as the failure to properly initiate a sound. They also talk about the disturbance in the go signal. Today, I read a very interesting article by Vanderbilt psychologists Boucher, Palmeri, Logan and Schall about how the brain decides when to start or stop movements. The graph above shows the activation of the start and stop neurons; the task is pressing a button when green and stop when red. There are really two processes in the brain battling for supremacy!

Here is the essence of the article:

"We think of people who are impulsive as acting too quickly," Logan said. "Kids with ADHD are actually slower on the 'go' task than the control kids. It's not that they go too quickly; they stop too slowly."

Take this concept to stuttering, and maybe people who stutter are actually OK on giving the go signal, but are BAD AT GIVING THE STOP SIGNAL TO THE PREVIOUS SOUND. So a block is not actually the failure to have a go signal, but a failure to initiate a stop signal to the previous sound. It is worth pointing out that every human often corrects sounds that are ready to say, but may be we cannot actually send the stop signal to this sound and therefore disturb the go signal for the next sound.

The 60-million dollar gamble!

You might criticize the pharmaceutical industry at times for high prices and profiteering, but they are really the only ones that are willing to take the high risk to heavily invest in pharmaceutical research.

Publicly traded companies need to disclose their quarterly financial reports. And you can access Indevus' financial report (thx for the tip, Holger!). These reports also include cashflow forecasts on the potential costs and income over the coming years. They estimate to spend about 60 million dollar on Pagaclone:

Given these uncertainties and other risks, variables and considerations related to each compound and regulatory uncertainties in general, we estimate remaining research and development costs, excluding allocation of corporate general and administrative expenses, from December 31, 2006 through the preparation of an NDA for our major compounds currently being developed as follows: approximately $9,000,000 for NEBIDO, $14,000,000 for PRO 2000, approximately $46,000,000 for IP 751 and approximately $61,000,000 for pagoclone for stuttering.

I am convinced that rival companies are currently working backstage to decide whether to join the hunt for a stuttering drug. We should be glad about such a development, as more money goes into stuttering research.

Does ADHD make it harder to become more fluent?

Lynne wrote an interesting comment:

My teen son has been diagnosed with ADHD-inattentive type and has stuttered since age 2 or 3. I have always felt that the two were tied together. He has a hard time using the techniques, and if he does, it's hard to concentrate on anything. Processing speed in class is much slower than what is expected and makes the task and speech worse if he is made to hurry.--Lynne

There are some researchers like Per Alm who see a link between the two. Lynne's son might have had some mild neurological incident around age 2-3 and therefore develop stuttering and ADHD-like symptoms. However, it might also be a pure coincidence. For example, some kids stutter and have asthma. That does not necessarily mean that both are related to each other. The kid might just have been unlucky to get both.

Nothing special here. But Lynne's observation is intriguing: ADHD-like symptoms seem to make applying speech techniques much more difficult. Based of this assumption, I can formulate the following prediction. Let's assume you have two possible casual sources of childhood dysfluencies: genetic (design flaw) and neurological incident (design implementation flaw). Kids with either or both of them start stuttering. Their brain tries to compensate to produce fluent speech. The kids with neurological incident are more likely to have ADHA-like symptoms, and are LESS ABLE to compensate. They are therefore LESS LIKELY to recover.

So I would predict that the share of stuttering kids with bad genes as opposed to neuro incident should DECREASE over time!!!

Looking to share room at ISA2007

After some hand-twisting from Suzana, the fabulous conference organiser, I am going to the ISA world congress in Dubrovnik in Croatia, and will give a talk on random control trials in stuttering.

I want to stay at the congress hotel itself, but a single is too expensive and therefore I am looking for someone to share a room with me.