The tell-signs of flawed research

Back to John Ioannidis's work (see my post) and his tell-signs of flawed research. Let's see how much is true in stuttering research.

Corollary 1: The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.

Absolutely, most samples are less than 30. He recommends 1000s!

Corollary 2: The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.

Most research does not include the effect size, but only look at statistically significant differences. If they did, they would find small effect sizes.

Corollary 3: The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.

Yes, most studies look at many different variables making it more likely that some correlate by chance.

Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.

Stuttering is very difficult to quantify unlike weight for example. It is a moving target, because people who stutter can fluctuate dramatically. Compare this to a weight measurement where the difference in weight between morning and evening is probably just a kg or so.

Corollary 5: The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

There are great financial incentives for stuttering medication: a potential market worth 100s of million dollars. There are financial incentives in AAF (altered auditory feedback) devices, for example SpeakEasy devices. They are pushing very hard on spinning the evidence. Just have a look at their website. Regarding conventional treatments including Lidcombe, there is some money to be made but the sums involved are peanuts in comparison. It is more a matter of justifying their existence as researchers and clinicians (giving their life a meaning and purpose) than about pure financial gains.

There are certainly prejudices. This is especially true for the clinicians who test their own treatment, or you try to validate their long-held beliefs. Compare this to a geneticist who studies the genetics of stuttering (he has no prejudice on what he wants to confirm). He has no hypothesis.

Corollary 6: The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.

The two hot areas I see are: Lidcombe treatment and emotionality/sensitivity studies... This is very different to the hot area of brain imaging or genetics where it is hot to do research but no-one knows what should be found!

If you want to do a post-doc in stuttering

Ludo Max is looking for a post-doc: check here:

The Laboratory for Speech Physiology and Motor Control in the Department of Communication Sciences at the University of Connecticut (Project P.I.: Ludo Max, Ph.D.) is seeking applications for a postdoctoral position to study various aspects of the neural systems underlying sensorimotor control of speech movements in individuals who stutter. This NIH-funded project involves both psychophysical and neuroimaging (fMRI) experiments, and the selected candidate will have opportunities to contribute to both lines of work. Facilities in the lab include, among other things, electromagnetic motion tracking for speech articulatory movements as well as for upper limb movements, real-time digital signal processors for auditory perturbations of speech and a Phantom 1.0 robot for mechanical perturbations of the jaw, tendon/muscle vibration, EEG/EP systems, and a virtual display environment for arm motor learning studies.

Candidates with a Ph.D. degree in cognitive/behavioral neuroscience, motor control, biomedical engineering, speech and hearing science, experimental psychology, and related fields are encouraged to apply. Good programming skills (Matlab and C++) are preferred. Candidates should be highly motivated and have an interest in publishing research in the area of speech motor control and stuttering.

I am sceptical he will find someone who can code well in Matlab and C, and at the same time knows something about stuttering. On the other hand, if the post-doc does not know anything about stuttering, he or she will be less biased and Ludo has the expertise anyway.

Another flawed Lidcombe study

And yet another flawed study on Lidcombe. This is especially disappointing because the group is independent of the Australian group around Mark Onslow. The co-author is Barry Guitar, university professor and the author of a well-known (and in general well written) text book on stuttering called
Stuttering: An Integrated Approach to Its Nature and Treatment . However, his research and his presentations on temperament and stuttering are suspicious to me. He might be a good clinician but a not very good science mind unfortunately. I would guess the first author is an enthusiastic and bright graduate student who has effectively wasted her or his time with research that has little relevance.

 Am J Speech Lang Pathol. 2008 Oct 9.

Long-Term Outcome of the Lidcombe Program for Early Stuttering Intervention.

Miller B, Guitar B.

University of Vermont.

PURPOSE: To report long-term outcomes of the first 15 preschool children treated with the Lidcombe Program by speech-language pathologists (SLPs) who were inexperienced with the program and independent of the program developers. Research questions were: Would the treatment have a similar outcome with inexperienced SLPs compared to outcomes when implemented by the developers? Is treatment duration associated with pre-treatment measures? Is long-term treatment outcome affected by variables associated with natural recovery? METHOD: Fifteen preschool children who completed the Lidcombe Program were assessed prior to treatment and at least 12 months following treatment. Pre-treatment data were obtained from archived files; follow-up data were obtained from interviews and recordings completed after the study had been planned. RESULTS: Measures of stuttering indicated significant changes from pre-treatment to follow-up in percent syllables stuttered (%SS) and Stuttering Severity Instrument-3 (SSI-3) scores. Pre-treatment severity was significantly correlated with treatment time. Handedness was the only client characteristic that appeared to be related long-term treatment outcome. CONCLUSIONS: The treatment produced significant long-term changes in children's speech, even when administered by SLPs newly-trained in the Lidcombe Program. Treatment results appear to be influenced by pre-treatment stuttering severity.

Without having read the article itself, I can see several flaws:

1) A sample size of 15 is much too small. Either you do at least 100 or you do not do it at all! The number is especially high because of the natural recovery rate increasing statistical fluctuations.

2) They have not controlled for natural recovery rate. It makes the results appear much more positive that they really are, because some will recover within one year naturally anyway and the stuttering severity will automatically go down. Here is a simple example. I have 20 kids. Let's assume 10 would have recovered within one year without treatment. Before, they all stutter at 5%. After one year without treatment, only 10 stutter, and the average stuttering rate is 2.5% (10 kids at 0 and 10 kids at 5%).

3) They try to find correlations in data with a very small sample size. Their finding on handedness is most likely a fluke.

But interestingly, the abstract seems to suggest that some kids are still dysfluent (I would have to read the article which costs money to access). The existence of dysfluent kids is not affected by statistics. So we can say that Lidcombe is not the cure it was claimed. In fact, I heard from many other therapists that some kids do not become fluent.

Why Most Published Research Findings Are False

John Ioannidis is saying exactly in this article what I have always believed that most published medical research is wrong. Stuttering is no except to the rule.

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

He should write a second article on what happens when others are pointing out issues in research. I tell you what happens when I point out issues: nothing, absolutely nothing. It is still being spread, and the only antitode is to shout as loud as possible: it's false. And of course I need to face up that people start to think of me as an eccentric outside who are no clue, really.

Correction on Indevus share price jump

I just realised that my post is probably not telling the whole story about Indevus' share price jump, because on the same day, they released information on another compound which this article by Anuradha Ramanathan claims has provoked the jump:

Shares of Indevus Pharmaceuticals more than doubled after the company reached a deal with U.S. health regulators to use its existing data for an early re-application seeking marketing approval for its testosterone replacement drug.

The agreement with the U.S. Food and Drug Administration removes the need for more studies and Indevus now plans to apply again for marketing approval in the first quarter of 2009, and launch the drug in the fourth quarter, the company said in a statement.

Half brain

I just saw a documentary on people who only have one half of their cortex (the high-level brain). When they spoke in front of the camera, they seem to have markable disfluencies. They did not stutter but they looked a bit like "recovered" stutterer who control themselves not to stutter but you perceive their mini-block: instead of getting out of control, they just move to the next word after a slight hesitation. Not sure whether it has a connection to stuttering.

Indevus shares jump by 50%

The shares of Indevus, the licence owner of Pagoclone, jumped by 50% in light of the announcement of a new partner Teva and the Phase IIb trial. The graph is from Yahoo!Finance.  It is a good example that you need to know both timing and impact of information to make money. I knew that the stock would jump upon such an announcement, but it did not know the timing or whether it would happen. Actually, I am an idiot. Come to think about it, I kind of knew, because in June I was told by reliable sources that there was a meeting for a new trial and that they would go ahead. I should have bought the stock and made a killing. Oh well...

Think about it, 50% higher means that the company is worth double from one day to the other. Why? Because investors believe that the future cashflows of the company are twice as high (neglecting discounting)! And it is twice as high, because 2-3 people at Teva think it is a risk worth taking. How much do they know about stuttering? Probably, very little. And I am sure they are not aware of many methodological pitfalls. But, the market trusts their judgement for the moment and so the value of Indevus goes up by 50%. I think the chances of clear success are moderate, but I also have not see the individual responsiveness of each patient.

Now everyone at Indevus who gets paid in shares for bonuses is worth twice the amount! Everyone will do everything to get Pagoclone approved. It could make or break for them as millionaires! Just imagine the other pharmaceutical companies looking at Indevus. Surely they will brainstorm on how to jump the band waggon. If any of you are such a company, you can hire me as a consultant! ;-)
 
Here is an Associated Press article, and here an extract:

Shares of Indevus Pharmaceuticals Inc. more than doubled Friday after the company said it is collaborating with Teva Pharmaceutical Industries Ltd. to develop a treatment for stuttering and could move forward with its delayed horomonal disorder drug.

Indevus' stock surged $1.78, more than doubling to close at $3.51. The Lexington, Mass.-based company's stock has traded between $1.19 and $8.22 over the past 52 weeks. Shares of Israel-based Teva, which makes both generic and branded drugs, rose 31 cents to $46.03.

100'000 visitors!!!!!!!!!!

I completely missed the 100'000 visitor celebration! Also the average number of visitors per day is 230! I have roughly 1000 unique visitors per week from all around the world; most are from the US.

Lidcombe treatment of choice?

Susan Block replied to my comments on her statement that "Lidcombe should be the treatment of choice" (see her ISAD article here):

I think we have discussed some of your comments before. It is the case, in my opinion that it has the best evidence to date for preschool children. The Franken et al study was almost impossible to replicate as their comparative treatment was not well defined. I think it is the case in the Jones et al study that the children in the Lidcombe treatment group made so much positive change that they could not justify maintaining children in a control group.

And I replied:

It is reasonable for you to say "in my opinion that it has the best evidence to date for preschool children". However. First, you actually wrote "should be the treatment of choice" implying a moral imperative i.e. it would be irresponsible for therapists not to use Lidcombe? Is it? Second, I repeat again that the trial described in Jones et al 2005 has a follow-up study Jones et al 2007 which you do not cite but in my opinion should. As I am sure you teach to your students, a treatment should be evaluated based on long-term outcome data and not short-term sucess. And the Jones et al 2007 paints a much more sober picture (apart from methodological issues). Have you read it? Regarding your comment "The Franken et al study was almost impossible to replicate as their comparitatve tretment was not well defined.", it is not relevant whether the trial is replicable or not for it to be true or not. In fact assuming the comparative treatment was completely ill-defined and chaotic, it managed to do as well as Lidcombe. This actually supports the alternative view that any treatment will be succesful. In any case, a new trial with a larger sample and extra care of defining the comparative treatment is under way. Regarding replicability, the same is true for the Lidcombe trial, because as you yourself say "it is the case in the Jones et al study that the children in the Lidcombe treatment group made so much positive change that they could not justify maintaining children in a control group.". We can never repeat it again with a control group! Would you therefore argue that it is not valid?

ISAD 2008

 

Check out Judith Kuster's online conference. With many good quality and no quality discussions on stuttering, research and treatment.

I have already posted a comment to Susan Block's article What clinicians should know to avoid the spreading of that myth Lidcombe should be used:

You are writing that "Current research indicates that the Lidcombe Program should be the treatment of choice for young children who stutter (Jones et al, 2005; Lincoln & Onslow, 1997)." This is misleading 1) It implies that Lidcombe is better than other early interventions. However, Lidcombe has never been tested against other forms of early intervention in a random control trial. It could well be that ANY intervention has a similar (or no) effect. There was only one pilot trial by Francken in the Netherlands and there was no difference with demands and capacity. She is currently conducting a large scale study between Lidcombe and DC. 2) You only cite the Jones 2005 article, but there is a follow-up paper from this year with long-term outcome. Three children have relapsed and many kids were not contactable any more. The sample is close to the natural recovery rate, not to speak 3. The study of Jones 2005 is questionable and has statistical and methodological flaws: wrong statistics, no long-term control group, and more. I think clinicians should know these facts. Evidence based practise is important but should be based on WELL ESTABLISHED evidence.

A revolution is happening.

A revolution is happening. It is the first time in the history of stuttering research that millions are being spent to test and develop a treatment for stuttering. Don't be mistaken. The stakes are dramatically increased. And do not expect some pseudo-science happening with waffling professors from the fiefdoms with no reality checks. The industry does not invest millions lightly, they know that they face reality checks, reality will hit them and they will do anything to get this working. And expect anything to include massive spinning of results in case the positive effects are there but very modest. And expect other pharmaceutical companies to at the very least do intensive brain storming to prevent them from getting a monopoly on a potential new market. We are entering the big bucks area of stuttering treatment.

I hope for the best (an effective treatment to reduce stuttering), but expect the worst (moderate efficacy in some with massive efforts of the companies to still make money but as a side effect a better understanding of stuttering).

Disclaimer to US readers: TheStutteringBrain blog specifically denies any responsibility for the wardrobe malfunction of the revolutionary Marie in the heat of the battle!

The pharmaceuticals are not sure themselves

And now the cold spin. Finally, Indevus managed to get an investor Teva in who will foot the bill for further clinical trials and thereby bets on Pagoclone being effective. Why did it take so long?

Indevus previously announced promising data from its 8-week, placebo controlled, double-blind, multi-center Phase II trial in patients with persistent stuttering which showed that pagoclone produced a statistically significant benefit in multiple primary and secondary stuttering endpoints compared to placebo. 

The text is a bit misleading in my view as it looks as if it is a significant effect. Don't confuse statistically significant with significant! Statistically significant means that with a high probability there is a difference in effect between the group that took Pagoclone and those who received a placebo. However, it does not say anything about how large this effect is! In fact, as I mentioned before. The reduction of stuttering in comparison to placebo was only about 10% in week 8. On the other hand, you can argue that you don't care about this fact, and only look at the absolute effect because it is the relevant measure for a person who stutters. However, it does show that the compound itself does not seem to be very effective at all! The open label phase (where everyone could choose to take the compound) was more successful. It is also possible that only a subgroup responds positively to Pagoclone and the other not at all, so the mean effect is not a good measure and does not show the large improvement on a sub group! Finally, it has not been published in a journal yet, as far as I know.

Under the terms of the Agreement, which is subject to applicable regulatory clearances and customary conditions, Indevus will conduct and Teva will reimburse Indevus for its expenses for a Phase IIb study. The placebo-controlled study will involve approximately 300 patients with stuttering in the U.S. treated for a period of six months and is expected to commence enrollment by Q1 2009.

I am also not exactly clear on what a Phase IIb study is. Wikipedia says: "Phase II trials are performed on larger groups (20-300) and are designed to assess how well the drug works, as well as to continue Phase I safety assessments in a larger group of volunteers and patients. Phase II studies are sometimes divided into Phase IIA and Phase IIB. Phase IIA is specifically designed to assess dosing requirements (how much drug should be given), whereas Phase IIB is specifically designed to study efficacy (how well the drug works at the prescribed dose(s))." I have the suspicion that in this case Phase IIb simply means "let's do it again to be sure before doing Phase III but let's do it a bit larger, longer and fine-tune", see above.

The main difference to the last Phase II trial is probably three-fold. First, 300 instead of 132 people will participate so the statistical error will be lower. Second, they will fine-tune their methodology from the last trial. Third, they observe them for 6 months. The open label was more successful, and they probably want to collect placebo-controlled data to confirm their suspicion that the full effect comes out over several months.

But the real reason might well be, because Tera and Indevus are not 100% convinced about the efficacy of Pagoclone. Stuttering is a very tricky disorder, and they might even have read my blog! ;-) So they probably want to play it safe, and re-do the last trial but with a bigger sample size and fine-tuned methodology. And the calculation is simple: if you do a real Phase III with 1000s of people, the costs are a multiple of the 300, and probably too high a risk. Wikipedia writes on Phase III: "Phase III studies are randomized controlled multicenter trials on large patient groups (300–3,000 or more depending upon the disease/medical condition studied) and are aimed at being the definitive assessment of how effective the drug is, in comparison with current 'gold standard' treatment. Because of their size and comparatively long duration, Phase III trials are the most expensive, time-consuming and difficult trials to design and run, especially in therapies for chronic medical conditions." I may add that medication can get approved after a successful Phase IIb and Phase III runs while the medication is already on the market.

We are excited that we have partnered pagoclone with a leading pharmaceutical company with a focus on central nervous system conditions,"said Glenn L. Cooper, M.D., chief executive officer and chairman of Indevus."There are currently no approved drugs anywhere in the world for patients with stuttering. Pagoclone has tremendous potential to become a highly significant commercial product, as well as to provide a ground-breaking therapy to then early three million Americans and millions of patients around the world who are afflicted with this condition. The deal we have negotiated with Teva allows us to conduct a definitive Phase IIb trial, funded by our partner. If the trial is positive, we believe that both companies will have a unique opportunity to commercialize the first pharmaceutical product for the millions of patients who stutter.

Typical CEO-bla bla bla! He doesn't really say anything, but it sounds good! For example, "tremendous potential" but of course potential might not translate into a real medication. So it is safe to say it anyway!

Watch the phrase "to conduct a definitive Phase IIb trial". Why use "definitive"? I think it is a slip of the hand, which reveals the mood they are in: They are not really sure themselves about the "tremendous potential" but this second trial will definitively give us a clear result. And any way Teva is footing the bill. And if it works fine, we just take 50% of the profits which is still millions.

Breaking News: Pagoclone is going to Phase IIb

LEXINGTON, Mass., Sept. 26 /PRNewswire/ -- Indevus Pharmaceuticals, Inc.(Nasdaq: IDEV) today announced that it has signed a development, license and commercialization agreement with Teva Pharmaceutical Industries Ltd. for the exclusive, worldwide rights to pagoclone. Indevus previously announced promising data from its 8-week, placebo controlled, double-blind, multi-center Phase II trial in patients with persistent stuttering which showed that pagoclone produced a statistically significant benefit in multiple primary and secondary stuttering endpoints compared to placebo. Pagoclone is a novel member of the cyclopyrrolone class of compounds and acts as a gamma aminobutyric acid (GABA) selective receptor modulator.

Under the terms of the Agreement, which is subject to applicable regulatory clearances and customary conditions, Indevus will conduct and Teva will reimburse Indevus for its expenses for a Phase IIb study. The placebo-controlled study will involve approximately 300 patients with stuttering in the U.S. treated for a period of six months and is expected to commence enrollment by Q1 2009.

Following the completion of a successful Phase IIb study, the Agreement provides for Indevus to participate on a 50/50 basis with Teva in the U.S.,sharing development and marketing costs, and splitting future profits, in addition to receiving milestone payments. Under certain circumstances, either party may convert the Agreement from the 50/50 arrangement to a royalty structure where Teva will be responsible for all development and commercial costs in the U.S. and Indevus would receive royalties on net sales, in addition to milestones. In either case, if the arrangement continues, Teva will be responsible for the conduct of the Phase III program.

For territories outside of the U.S., Teva will be responsible for all future development and commercialization and Indevus will receive milestones and royalties on net sales.

Under the 50/50 participation, Indevus could receive up to $92.5 million(including the Phase IIb study expenses) in U.S. and European development milestones and R&D reimbursement. In the event of a conversion to the royalty structure, in addition to the $92.5 million of milestones and reimbursements,Indevus could receive up to $50.0 million in U.S. based sales threshold milestones.

We are excited that we have partnered pagoclone with a leading pharmaceutical company with a focus on central nervous system conditions,"said Glenn L. Cooper, M.D., chief executive officer and chairman of Indevus."There are currently no approved drugs anywhere in the world for patients with stuttering. Pagoclone has tremendous potential to become a highly significant commercial product, as well as to provide a ground-breaking therapy to then early three million Americans and millions of patients around the world who are afflicted with this condition. The deal we have negotiated with Teva allows us to conduct a definitive Phase IIb trial, funded by our partner. If the trial is positive, we believe that both companies will have a unique opportunity to commercialize the first pharmaceutical product for the millions of patients who stutter.

(Thanks to Holger!)

Can you help Connecticut scientists?

Ludo Max is looking for children and adults who stutter: see here. When I visited him last autumn, I also participated in one experiment, and I can confirm that I survived!

If you stutter or if your child stutters, you can make an important and valuable contribution to the scientific understanding of stuttering.

Researchers in the Laboratory for Speech Physiology and Motor Control in the Department of Communication Sciences at the University of Connecticut are conducting several studies that may result in new scientific knowledge about the problems involved in stuttering. Participating in any of these projects provides a wonderful opportunity to make a contribution that may benefit all individuals who stutter.

We are currently recruiting children to participate in our studies in Storrs (CT) and adults to participate in our studies in Storrs or New Haven (CT). All participants receive financial compensation and free speech, language, and hearing testing. Children also receive a small book or toy.

Children between the ages of 3 and 9, if eligible, will be invited to complete tasks such as speaking into a microphone while wearing earphones, listening to tones while wearing a cap with sensors that record the brain's responses to those tones (see top picture on the left), or pointing to visual targets with a small movement sensor taped to the finger (see middle picture on the left).

Adults between the ages of 18 and 50, if eligible, will be invited to complete tasks such as speaking into a microphone while wearing earphones, speaking with small movement sensors attached to the lips, jaw, and tongue (see bottom picture on the left), or speaking while lying in an fMRI scanner at Haskins Laboratories/Yale University.

If you want more information about any of these studies, or if you want to find out if you or your child are eligible to participate, please contact Dr. Ludo Max by e-mail (ludo.max@uconn.edu) or telephone (860-486-2630). Thank you in advance for your consideration.

Toronto and Does your child stutter?

If your child stutters (or not!) and you can make it to Toronto, please consider taking part in this research project. It is important for us to increase our understanding of stuttering, and hopefully one day your child and everyone else can profit from improved treatments resulting from such research!

Does your child stutter? The University of Toronto Speech Fluency Laboratory needs both children who stutter and children who do not stutter to participate in a research project. If your child qualifies for the study you will receive a speech, language and hearing assessment. You will be compensated for your time.

Please email d.beal@utoronto.ca if you have a child who is:

1. Between the ages of 7 and 12 years old
2. Right handed

Benefits of the study include:

1. A speech, language and hearing screening for your child
2. A free 100 page booklet on stuttering
3. Compensation for your participation and travel
4. Help advance our knowledge of the brain and the role it plays in hearing, speech and stuttering!

Back from holiday

Two major events happening in my holidays.

First, the meltdown on the investment banks. WOW! The Masters of the Universe are gone just like that. Bear Stearns (for which I worked as a risk manager, see a previous post) gone, Lehman Brothers gone, Merrill Lynch bought up, and Morgan Stanley and Goldman Sachs becoming commercial banks. They all need liquidity to work, and they could not get sufficient liquidity anymore. It will change Wall Street and finance forever, after all they were the drivers of most innovations and hired the brightest, most focused, hard-working people around.
Second, the message is going around that bilingualism is a risk factor in non-recovery of stuttering based on research done by Prof. Howell's team at University College London. The Welcome Trust has financed the research and is pushing the message: see here. Here is the summary:

Bilingual children who learn two languages in early childhood are more likely to develop stuttering than those who speak another language in the home and do not learn English until they attend school, according to research funded by the Wellcome Trust.

I will have a closer look and tell you my impression. Two immediate thoughts: bilingualism means more work for the brain but would it affect stuttering, and the sub-sample of bilingual kids is too small and induces large statistical uncertainty. But I need to read the article and do the statistical calculations.

The StutteringBrain is on holiday

I am on holiday until September 19th, but might moderate comments from time to time. Till then....

AAF not effective?

Here is a study that claims that there is no clear fluency gains when using altered auditory feedback in a real environment:

1: J Speech Lang Hear Res. 2008 Aug 11.

Effects of the SpeechEasy on objective and perceived aspects of stuttering: a six-month, Phase I clinical trial in naturalistic environments.

Pollard R, Ellis JB, Finan D, Ramig PR.

University of Colorado at Boulder.

PURPOSE: Effects of the SpeechEasy when used under extra-clinical conditions over several months were investigated. Primary purposes were to help establish Phase I level information about the therapeutic utility of the SpeechEasy and compare those results to previous findings obtained in laboratory and clinical settings. 

METHOD: Eleven adults who stutter participated. A nonrandomized, ABA group design was utilized. Speech samples were collected every two weeks in extra-clinical environments. Qualitative data was collected through weekly written logs and an exit questionnaire.

 RESULTS: Group analyses revealed a statistically significant effect of the SpeechEasy immediately post-fitting, but no treatment effect across four months' time. Individual responses varied greatly with regard to stuttering frequency and subjective impressions. Relatively more stuttering reduction occurred during oral reading than formulated speech. 

CONCLUSIONS: Based on this protocol, Phase II trials are not indicated. However, positive individual responses and self-reports suggest some clinical utility for the SpeechEasy. The use of more challenging sampling procedures strengthened external validity and captured more modest altered auditory feedback effects compared to those previously reported in laboratory settings. Device use coincided more so with positive subjective impressions than measurable fluency improvement, highlighting challenges facing clinicians when implementing principles of evidence-based practice, including client-based preferences.

My interview on StutterTalk

You can listen an interview with me on StutterTalk.

Here is a summary: (from what I remember, which is probably very biased!)

We started out having a speech goal. Mine was to speak slowly with pauses, and I was able to maintain this for about 3 seconds. 

Crackpot Award:

I explain that I want to break the code of silence and consensus by naming and shaming the most extreme offenders in ignoring scientific evidence and promising cures. I read out loud an email, where I show how crackpots are unable to engage in a constructive debate on stuttering. Greg agrees with me that no matter how nice you are or how hard you worked if your idea is wrong it is wrong and you need to be told loudly and clearly and why it is wrong. 

Measurement of sucess:

Peter complains about the excessiveness of measuring treatment sucess. I agree with him. It is like telling someone that your new car costs 17'4563 dollar and 34 cents. For all practical purposes, the car costs 17'500 dollars. There is no need for great accuracy. The false accuracy fallacy is a warning sign for pseudo-science at work. For therapy work, rough qualitative assessment by the therapist and patient is fine for me. What would you do with all this information anyway? Of course, if you want to evaluate a treatment method, you need to do it thoroughly. So my stance is: Either do it very big and very well, or don't do it at all. Greg came in with the needs for standards, and I said that the health services like to see such standard as they now need to set targets.

Lidcombe & early intervention:

I say that I am not against the treatment method per se but the way it is marketed promising a lot and based on weak and flawed research. In fact, I have no issue send my kids there. However, they have not showed long-term efficacy as they claimed. And most importantly, they have not shown that Lidcombe is better than any other treatments, because it has not been tested against others. Greg made a blatantly obvious observation that totally escaped my mind so far: They (different approaches) all talk about theory, but when you actually look at what they do, it is very similar! They all talk to parents, the kids know that it is about their speech, do speech exercises, and so on. I mention that Marie-Christen Franken from the Netherlands is currently running a trial where she compares Lidcombe with demands and capacity theory. 

Medication:

I say that medication is the only hope for improvement of ALL people who stutter on a large scale, because behavioural therapy might work for some but not for all. In the same way as diets work for few, because most are not able to stick to the new behaviours. Peter is unhappy, and explains that for two years he consistently worked on changing his behaviour. I reply with: Good on you, but many including myself were, are or will not be able to do what you have done. We are not Peters! Peter felt embarassed and somehow emotionally cannot handle the fact that he is special in this respect. I also emphasised that I am not saying that there is such medication now, or that there even will be such a medication, and that it will be a cure. My prognosis is rather than there will be better (and possibly individualised) medication within 10 years in combination with behavioural therapy that makes stuttering more or less manageable for all. It is a guess, no more. I am not fond of medication pe se also due to side-effects, but I do not see another efficient way forward unless we get some very efficient behavioural therapy.

Future:

I said that I am waiting for old professor with old ideas to retire. In physics, we say that progress is not made by old professors changing their old ideas but by them dying out! 

Joe Biden: where are his actions?

When I was a trustee at the British Stammering Association, our director Norbert Liekfeldt used to say that we do not do Politics with a big P, party politics, but definitely politics with a small p, namely lobbying for the causes of stuttering and supporting the politicians that actively support us!

Many are thrilled that the Democratic Vice-President candidate Joe Biden has been stuttering as a child and teenager, and he is one of us as he attended a conference where he spoke about his stuttering openly. But I say about a politician: Don't watch their words, watch their actions! Biden is a wealthy politician with a lot of power and networks, and direct access to any decision maker. So what has he done for the causes of stuttering? Where are his actions to help the stuttering community?

My questions are:

1) Has he proposed legislation to support our causes?
2) Lobbied for more research money into stuttering?
3) Donated money? (anyone can do that!)
4) Spoke on our behalf in the Senate or asked a member of Congress to talk about stuttering in Congress?
5) Opened up his network to connect our associations to important contacts?
6) Given us direct access to himself?
7) Invested political capital to further our causes as opposed to just gaining capital by speaking at a conference?

If many answers are NO, then I am sorry but he is not one of me!