Tuesday, October 06, 2009

English translation of PEVOS article

WOW, the blogosphere has learned to translate. Here is the English translation of the German PEVOS article. This should give non-German speakers an idea of what has been done in Germany regarding outcome studies. (Thanks to Norbert, no guarantee for accuracy, copyright to the authors).

PEVOS pilot phase evaluated

PEVOS: Programme for the Evaluation of Stammering Therapies

Authors: Anke Alpermann, Ulrich Natke, Horst M. Oertle, Julia Pape-Neumann & Peter Schneider

The aim of PEVOS is to apply scientific methods to gain data about the effectiveness of stammering therapies that are offered in Germany.

Attentive readers of the Kieselstein may have been asking themselves ‘what ever happened with PEVOS’? With this report we are trying to give you an answer.

The BVSS (German Stammering Association) is constantly asked by many enquirers about ‘good stammering therapies’. For many years, there have been controversial debates about which criteria should be used to evaluate stammering therapies. This has led to very different research methods resulting in the fact that it is almost impossible to compare different therapy approaches. Making meaningful statements about the effectiveness of a specific approach thus is rather difficult.

This led to the idea of a large-scale study to find out what kind of therapies are actually being offered under what conditions, and what kind of results can typically be expected. The long-term outcomes of therapy from therapists all over Germany was to be investigated. This was to be done using standardised and encompassing scientific methods which take into account the diversity of the stammering experience.

A working party consisting of specialists of various academic disciplines as well as people who stammer was founded and tasked by the BVSS to research into the effectiveness of stammering therapies, i.e. carry out an evaluation. This was the foundation for PEVOS.

In a survey, about 50 therapists declared themselves willing to take part. It was assumed that each one of these would, on average, treat 10 clients. Assuming the study would take about two years we could expect information from about 1,000 therapy episodes. No research project of this size had existed previously, either in Germany or abroad.

Members of the working party assumed that therapists as well as clients could only be enticed to participate if there was some kind of benefit for them. The study was therefore developed in such a way that therapist as well as clients would receive feedback about what had been achieved, following analysis of their results.

If there was sufficient data, it would then be possible to compare any individual therapy episode with the anonymised overall results of the therapy approach. Thus it would also become obvious to the therapist where improvements in their practice might be required.

To find out whether fluency increases, speech tempo is normalised and speech naturalness does not deteriorate it was planned to gain speech samples at different stages, using telephone calls with clients. Various questionnaires were meant to provide information about changes in emotions, thoughts and actual behaviours.

As the expected amount of data was large, a database was set up to cater for the input. All results from the speech samples and from the questionnaires, as well as additional information about the practice of the therapists and the clinical history of the clients could be entered and could be analysed according to various constellations.

1 Pilot phase

After the concept was developed it was due to be tested in a pilot phase. 9 therapists with 99 clients took part. The aim of the pilot phase was to test the process and suitability of the measuring scales and, if possible, to generate hypotheses about why certain therapy modules work in conjunction with others. These hypotheses would then be tested in individual research studies. The pilot phase would not have been capable of producing other than limited results – for statistically valid findings, a large-scale research project was planned following the pilot phase.

99 clients took part in the pilot study; 49 of them were children and young people up to an age of 16.1 years, as well as young adults and adults from 17 years upwards. In both age ranges, about 80% were male, 20% female. Only 9 of these were treated as out patients, while 90 took part in various group therapy sessions as in-patients.

To be able to collect the changes effected by the various therapy approaches data was collected at four points: directly before the start of therapy (U1), 406 weeks after the end of therapy (U2), one year after the end of therapy (U3) and 2 years after the end of therapy (U4).

The data was collected independently from the therapists by staff members of the BVSS who were also members of the PEVOS working group.

To obtain subjective as well as objective information, data was collected from clients in two ways. At all four stages (U1 to U4) a telephone interview was conducted with all test subjects, during which six fixed questions had to be responded to at length. These recordings were then used to evaluate the fluency of the clients.

To evaluate the changes in attitudes, emotions and behaviours connected to speaking and stammering, the clients were sent questionnaires at each of the four intervals which were to be completed and sent back to the researchers.

All data from the telephone interviews and from the questionnaires were entered into the database and were available for statistical analysis.

In additions, therapists were required to provide information about the time frame and setting of their therapy as well as about therapy modules used. This information, too, was entered into the database.

When carrying out the pilot phase it some became apparent that it would be difficult to reach clients. As all clients were to be called for the telephone interviews randomly without having previously agreed a date the researchers had to show a significant amount of flexibility in terms of their time commitment. At stages U3 and U4, some clients had moved away or were no longer interested and withdrew from the speech sampling. Likewise, the return of questionnaires became more difficult the greater the distance from the actual therapy. Despite several attempts through the telephone and in writing, not all clients could be reached to complete their data sets. The therapists, too, did not always provide the complete data so that in the end complete data sets were only available for 66 out of 99 clients (c. 66%).

2. Results of the pilot phase

The speech samples from telephone interviews were analysed for their stammering frequency, naturalness and speed. The graph shows the stammering frequency amongst adults. We can see a typical example for stammering therapies: directly after therapy (U2) stammering frequency initially decreased significantly; following that, it increases again, but even at U4 is remains significantly below the initial measurement immediately prior to therapy.

The same picture is evident when looking at speech naturalness and speed. In the long term, the therapies have the effect of increasing naturalness and decreasing speed.

These positive results are reflected when looking at the outcomes of the questionnaires. The self evaluations on the whole are more positive than before therapy, avoidance behaviours are reduced and attitudes towards communication are improved. We have found very similar results for adults and for children.

The therapists who participated were expected to provide data about the type and the extent of the therapy elements used (e.g. relaxation, identification, soft onsets). Using statistical methods we were aiming to identify the influence of various therapeutic elements on therapeutic effects. Due to the limited sample size of some of the therapy approaches the results are not statistically valid. A large-scale research study could provide us with important insights about the effectiveness of different therapy elements which could form the basis for a further optimisation of stammering therapy.

Overall results of the pilot phase demonstrate that in the long term stammering therapy does have positive effects. This is demonstrated through quantitative measurements (speech samples) as well as qualitative data (questionnaires). A significant improvement directly following therapy is no guarantee that this can be maintained. In the longer term, relapse is a possibility. The quality of stammering therapy should therefore not be judged by outcome measures taken directly after the end of therapy. In addition, therapy provider should offer a long-term after care as a fixed part of their provision and agree this with the participants.

The results of the pilot phase have been described in detail in a final report, including the individual results.

3. Conclusion and Possibilities for a large-scale study

PEVOS became noticed internationally at the 3rd IFA World Congress in Nyborg (Denmark, 2000) where Horst M Oertle presented about the project and at the 4th IFA World Congress in Montreal (2003) where Julia Pape-Neumann reported on the results. In addition, PEVOS has been cited and commented on in a number of journal articles and books. It is unanimously regarded as a unique and unprecedented project to enable us to determine objectively the quality of stammering therapies.

The pilot phase demonstrated that the processes and the instruments used by PEVOS are basically suitable to measure the effectiveness of stammering therapies on various levels. The resources required for of a large-scale study can be calculated. The pilot phase has provided us with much information about how this should be carried out – some of this will be briefly presented in the following paragraphs.

Evaluating therapy for children is much harder than those for teenagers and adults. The problem lies in collecting data through speech samples as well as data through questionnaires but also in the fact that stammering symptoms, especially in young children, can spontaneously go into remission even without therapeutic intervention. It seems therefore to be opportune initially to carry out a PEVOS study with young adults and adults. This will provide insights into the effectiveness of therapeutic approaches which could then also be used in therapy for children. Evaluation of therapy for children could then be undertaken in a follow-on study.

Collecting speech samples is labour intensive but is still the only way to gain objective, quantitative data about the effectiveness of stammering therapies. A sample collection using the telephone remains controversial as it is only a snapshot and cannot reflect the variability of stammering symptoms; on the other hand it is preferable, for this project, to individual contact because it is economical and is most likely to guarantee objectivity.

Collecting data via questionnaires is manageable with reasonable effort. Before the start of the study, researchers should look at which recent and thoroughly tested German-language questionnaires about emotions, attitudes and behaviours already exist which adhere to ICF classifications. If the previously used questionnaires were to be used, they must be reduced to those questions which will give valid results.

To enable the collection of valid data and have meaningful results it is necessary to take into account the different therapeutic approaches in equal measure. For that there are still some outstanding logistical problems which need solving, especially in respect of outpatient treatments.

It should be tested whether an web-based approach might be feasible. As internet access is becoming ever more wide spread it seems likely that it would be possibly in this fashion tor each a suitably large number of research subjects.

The main phase of PEVOS could begin after a short, preparatory phase. Unfortunately, it has not as yet been possible to gain funding. We are lacking funders, or rather a solid business model for the project. It is for this reason that PEVOS is currently in abeyance. This is regrettable as many member of the working party have invested much of their time in a voluntary capacity.

Only when we have comparable data about the effectiveness of German-language based stammering therapies would we be able to provide solid advice for those looking for therapy, and only then can clients make an evidence-based decision about which approach to pursue. This is also required for therapists who will need this information to continually improve the therapy which they offer to their clients.

1 comment:

Blanka Koffer said...
This comment has been removed by the author.