Validity of MRS as outcome measure
How is the quality of the MRS scale to reliably measure treatment effects on HRQoL - even if symptoms are not so impressive? Can the result of the scale the judgment of the treating physician concerning treatment success predict? To this end, many clinicians use the term "validity" and mean high utility for clinical work or research. In so far it is important to address this issue.
Recently, we analyzed data of 9300 women with pre- and post- treatment data of the MRS scale [ 13 ]. The data came from an open post-marketing study in Germany with a product for hormone replacement therapy and the MRS scale was used as outcome measure (before an 6 months after hormone treatment). This was the basis to critically evaluate whether the scale is sensitive enough to measure the effect of hormone treatment independent from the severity of complaints.
The improvement of symptoms during treatment was equivalent to 36% of the baseline score, and similar also for all three subscales.
An improvement of complaints/QoL was seen in an increasing degree of severity of symptoms. An improvement of menopausal complaints during/after treatment was observed. The change relative to the baseline score was 36% in average. As can be seen in Figure 1, patients with little/no complaints before therapy improved by 11%, those with mild complaints at entry by 32%, with moderate by 44%, and with severe symptoms by 55% - compared with the baseline score. The scale is sensitive enough to detect even positive treatment effects in women with little complaints – although to a lesser degree.
This can be illustrated also with a different approach – the comparison with norm values for the population (Table 1). The percentage of women with no/little symptoms is much higher than those with severe complains in the normal population. Women eligible for therapy depict a much lower proportion with little or mild complaints and a much higher proportion of moderate or sever complaints as compared with the normal population. After 6 months of hormone treatment, however, the frequency distribution of patients with a certain severity of complaints returned towards a similar distribution as observed in the general population. The extreme proportion of patients with no/ little complaints after therapy should be interpreted with reservation, because of a possible patient selection that may have led to an overestimation of the treatment effect. Such an visual comparison (as in table 1) should be used only to get a crude idea for the interpretation of results because patients are usually differ from the general population, a difference hard to adjust for.
That means that the MRS scale showed a convincing ability to measure treatment effects on quality of life across the full range of severity of menopausal complaints. The MRS scale was also tested whether it predicts the subjective, clinical impression/opinion of the treating physician. At face value, the individually assessed efficiency of hormone treatment by the treating gynaecologists was comparable with the assessment by the MRS scale, i.e. using a simple dichotomization of the treatment effect in “successful” and “not successful” for both the subjective opinion of the physician and the result of the MRS scale (Table 2): The sensitivity (correct prediction of a positive assessment by the physician) was 70.8% and specificity (correct prediction of a negative assessment by the physician) 73.5%. In other words, the MRS scale fits well with the subjective assessment of the treatment by the physician. Both, sensitivity & specificity however might be over-estimated, because it is hard to estimate to which degree "clinical judgment" and "MRS assessment were really independent.The attached poster presented in 2005 at the Congress of the International Society of Pharamacoepidemiology (Nashville, USA) provides some additional aspects.
A new study based on 3282 women with pre- and post- hormone - treatment data was published 2006 [ 17 ]. The application of the MRS scale differed compared to the previous study [13 ]: self-administered vs. interview by physician. The hormone-therapy related improvement of complaints relative to the baseline score was about or less than 30% in total or domain scores, whereas it exceeded 30% improvement in the old study [ 13 ].
Similarly, the relative improvement after therapy, stratified by the degree of severity at baseline, was lower in the new than in the old study, but had the same slope.
Although one cannot exclude different treatment effects with the study method used, this supports our hypothesis that the individual MRS interviews performed by the physician obviously biased the results towards over-estimation of the treatment effects.
This hypothesis was underlined by the degree of concordance of physician’s assessment and patient’s perception of treatment success (MRS results): Sensitivity (correct prediction of the positive assessment by the treating physician) of the MRS and specificity (correct prediction of a negative assessment by the physician) were lower than the results obtained with the interview-based MRS scale in the previous publication.
Although the new study confirmed evidence for the capacity of the MRS scale to measure treatment effects on quality of life across the full range of severity of complaints before treatment, the authors concluded that the MRS scale should be used as self-administered instrument in clinical studies to avoid overestimation of treatment effects.
[13]. Heinemann LAJ, DoMinh T, Strehlow F, Gerbsch S, Schnitker J, Schneider HPG. Validity of the Menopause Rating Scale as outcome measure. A clinical outcome study. Health and Quality of Life Outcomes 2004;2:45 (2 September 2004). www.hqlo.com/content/2/1/67
[17] Dinger J, Zimmermann T, Heinemann LAJ, Stoehr D. Quality of life and hormone use: new validation results of MRS scale. Health and Quality of Life Outcomes, 2006;4:32. www.hqlo.com/content/4/1/32.
