Core set of patient-reported outcomes for myelodysplastic syndromes: an EUMDS Delphi study involving patients and hematologists

Myelodysplastic syndromes (MDS) represent clonal hematopoietic stem cell disorders, characterized by dysfunctional hematopoiesis, cytopenias, and a high symptom burden.¹  The disease trajectory is highly variable and may range from an indolent course to an early transformation to acute myeloid leukemia (AML). Patients often present with signs and symptoms related to cytopenias, such as impaired quality of life (QoL), weakness, bleeding episodes, and recurrent infections.

Treatment options for MDS range from supportive care, typically red blood cell and/or platelet transfusions, to disease-modifying therapies, including immunomodulators, chemotherapy drugs, and hematopoietic stem cell transplantation. Thus, risk assessment and individualized therapy planning represent an essential part in personalized care and efficient allocation of limited health resources.^2,3  As many therapeutic options are not curative in patients with MDS, it is essential to include outcome parameters that integrate changes in the MDS-related disease burden with a focus on QoL and functional outcomes.⁴  Therefore, patient-shared decision making and patient-reported outcomes (PROs) are increasingly in the focus of patients, authorities, and health care providers.^5,6  PROs are defined as “any report of the status of a patients’ health condition that comes directly from the patient, without interpretation of the patients’ response by a clinician or anyone else”⁷  and are therefore essential in health care to integrate the perception of patients with MDS. PROs can provide unique information on disease symptoms, treatment burden, and how these factors can impact everyday life.^8,9  In addition, assessment of PROs can support the dynamic treatment course by observing effectiveness and safety changes of the treatments over time and is now also valued by regulatory stakeholders.^5,6,10-12  The importance of PROs has also been confirmed within the ongoing European LeukemiaNet MDS (EUMDS) registry. The EUMDS registry was established in 2008 by a large group of hematologists who were part of the LeukemiaNet collaboration. As an observational, longitudinal study the EUMDS registry collects data on patients newly diagnosed with MDS, including 150 active centers from 18 countries. Clinical and patient-relevant data, including QoL, are collected in 6-month intervals and used to better understand the MDS course and improve treatment outcomes.¹³  QoL, as a PRO, also reached an agreement among EUMDS clinical experts to be included into a recently developed MDS core outcome set comprising a minimum set of outcomes for assessing treatment effectiveness in future MDS clinical trials.¹⁴  The researchers concluded that the inclusion of the perspective of patients and a stronger focus on PROs in MDS is essential to capture treatment success, also from the patient perspective.

Currently, a wide range of PRO measures (PROMs) are used to determine the QoL of patients with MDS, with cancer-specific or generic PROMs (eg, European Organization for Research and Treatment of Cancer [EORTC] Core 30 Items; EQ-5D) being the most commonly used.¹⁵  MDS-specific PROMs such as QoL-E¹⁶  and Quality of Life in Myelodysplasia Scale (QUALMS),¹⁷  are still rarely used despite having undergone validation and reliability testing.^18,19 

A recent review by DeMuro et al²⁰  reported on the implementation of new label claims until 2010 by comparing approvals from the European Medicines Agency and the US Food and Drug Administration. The results showed a wide range of plausible PROMs and lack of consistency in the PROs assessment between the 2 agencies. PROMs differed greatly in the structure and health aspects and domains they evaluated. Some PROMs are designed to be domain specific, focusing on specific aspects of health (eg, pain, fatigue, and activity limitations), or multidimensional with subscales that measure various aspects of health (eg, QoL, social functioning, or emotional functioning). A similar conclusion was reached by Gnanasakthy et al²¹  in the recent comparison of the same 2 regulatory agencies in terms of PRO labeling for oncological drugs. One of the reasons that clinical trials often combine multiple PROMs is to assess a wider range of possibly relevant health domains, leading to multiplicity and redundancy within studies and lack of consistency across studies.²² 

In the field of hematology, PROs have often been used as treatment outcome measures in clinical trials, whereas they are still poorly integrated into routine practice.²³  Although the importance of PROs has been well outlined in several MDS studies and international recommendations,^4-6,24,25  a wide range of PROMs has been applied, thereby often hampering a critical appraisal of study findings.¹⁵  The partly incomplete and redundant measurement of PRO items and domains, as well as the simultaneous usage of multiple PROMs may result in evidence gaps or biased outcome reporting, because of the selective choice of results, particularly if their utilization is not predetermined.²⁶  Given the need for guidance on the systematic use of PROs in MDS, we aimed to propose a minimum set of the most important PROs (core set of PROs) to be used in both clinical research and routine clinical practice.

Identification and selection of PROs was performed in 2 phases: (1) the development of a list of candidate PROs and (2) the selection of the core PROs. Ethical approval for the study was obtained from the Research Committee for Scientific and Ethical Questions at UMIT-University for Health Sciences, Medical Informatics, and Technology. The EUMDS registry (www.clinicaltrials.gov as NCT00600860) has been approved by the ethics committees of all participating centers and in accordance with the Declaration of Helsinki.

After the recommendations of the Core Outcome Measures in Effectiveness Trials (COMET) Initiative for developing core PROs,²⁷  we used available PROMs in MDS for deriving potential PROs. As QoL measures are a comprehensive source of PROs and are also frequently used PROMs in MDS,²⁵  we performed a systematic literature search (PubMed, Cochrane Library, Scopus, and Web of Science) on QoL instruments among MDS studies up to January 2016, to create a pool of PRO candidates. As suggested by Macefield et al,²²  the domains and items within each of the observed QoL instruments were extracted and categorized into potential PROs, involving researchers with MDS expertise.

The selection of core PROs was performed according to the recommendation of the COMET Initiative.^22,27 

In a 2-round Delphi survey, participants were asked to rate each PRO, expressing the perceived importance for using that specific PRO in future evaluation of MDS on a 9-point scale. First round results were not final but were reported back to participants in the second round to raise group consensus. In the second and final Delphi round, outcomes rated 7 to 9 by at least 70% of the participants and not rated 1 to 3 by more than 15%, were included among the core set of PROs. In contrast, outcomes rated 1 to 3 by at least 70% of the participants and not rated 7 to 9 by more than 15%, were excluded.²⁷  All remaining outcomes, were labeled “without consensus.”

The proposed inclusion/exclusion criteria was relevant for both groups. After the second Delphi round, any PRO that met the inclusion/exclusion criteria in either of the 2 groups was added/excluded from the core PROs for MDS.

For the Delphi survey, hematologists with clinical expertise in MDS were recruited within the European Horizon 2020 MDS-RIGHT project (https://mds-europe.eu/right) and the database of the EUMDS registry.¹³  Patients with MDS were recruited from the outpatient clinics of Saint Louis Hospital, Paris, France during routine visits and were not subjected to further eligibility requirements. The survey was conducted online (Google forms in English) by the hematologists, whereas a translated paper-based survey was distributed to the patients with MDS. The translation into French was performed by a certified translation company and reviewed by the MDS medical team in France (P.F., F.C.).

To assess the similarity of ratings between patients and hematologists for each of the 40 PROs and within each of the 2 Delphi survey rounds, we compared the PRO rating distribution (ie, shape, location, and spread) between the 2 groups by using a Mann-Whitney U test. Moreover, we calculated a mean rating for each of the 40 PROs and used Spearman’s rank correlation coefficient (ρ) to assess the correlation between patients’ and hematologists’ mean ratings across the 40 PROs, for each of the 2 Delphi rounds separately.

Missing data from the patients’ survey ratings (first round, 4.8%; second round, 2.0%), were imputed by the items’ median rating. Analyses were performed with STATA 15 (Stata Corp., College Station, TX).

Further details are available in supplemental Methods.

Results are presented for the 2 phases of our study, starting with the set of potential PROs derived from the literature review, followed by the Delphi survey-based selection of the core set of PROs.

The systematic literature search resulted in 2863 studies. After removal of the duplicates and screening publications by title/abstract, 104 publications were screened as full text. Twenty-three studies were excluded because they did not include a QoL instrument. The remaining 81 studies fulfilling the inclusion criteria were included in the review (Figure 1). Overall, we identified 12 generic, 6 cancer-specific, and 2 MDS-specific QoL instruments. Most commonly used were EORTC QLQ-C30 (26 studies), the Functional Assessment of Cancer Therapy-Anemia questionnaire (17 studies), the 36-item Short Form Health Survey questionnaire (16 studies), the Quality of Life for Hematologic Diseases questionnaire (10 studies), and the Functional Assessment of Cancer Therapy: Bone Marrow Transplantation questionnaire (7 studies). These 5 QoL instruments composed more than two-thirds (68%) of a total of 112 QoL assessments. MDS-specific QoL instruments (ie, QoL-E or QUALMS) were applied in only 10% of studies (supplemental Figure 1).

The extraction of PRO items and domains from these 20 QoL instruments yielded 40 nonredundant PROs, which were included in the 2-round Delphi survey process (Table 1).

Details on the process of extracting and categorizing PROs including the survey questionnaires are provided in the supplemental Information.

In the first Delphi round, 239 invitations were sent to hematologists from 18 countries. Overall, 1 person declined participation, 9 could not be reached, and 38 (17%) with long-term experience in MDS participated in the survey (Table 2). Eight of the 40 PROs were highly rated (>70% ratings of 7-9, sorted by the highest rate; Figure 2): “transfusion-dependency burden,” “general quality of life,” “basic mobility,” “ability to work/activities of daily living,” “physical activity,” “fatigue,” “general health,” and “loss of independence.”

For the first Delphi round, 40 patients with MDS from Saint Louis Hospital, Paris, France were asked and agreed to participate in the survey. Participating patients had an average age of 74 years, with a median of 21 months after diagnosis, mainly receiving supportive care (Table 3). Based on the obtained patient assessments, none of the proposed 40 PROs was rated 7 to 9 by more than 70% (Figure 2). Still, we observed lower ratings (>70% ratings of 1-3) for the following 6 of the proposed 40 PROs (sorted by lowest rates): “tremor,” “financial difficulties,” “headache,” “speaking difficulties/speech-language problems,” “urinary incontinence,” and “loss of independence.” Three additional PROs were suggested by the patients with MDS to be included in the next survey round: “difficulty in concentrating,” “dental problems,” and “fear of falling.”

All 239 hematologists were contacted again in the second round; 32 responded (14%). Thirty hematologists participated in both rounds (94%), among which 25 had more than 10 years of experience in MDS (Table 2). The final rating of the PROs resulted in the selection of the following 3 core PROs (sorted by highest rates, Figure 2): “transfusion-dependency burden,” “general quality of life,” and “ability to work/activities of daily living.”

In the second Delphi round, 38 patients with MDS completed the questionnaire, among which 14 participated in both rounds. The characteristics of the participating patients were similar to those of the previous round, with an average age of 76 years and a median of 17 months since diagnosis. Most of the patients were receiving disease-modifying therapy, followed by supportive care (Table 3). Despite the patients’ higher ratings in the second survey round, “general quality of life” was the single outcome fulfilling the strict inclusion criteria of a core PRO (Figure 2). Remaining (nonincluded) highly rated PROs among patients were “general health,” “fatigue,” “confidence in health care services,” “weakness,” and “disease knowledge.”

The overall 2-round Delphi rating of the PROs is represented in Figure 2 and supplemental Table 1.

In the first Delphi round, 28 of the 40 compared PRO rating distributions showed relevant differences (P < .05; Table 4) between the patients and hematologists. The second Delphi round decreased the differences in the ratings, with differences observed in only 19 of the 40 compared PROs, and most of the significant mean differences ranging from 1 to 2 points. From all the statistically significant rating differences the importance of “confidence in health care services,” “disease knowledge,” “need to rest,” and “body image” was rated higher by patients than by hematologists. All other significantly different PROs received higher importance ratings by physicians.

The overall mean rating comparison between patients and hematologists showed moderate correlation in both rounds (first round: ρ = 0.51; P < .001; second round: ρ = 0.54; P < .001).

In summary, the PRO rating distributions matched better between the 2 groups and the correlation of mean PRO ratings between groups increased in the second round when compared with the first round.

We used a systematic literature search and a 2-round Delphi process to develop the first core set of PROs (“general quality of life,” “transfusion-dependency burden,” and “ability to work/activities of daily living”), as a minimum set of PROs to be used for MDS health assessment in clinical research and daily practice.

QoL in patients with MDS is impaired in many aspects. Symptom burden clusters (eg, pain, fatigue, dyspnea, anxiety, and stress) related to disease activity, per se, or the treatment of MDS, play a significant role in QoL deterioration.^28,29  Consequently, an early detection of signs and symptoms along with multidimensional geriatric assessment, are important aspects of MDS care.³  However, many physicians still have limited understanding of the impact of the MDS diagnosis itself on a given patient, and they may have heterogeneous judgment of the symptom burden or patients’ adjustment to disease.^30,31  Therefore, QoL assessment has the potential to provide unique information that cannot be inferred by clinical and laboratory biomarkers alone. The joint agreement on the “general quality of life” by both groups confirms the high priority of this PRO in MDS for patients, as well as for hematologists. More and more MDS studies apply QoL evaluations, as most treatment options cannot change the disease course, and symptom relief is therefore a relevant and valuable outcome in most patients.³²  QoL is also part of the recently established quality care indicators for MDS³³  and has been valued as an important concept in shared decision making for MDS treatment.³⁴  Still, broader QoL implementation, particularly for regular monitoring in the daily clinical practice is conditioned on using validated and well-established QoL instruments that can properly evaluate patients’ perception in MDS. This and other obstacles including limited administrative staff, budget, and other logistic challenges could be overcome by applying electronic PRO surveys.⁶  According to a guideline of the COMET/Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative,^35,36  selection of outcome measurement instruments should be performed in 4 steps: (1) conceptual considerations (ie, specifying outcomes, domains, and targeted population); (2) performing a systematic analysis and/or a literature search to identify current measurement instruments; (3) evaluation of measurement properties and feasibility aspects of the identified measurement instruments (eg, content validity, reliability, responsiveness); and (4) general agreement in choosing measurement instruments (ie, consensus process among stakeholders for selecting a measurement instrument). Our rating results could serve in the selection of PRO domains, could provide an overview of actual QoL instruments and support future selection or development of an adequate QoL instrument.

In addition to “general quality of life,” the importance of the “transfusion-dependency burden” has been highlighted by the hematologists and also received a relatively high rating from patients. Considering that cytopenias are a major clinical problem and a risk factor for morbidities, hospitalization, and MDS progression,³⁷  transfusions of red blood cells and platelets form the basis of supportive care.³⁸  Transfusion dependency is a well-known prognostic factor, associated with adverse outcome, fatigue, and impaired QoL.^29,39  The transfusion threshold and intensity is not standardized and needs to be assessed individually.⁴⁰  This aspect is supported in the study by Buckstein et al⁴¹  who reported that more than 40% of the transfusion-dependent patients would like to receive their transfusions at higher hemoglobin (Hb) levels. More liberal transfusion schedule (>105 g/L Hb) was also associated with an improved QoL outcome and domains, in comparison with a restrictive transfusion arm (80 g/L Hb).⁴²  Therefore, wider inclusion of the patients’ preferences and assessment of the overall burden of regular transfusions, including side effects and relieve of anemic symptoms is important to optimize transfusion therapy and may lead to policy changes.

Associated mainly with symptoms of fatigue, MDS affects patients’ ability to work, lifestyle, and physical ability.^29,43  Reduced work productivity can have a serious impact on patients’ self-esteem, financial security, social relationships, and psychological well-being, especially for patients <65 years of age.^19,44  The ability to perform activities of daily living is therefore an important third core PRO. Health care providers have a key role and should regularly take the opportunity to assess the consequences of MDS, including patients’ capability to perform fundamental and routine daily activities (eg, ability to eat, walk, dress, maintain hygiene and good continence, home maintenance, and shopping for necessities).⁴⁵ 

Overall, PROs were rated lower by patients with MDS in comparison with hematologists, which may be explained in part by the fact that patients mostly rely on their own experience. However, given that enrolled patients had a median period after diagnosis of ∼20 months and >90% received some MDS therapy (Table 3), lack of experience may not be the only reason for these differences.

The PROs that were highly rated by patients (although not passing the inclusion threshold for the core PROs) included “general health,” “fatigue,” “confidence in health care services,” “weakness,” and “disease knowledge.” “Fatigue” and “weakness” are well-known disease symptoms among patients with MDS and can be only partly explained by Hb levels, and therefore, the subjective perception of these symptoms represents the gold standard for severity assessment.^28,46  Both “fatigue” and “weakness” are also part of the developed MD Anderson Symptom Inventory measure⁴⁷  for assessing symptom burden for AML and patients with MDS, which confirms our results from the patients’ perspective. Our findings are in line with the recent study by Bell et al⁴⁸  intended to increase its measurement sensitivity and to adapt the generic EORTC instrument to rare hematological diseases, including MDS. “Fatigue,” “weakness,” and “shortness of breath” were among the 10 included items.

A self-rated general health status is often included in questionnaires as a single PRO item, using question variations with different rating descriptions or linear scales in asking patients to rate their overall health.⁴⁹  Being associated with mortality, self-rated health is also arguably a useful single measure that can serve as an overall summary end point, thus reducing the burden of answering multiple items and being easier to understand and implement.^49,50 

For the PROs “disease knowledge” and “confidence in health care services,” patients gave significantly higher ratings than hematologists. These results are in line with findings from Sekeres et al⁵¹  who found poor knowledge and limited understanding of health status, disease classification, and treatment characteristics of patients with MDS. Similar evidence points to inconsistency between the preferences and disease perspectives of patients with MDS and physicians, with a need for better communication and information sharing.⁵² 

Differences in PRO prioritization between patients and physicians have also been observed in other comparable studies. Weisshaar et al⁵³  found substantial variations in ratings between patients and physicians, in a study for PRO selection in aplastic anemia and paroxysmal nocturnal hemoglobinuria. Issues related to QoL (eg, mood disturbances, fatigue, and impaired activities of daily living) were rated higher by patients than by expert hematologists, who prioritized physical constraints (eg, bleeding, fever, and hemoglobinuria).

In a study by Gerritsen et al⁵⁴  on identifying core PROs in pancreatic cancer, health care providers gave overall higher ratings than patients, especially for physical PROs such as nausea, pain, stomach problems, and vomiting.

Overall, previous oncology research involving consensus approaches has shown discrepancies in perception and valuing of PROs, in which patients prioritize symptoms related to their everyday health status, whereas physicians assess unfavorable clinical results (eg, emergency department admissions, mortality).⁵⁵ 

Our study has several limitations. First, despite our thorough search, it is still possible that we missed some QoL studies in our literature review. Second, for feasibility reasons we used a monocentric enrollment of patients with MDS from France, thereby possibly limiting generalizability of findings to patients in other countries and cultures. However, considering the patients’ characteristics, in terms of age, sex, and received therapies (Table 3), our study population is similar to that of other MDS registries and studies,^13,56  indicating good validity within the European setting. Third, as expected due to disease prognosis and the fact that MDS represents rare diseases, the number of patients participating across both Delphi rounds declined, which may influence the results of the Delphi survey by introducing attrition bias.²⁷  We attempted to reduce this bias by actively presenting the group results from the first round during the second round of the Delphi process, to provide equal knowledge within the Delphi rounds. Further research may validate our results with independent and larger samples of clinical experts and patients. Fourth, we restricted our work to hematologists and patients with MDS. However, other relevant stakeholders such as other health care professionals, regulatory agencies as well as informal caregivers should be included to increase broad relevance and wide applicability of PROs in research and practice.⁵⁷  Therefore, our work should be seen as an initial step in the process of continuing research rather than a set-in-stone core PRO set. For clinical trials, the core set of PROs should be integrated into the recently developed general core outcome set for MDS by Rochau et al¹⁴  as a set of minimum standardized outcomes for clinical studies.

In summary, the core set of PROs for MDS, derived from both patients and hematologists in this project should guide further research, facilitate the assessment of patients’ preferences, and support the communication toward informed and shared decision making. The choice of measurement instrument, timing, and frequency of measuring the 3 core PROs should be clearly predefined and reported. The COMET/COSMIN guideline³⁶  facilitates the selection of appropriate measurement instruments, whereas the overall plan for evaluating the core PROs, including timing and frequency, should be in accordance with the research question, natural disease course, treatment expectations and symptom trajectories.⁵⁸  Additional recommendations for reporting PROs from randomized clinical trials⁵⁹  or use of PROs for labeling claims^7,60  should be considered.

Future updates of the core set of PROs including other relevant MDS stakeholders and testing their validity in broader patient populations and wider settings is recommended.

The authors thank all patients, health professionals, and collaborators who generously shared their time and helped us in this research study, particularly the survey participants from the EUMDS Registry and MDS-RIGHT project; Christopher Marquardt (UMIT TIROL) for assistance in the graphical preparations; Helena Borba (Universidade Federal do Paraná, Paraná, Brazil) for assistance in the screening process; and Sabine Mair, translator and interpreter at Tradukisto (www.tradukisto.eu), for translation into French.

This study was supported by the European Union's Horizon 2020 Research and Innovation Program (grant agreement 634789, MDS-RIGHT), within a Personalising Health and Care Program (PHC-2014-634789); Translational Implementation of Genetic Evidence in the management of TRIAGE-MDS; Austrian Science Found I 1576) within the TRANSCAN, Primary and Secondary Prevention of Cancer Call (ERA Net); Erasmus Mundus Western Balkans (ERAWEB), a project established by the European Commission; and by Verein Senioren-Krebshilfe. The work was carried out within the EUMDS Registry, which is supported by an educational grant from Novartis Pharmacy B.V. Oncology Europe, Amgen Limited, Celgene International, and Janssen Pharmaceutica.

Contribution: A.C.-F., I.S., K.K., R.S., U.R., and U.S. designed the research; A.C.-F., A.S., C.v.M., D.B., E.M., F.C., F.E., H.G., I.S., K.K., M.A., M.M., P.F., R.S., S.P., T.d.W., U.R., and U.S. performed the research; A.C-.F., C.v.M., F.C., I.S., P.F., R.S., and U.R. collected the data; A.C.-F., A.S., C.v.M., D.B., E.M., F.C., F.E., H.G., I.S., K.K., M.A., M.M., P.F., R.S., S.P., T.d.W., U.R., and U.S. analyzed and interpreted the data; I.S., M.A., and S.P., performed the statistical analyses; A.C.-F., I.S., R.S., U.R., and U.S. wrote the manuscript; and A.C.-F., A.S., C.v.M., D.B., E.M., F.C., F.E., H.G., I.S., K.K., M.A., M.M., P.F., R.S., S.P., T.d.W., U.R., and U.S. reviewed the manuscript.

Conflict-of-interest disclosure: A.S. has received research funding from Pfizer and Sanofi and has received research funding from and has served on advisory committees for Celgene, Gilead, Janssen, MSD, Novartis, Roche, and Takeda. C.v.M. has been the project manager of the EUMDS Registry and MDS-RIGHT project and is funded by the EUMDS and MDS-RIGHT project budgets. E.M. has been a consultant for Alexion, Amgen, Celgene, Incyte, Novartis, Sanofi; has received travel reimbursements from Amgen, Celgene, Novartis, Pfizer, and Sanofi; and has received research funding from Gilead. F.E. has received personal fees from AbbVie, Amgen, Orsenix, Takeda, and Janssen and a research grant (institutional) from Amgen outside of the submitted work. M.M. has received research grants from Abbvie, Janssen, Medison, Neopharm/BMS, Novartis, Roche, and Takeda; has served on the advisory boards of Novartis and Onconova; has worked on clinical trials sponsored by BMS, Geron, Fibrogen, Novartis, Oconova, and Takeda; and has received consulting fees from Karyopharm. P.F. has received research funding (as GFM chairman) from Abbvie, Agios, Celgene/BMS, Jazz, and Novartis. R.S. has received honoraria from and held membership on the board of directors or an advisory committee of Celgene; has received honoraria from and has held membership on the board of directors or advisory committee of Novartis; and has received research funding from Teva. T.d.W. has received research funding from Amgen, Celgene, and Novartis, and an honorarium from Novartis as project coordinator of EUMDS. A.C.F., D.B., F.C., H.G., I.S., K.K., M.A., S.P., U.R., and U.S. declare no competing financial interests.