Key Points
The National MDS Natural History Study prospectively enrolls and banks samples from patients with a cytopenia and suspected MDS.
A 2-stage classifier was built to discriminate the presence of a myeloid malignancy or MDS in 1298 patients using mutations in 53 genes.
Abstract
The National Heart, Lung, and Blood Institute–funded National MDS Natural History Study (NCT02775383) is a prospective cohort study enrolling patients with cytopenia with suspected myelodysplastic syndromes (MDS) to evaluate factors associated with disease. Here, we sequenced 53 genes in bone marrow samples harvested from 1298 patients diagnosed with myeloid malignancy, including MDS and non-MDS myeloid malignancy or alternative marrow conditions with cytopenia based on concordance between independent histopathologic reviews (local, centralized, and tertiary to adjudicate disagreements when needed). We developed a novel 2-stage diagnostic classifier based on mutational profiles in 18 of 53 sequenced genes that were sufficient to best predict a diagnosis of myeloid malignancy and among those with a predicted myeloid malignancy, predict whether they had MDS. The classifier achieved a positive predictive value (PPV) of 0.84 and negative predictive value (NPV) of 0.8 with an area under the receiver operating characteristic curve (AUROC) of 0.85 when classifying patients as having myeloid vs no myeloid malignancy based on variant allele frequencies (VAFs) in 17 genes and a PPV of 0.71 and NPV of 0.64 with an AUROC of 0.73 when classifying patients as having MDS vs non-MDS malignancy based on VAFs in 10 genes. We next assessed how this approach could complement histopathology to improve diagnostic accuracy. For 99 of 139 (71%) patients (PPV of 0.83 and NPV of 0.65) with local and centralized histopathologic disagreement in myeloid vs no myeloid malignancy, the classifier-predicted diagnosis agreed with the tertiary pathology review (considered the internal gold standard).
Introduction
Patients with peripheral blood cytopenia can present a diagnostic and therapeutic challenge to clinicians and pathologists, with etiologies that range from nutritional deficiencies or infections to bone marrow failure disorders or malignancies. Refractory anemias, or bilineage and trilineage cytopenia in particular, can be typical manifestations of myelodysplastic syndromes (MDS) or of related, but distinct, other myeloid disorders.1,2 MDS is a heterogeneous collection of clonal hematopoietic malignancies characterized by poor overall survival because of ineffective hematopoiesis, progressive cytopenia, and transformation to acute myeloid leukemia (AML).3 It is critical to rule out MDS in patients who have alternative diagnoses as an explanation for their cytopenia to avoid overtreatment with potentially toxic agents and undue worry about future AML. It is also clinically valuable to discern other non-MDS myeloid malignancies for therapeutic intervention. Much of this diagnostic accuracy requires hematopathologic data synthesized with clinical and genetic phenotypes to provide a formal MDS diagnosis to the patient. At times, this is challenging.
Understanding the natural history and the risk of developing leukemia and other adverse outcomes is paramount in MDS and dependent on access to well-annotated biospecimens linked to robust clinical and molecular data. The National MDS Natural History Study (National MDS Study) funded by the National Heart, Lung, and Blood Institute is well positioned to facilitate these endeavors and answer diagnostic quandaries among patients with cytopenia. This initiative is being conducted in collaboration with community hospitals and academic medical centers supported by the National Cancer Institute.4 It enrolls patients with cytopenia undergoing evaluation for suspected or untreated MDS who undergo 2 independent histopathologic reviews (local and centralized) and are independently assigned a diagnosis, including MDS, myelodysplastic/myeloproliferative neoplasms (MDS/MPNs), AML with blasts <30%, or an alternative diagnosis. The National MDS Study relies on additional tertiary pathology review to adjudicate disagreements for disease classification when local and centralized pathology review are discordant. This is critical because previous reports note that there can be discordance in a diagnosis of MDS between pathologists, which can affect patient care.5,6
Emerging data suggest that next generation sequencing, along with cytogenetics and clinical variables, may improve MDS diagnostic and prognostic precision.7-10 This reflects the complexity of these syndromes and the evolving role of mutational analyses to complement morphology and conventional metaphase karyotyping to establish a diagnosis of MDS. The goal of this study was to evaluate the extent to which targeted exon gene sequencing of bone marrow-derived DNA and associated single nucleotide variants and small insertions and deletions in 53 genes could be used to accurately diagnose myeloid malignancy and MDS, and how this information could be used reduce disagreement between histopathology assessments. Clinically, this is of paramount importance for hematopathologists to make accurate and timely diagnoses to guide hemato-oncologists caring for patients with cytopenia. In addition, we wanted to understand the impact of using variant allele frequencies (VAFs; with mutation percentage as a continuous variable) vs binary (simply the presence or absence of the mutant gene) mutational profiles on classifier performance because reporting practices may vary in the clinic.
Methods
Samples and pathology review
Figure 1A shows the path of a case through the diagnosis process for the National MDS Study. Clinic sites submit unstained slides of diagnostic bone marrow aspirates and biopsies and peripheral blood samples to the central lab along with a report containing the diagnosis determined by the pathologist at the local site, using the World Health Organization (WHO) 2016 classification.11 Of note, the central and tertiary reviewers (board-certified hematopathologists) practice in tertiary referral academic practices, whereas local pathology review is a blend of academic and community practitioners. Supplemental Table 1 provides an additional description of the type of data that were available to pathologists. The pathologists did not have consistent access to the centrally performed targeted exon sequencing results generated by the study. However, if genetic testing was performed at the local site, these results were shared with all pathologists.
The protocol is approved through the National Cancer Institute Central Institutional Review Board.
Initial classification for the National MDS Study
The bone marrow slides and peripheral blood smears provided by the local centers were collected and subsequently stained at the central lab, where the study pathologist conducted a blinded review of the submitted clinical data for diagnostic determination. For disagreements (ie, discordance), a designated independent pathologist (tertiary review) who was external to the study provided an unblinded final determination and classification of the patient’s diagnosis (Figure 1A). For the analysis, local, central, and tertiary diagnoses were classified as either a myeloid malignancy or not, followed by a classification to either MDS or non-MDS myeloid malignancy. Myeloid malignancies were defined as any diagnosis with MDS, MDS/MPN, AML with blasts <30%, AML, and other (not included in WHO 2016 classification), with the subclass information entered by the pathologist indicating a myeloid malignancy (eg, other MPNs). MDS/MPN and other MPNs were classified as non-MDS myeloid malignancy. Clonal cytopenia of undetermined significance (CCUS) referred to a cytopenia with the absence of a histopathologic myeloid cancer but the presence of an abnormal clonal karyotype (including loss of chromosome Y) or a gene mutation with a VAF ≥ 2% (single nucleotide variants) and ≥ 5% (insertions and deletions) in 1 of the 53 manually reviewed genes. Dysplasia alone in <10% of the cells, without evidence of clonality by metaphase karyotype or mutations, did not meet criteria for MDS or CCUS. Patients with CCUS were labeled as having no myeloid malignancy for this study.
Targeted gene-panel sequencing
Targeted exon sequencing of 96 genes (supplemental Table 2) was performed using DNA extracted from marrow specimens, as previously described.12,13 NovaSeq 6000 was used for deep sequencing, at a mean coverage of 1284× and mean breadth (bases covered at ≥100×) of 99.87%. Reads were aligned against a patched version of the build GRCh38 using BWA-MEM (version 0.7.15).14 VarScan2 was used to detect single nucleotide variants and insertions and deletions with a minimum VAF of 2% and 5%, respectively.15 Of note, FLT3 internal tandem duplication variants and their allele frequencies were excluded from the model because of the inaccuracy of VAF estimates. VAFs were not adjusted for chromosome X or autosome ploidy. Resulting variants from 53 of the 96 genes were manually reviewed (by authors M.J.W., R.C.L., and R.B.) and included in the final analysis if they were considered likely disease-causing variants (somatic and germ line; supplemental Table 2). The following 2 sets of inputs were separately considered for building the 2-stage diagnostic classifier: (1) the maximum VAF across ≥1 mutations for each gene and (2) a binary indicator variable for each gene that indicated whether the gene had ≥1 mutation.
Two-stage diagnostic classifier
The diagnostic classifier consists of 2 stages, including an outer model and a conditional inner model. The outer model first predicts myeloid malignancy vs no myeloid malignancy. Then, for those patients predicted to have a myeloid malignancy, an inner model predicts MDS vs non-MDS myeloid malignancy. Both the outer and inner models were fit using lasso-regularized logistic regression, using a 10-fold diagnosis-stratified cross validation as implemented in the glmnet R package.16 The outer model was stratified using the following 3 strata: (1) myeloid malignancy = no; (2) myeloid malignancy = yes and MDS = no; and (3) myeloid malignancy = yes and MDS = yes. The inner model was stratified as MDS = yes/no. The lasso-regularized logistic regression parameter λ that resulted in the most regularized model within 1 standard error of the minimum mean cross-validated deviance was used to determine the best model. A probability threshold that maximized the F0.5 score, valuing precision twice as high as recall, was chosen to classify diagnoses from the predicted probabilities for the outer and inner models. To assess the performance, robustness, and ranking of estimated coefficients, the 2-stage conditional model was fit for 1000 bootstrap iterations, obtained by resampling, with replacement, the input data (VAF or binary mutational profiles) representing random draws of the sample size from our patient population. Each bootstrap sample was used as training data for the outlined 2-stage conditional model process, and those samples that were not selected in the bootstrap sample were used for testing. Performance metrics, selected genes, and coefficient ranking (1 being highest and 53 being lowest based on absolute model coefficients) were recorded for the 1000 bootstrap replicates, and the 2.5th, 50th, 97.5th percentiles were used to determine a 95% confidence interval (CI) and the median for each metric. To assess how this approach can improve histopathology, we ran the model fitting and performance evaluation process among those patients who had agreeing local and central histopathology reviews. The fitted model was then tested by using it to classify patients with discordant local and central histopathology diagnoses and comparing the predicted classifications to the third-party histopathology diagnoses to assess performance.
Results
Demographic characteristics of analyzed patients
Of the 1354 patients enrolled in the National MDS Study who had baseline bone marrow sequenced, 11 patients withdrew their consent before review, 1 met an exclusion criterion before histopathology assessment, and 44 had insufficient sample material to make a diagnosis (Figure 1B). The remaining 1298 participants were used for the analysis: 509 patients (39%) had a final diagnosis of myeloid malignancy and 789 did not have this diagnosis. Within the myeloid malignancy cohort, 342 (67%) had a diagnosis of MDS (and 167 did not) after resolving any ties between local and central pathology disagreements via tertiary pathology reviews. For a more detailed summary on alternative diagnoses that were not myeloid diagnoses, see supplemental Table 3A. Forty-seven patients, whose final diagnoses were not included in the WHO 2016 classification but the subclass indicated a myeloid malignancy, were also included in the myeloid malignancy cohort. Demographics of the 1298 analyzed patients are summarized in Table 1 and supplemental Table 3B. Subject-level metadata are available in supplemental Table 4.
Disease group . | . | N (%) . | ||||
---|---|---|---|---|---|---|
MDS . | Non-MDS myeloid malignancy . | All myeloid malignancies . | No myeloid malignancy . | All . | ||
342 (26) . | 167 (13) . | 509 (39) . | 789 (61) . | 1298 . | ||
MDS WHO diagnosis | MDS with single lineage dysplasia | 20 (6) | - | - | - | - |
MDS with single lineage dysplasia and ring sideroblasts | 27 (8) | - | - | - | - | |
MDS with multilineage dysplasia | 93 (27) | - | - | - | - | |
MDS with multilineage dysplasia and ring sideroblasts | 56 (16) | - | - | - | - | |
MDS with excess blasts-1 (5%-9% blasts) | 53 (15) | - | - | - | - | |
MDS with excess blasts-2 (10%-19% blasts) | 55 (16) | - | - | - | - | |
MDS with isolated del(5q) | 16 (5) | - | - | - | - | |
MDS, unclassifiable | 22 (6) | - | - | - | - | |
Undefined | 0 (0) | - | - | - | - | |
Sex | Female | 118 (35) | 59 (35) | 177 (35) | 343 (43) | 520 (40) |
Male | 224 (65) | 108 (65) | 332 (65) | 446 (57) | 778 (60) | |
Age | Median (minimum-maximum) | 74.0 (30.0-95.0) | 74.0 (25.0-95.0) | 74.0 (25.0-95.0) | 71.0 (18.0-95.0) | 72.0 (18.0-95.0) |
<50 | 9 (3) | 9 (5) | 18 (4) | 62 (8) | 80 (6) | |
50-59 | 26 (8) | 10 (6) | 36 (7) | 107 (14) | 143 (11) | |
60-69 | 73 (21) | 36 (22) | 109 (21) | 201 (25) | 310 (24) | |
70-79 | 152 (44) | 64 (38) | 216 (42) | 277 (35) | 493 (38) | |
80-89 | 70 (20) | 41 (25) | 111 (22) | 130 (16) | 241 (19) | |
90+ | 12 (4) | 7 (4) | 19 (4) | 12 (2) | 31 (2) | |
Race | American Indian or Alaska Native | 2 (1) | 0 (0) | 2 (<1) | 7 (1) | 9 (1) |
Asian | 2 (1) | 6 (4) | 8 (2) | 16 (2) | 24 (2) | |
Native Hawaiian or other Pacific Islander | 1 (<1) | 0 (0) | 1 (<1) | 1 (<1) | 2 (<1) | |
Black or African American | 13 (4) | 4 (2) | 17 (3) | 49 (6) | 66 (5) | |
White | 316 (92) | 156 (93) | 472 (93) | 701 (89) | 1173 (90) | |
Multiracial | 0 (0) | 0 (0) | 0 (0) | 3 (<1) | 3 (<1) | |
Unknown/not reported | 8 (2) | 1 (1) | 9 (2) | 12 (2) | 21 (2) | |
Ethnicity | Hispanic or Latino | 11 (3) | 5 (3) | 16 (3) | 21 (3) | 37 (3) |
Not Hispanic or Latino | 320 (94) | 160 (96) | 480 (94) | 753 (95) | 1233 (95) | |
Unknown/not reported | 11 (3) | 2 (1) | 13 (3) | 15 (2) | 28 (2) | |
IPSS-R | Very low | 68 (20) | - | - | - | - |
Low | 103 (30) | - | - | - | - | |
Intermediate | 62 (18) | - | - | - | - | |
High | 35 (10) | - | - | - | - | |
Very high | 32 (9) | - | - | - | - | |
Missing | 42 (12) | - | - | - | - | |
Karyotype classification | Normal | 156 (46) | 75 (45) | 231 (45) | 483 (61) | 714 (55) |
Abnormal | 136 (40) | 61 (37) | 197 (39) | 95 (12) | 292 (22) | |
Undetermined (no metaphases reported) | 5 (1) | 1 (1) | 6 (1) | 24 (3) | 30 (2) | |
Missing | 45 (13) | 30 (18) | 75 (15) | 187 (24) | 262 (20) |
Disease group . | . | N (%) . | ||||
---|---|---|---|---|---|---|
MDS . | Non-MDS myeloid malignancy . | All myeloid malignancies . | No myeloid malignancy . | All . | ||
342 (26) . | 167 (13) . | 509 (39) . | 789 (61) . | 1298 . | ||
MDS WHO diagnosis | MDS with single lineage dysplasia | 20 (6) | - | - | - | - |
MDS with single lineage dysplasia and ring sideroblasts | 27 (8) | - | - | - | - | |
MDS with multilineage dysplasia | 93 (27) | - | - | - | - | |
MDS with multilineage dysplasia and ring sideroblasts | 56 (16) | - | - | - | - | |
MDS with excess blasts-1 (5%-9% blasts) | 53 (15) | - | - | - | - | |
MDS with excess blasts-2 (10%-19% blasts) | 55 (16) | - | - | - | - | |
MDS with isolated del(5q) | 16 (5) | - | - | - | - | |
MDS, unclassifiable | 22 (6) | - | - | - | - | |
Undefined | 0 (0) | - | - | - | - | |
Sex | Female | 118 (35) | 59 (35) | 177 (35) | 343 (43) | 520 (40) |
Male | 224 (65) | 108 (65) | 332 (65) | 446 (57) | 778 (60) | |
Age | Median (minimum-maximum) | 74.0 (30.0-95.0) | 74.0 (25.0-95.0) | 74.0 (25.0-95.0) | 71.0 (18.0-95.0) | 72.0 (18.0-95.0) |
<50 | 9 (3) | 9 (5) | 18 (4) | 62 (8) | 80 (6) | |
50-59 | 26 (8) | 10 (6) | 36 (7) | 107 (14) | 143 (11) | |
60-69 | 73 (21) | 36 (22) | 109 (21) | 201 (25) | 310 (24) | |
70-79 | 152 (44) | 64 (38) | 216 (42) | 277 (35) | 493 (38) | |
80-89 | 70 (20) | 41 (25) | 111 (22) | 130 (16) | 241 (19) | |
90+ | 12 (4) | 7 (4) | 19 (4) | 12 (2) | 31 (2) | |
Race | American Indian or Alaska Native | 2 (1) | 0 (0) | 2 (<1) | 7 (1) | 9 (1) |
Asian | 2 (1) | 6 (4) | 8 (2) | 16 (2) | 24 (2) | |
Native Hawaiian or other Pacific Islander | 1 (<1) | 0 (0) | 1 (<1) | 1 (<1) | 2 (<1) | |
Black or African American | 13 (4) | 4 (2) | 17 (3) | 49 (6) | 66 (5) | |
White | 316 (92) | 156 (93) | 472 (93) | 701 (89) | 1173 (90) | |
Multiracial | 0 (0) | 0 (0) | 0 (0) | 3 (<1) | 3 (<1) | |
Unknown/not reported | 8 (2) | 1 (1) | 9 (2) | 12 (2) | 21 (2) | |
Ethnicity | Hispanic or Latino | 11 (3) | 5 (3) | 16 (3) | 21 (3) | 37 (3) |
Not Hispanic or Latino | 320 (94) | 160 (96) | 480 (94) | 753 (95) | 1233 (95) | |
Unknown/not reported | 11 (3) | 2 (1) | 13 (3) | 15 (2) | 28 (2) | |
IPSS-R | Very low | 68 (20) | - | - | - | - |
Low | 103 (30) | - | - | - | - | |
Intermediate | 62 (18) | - | - | - | - | |
High | 35 (10) | - | - | - | - | |
Very high | 32 (9) | - | - | - | - | |
Missing | 42 (12) | - | - | - | - | |
Karyotype classification | Normal | 156 (46) | 75 (45) | 231 (45) | 483 (61) | 714 (55) |
Abnormal | 136 (40) | 61 (37) | 197 (39) | 95 (12) | 292 (22) | |
Undetermined (no metaphases reported) | 5 (1) | 1 (1) | 6 (1) | 24 (3) | 30 (2) | |
Missing | 45 (13) | 30 (18) | 75 (15) | 187 (24) | 262 (20) |
MDS/MPN, AML with blasts less than 30%, and ICUS.
ICUS, idiopathic cytopenias of undetermined significance; IPSS-R, Revised International Prognostic Scoring System.
Genetic characteristics of analyzed patients
Of these 1298 patients, a total of 314 of 789 patients with no myeloid malignancy (40%), and 451 of 509 with a myeloid malignancy (89%) including 299 of 342 cases with MDS (87%) had at least 1 variant detected in 46 of 53 manually reviewed genes. These genes are known to be mutated in myeloid diseases, including spliceosome genes, epigenetic modulators, transcription factors, activated signaling/RAS pathway genes, and cohesin genes. Subject-level variant information is available in supplemental Table 5.
Genes ranked based on the decreasing median VAF and binary mutational profiles for myeloid vs no myeloid malignancy and MDS vs non-MDS myeloid malignancy showed distinct mutational profiles (Figure 2). The top 10 mutated genes based on the final diagnosis in the group with myeloid malignancy included TET2 (29.1%), ASXL1 (25.1%), SF3B1 (18.5%), SRSF2 (17.5%), DNMT3A (14.9%), TP53 (12.2%), RUNX1 (10.6%), U2AF1 (10.6%), IDH2 (8.4%), and STAG2 (6.9%), whereas the top genes for the MDS group included TET2 (26.3%), SF3B1 (23.1%), ASXL1 (22.8%), DNMT3A (16.3%), SRSF2 (15.8%), TP53 (14.3%), U2AF1 (11.7%), RUNX1 (9.6%), STAG2 (9.6%), and IDH2 (6.7%; Figure 2).
Building a diagnostic classifier based on gene-level mutational profiles for refining the diagnosis of myeloid malignancy and MDS
Our goal was to build a 2-stage classifier that could predict the presence of any myeloid malignancy in a patient with cytopenia and then predict the diagnosis of MDS. Patients would first be classified as either having a myeloid or no myeloid malignancy, followed by a MDS vs non-MDS myeloid malignancy, if the first outcome was favoring myeloid malignancy. For each of these 2 stages, we applied a 10-fold diagnosis-stratified cross validation to fit a lasso-regularized logistic regression model to determine a subset of the 53 reviewed genes that was sufficient for best predicting the respective outcome. The inner model (MDS vs non-MDS myeloid malignancy) was programmed and validated using those cases for which the outer model (myeloid vs no myeloid malignancy) predicted a myeloid malignancy. To perform these analyses, we used 1298 participants with a final diagnosis of myeloid malignancy (yes: N = 509 and no: N = 789) and within this myeloid malignancy cohort, with a final diagnosis of MDS (yes: N= 342; no: N= 167).
The performance of the diagnostic classifier for each stage was evaluated via bootstrap resampling (Table 2). The VAF based model achieved a median bootstrap positive predictive value (PPV) of 0.84, negative predictive value (NPV) of 0.80, sensitivity of 0.66, specificity of 0.92, and an area under the receiver operating characteristic curve (AUROC) of 0.85 when classifying patients as having myeloid vs no myeloid malignancy, and a PPV of 0.71, NPV of 0.64, sensitivity of 0.70, specificity of 0.66, and an AUROC of 0.73 when classifying patients as MDS vs non-MDS malignancy.
Metric . | Median, lower CI, upper CI, and CI width based on 1000 bootstrap replicates . | |||
---|---|---|---|---|
Outer model (myeloid vs no myeloid malignancy) . | Inner model (MDS vs non-MDS malignancy) . | |||
VAF (17 genes) . | BIN (17 genes) . | VAF (10 genes) . | BIN (7 genes) . | |
Sensitivity | 0.66, 0.53, 0.76, and 0.23 | 0.65, 0.54, 0.76, and 0.22 | 0.70, 0.45, 0.87, and 0.42 | 0.73, 0.48, 0.9, and 0.42 |
Specificity | 0.92, 0.88, 0.95, and 0.07 | 0.91, 0.87, 0.94, and 0.07 | 0.66, 0.45, 0.84, and 0.39 | 0.58, 0.33, 0.76, and 0.43 |
Accuracy | 0.81, 0.77, 0.85, and 0.08 | 0.81, 0.77, 0.84, and 0.07 | 0.67, 0.58, 0.75, and 0.17 | 0.66, 0.57, 0.73, and 0.16 |
PPV | 0.84, 0.78, 0.89, and 0.11 | 0.82, 0.77, 0.88, and 0.11 | 0.71, 0.61, 0.82, and 0.21 | 0.68, 0.59, 0.77, and 0.18 |
NPV | 0.8, 0.75, 0.86, and 0.11 | 0.80, 0.74, 0.85, and 0.11 | 0.64, 0.51, 0.77, and 0.26 | 0.63, 0.48, 0.78, and 0.3 |
F0.5 | 0.79, 0.75,0.83, and 0.08 | 0.78, 0.73, 0.82, and 0.09 | 0.70, 0.62, 0.78, and 0.16 | 0.69, 0.6, 0.76, and 0.16 |
AUROC | 0.85, 0.82, 0.88, and 0.06 | 0.85, 0.81, 0.88, and 0.07 | 0.73, 0.66, 0.8, and 0.14 | 0.69, 0.61, 0.77, and 0.16 |
Percent selection MDS∗ | 0.68, 0.57, 0.76, and 0.19 | 0.69, 0.6, 0.77, and 0.17 | n/a | n/a |
Metric . | Median, lower CI, upper CI, and CI width based on 1000 bootstrap replicates . | |||
---|---|---|---|---|
Outer model (myeloid vs no myeloid malignancy) . | Inner model (MDS vs non-MDS malignancy) . | |||
VAF (17 genes) . | BIN (17 genes) . | VAF (10 genes) . | BIN (7 genes) . | |
Sensitivity | 0.66, 0.53, 0.76, and 0.23 | 0.65, 0.54, 0.76, and 0.22 | 0.70, 0.45, 0.87, and 0.42 | 0.73, 0.48, 0.9, and 0.42 |
Specificity | 0.92, 0.88, 0.95, and 0.07 | 0.91, 0.87, 0.94, and 0.07 | 0.66, 0.45, 0.84, and 0.39 | 0.58, 0.33, 0.76, and 0.43 |
Accuracy | 0.81, 0.77, 0.85, and 0.08 | 0.81, 0.77, 0.84, and 0.07 | 0.67, 0.58, 0.75, and 0.17 | 0.66, 0.57, 0.73, and 0.16 |
PPV | 0.84, 0.78, 0.89, and 0.11 | 0.82, 0.77, 0.88, and 0.11 | 0.71, 0.61, 0.82, and 0.21 | 0.68, 0.59, 0.77, and 0.18 |
NPV | 0.8, 0.75, 0.86, and 0.11 | 0.80, 0.74, 0.85, and 0.11 | 0.64, 0.51, 0.77, and 0.26 | 0.63, 0.48, 0.78, and 0.3 |
F0.5 | 0.79, 0.75,0.83, and 0.08 | 0.78, 0.73, 0.82, and 0.09 | 0.70, 0.62, 0.78, and 0.16 | 0.69, 0.6, 0.76, and 0.16 |
AUROC | 0.85, 0.82, 0.88, and 0.06 | 0.85, 0.81, 0.88, and 0.07 | 0.73, 0.66, 0.8, and 0.14 | 0.69, 0.61, 0.77, and 0.16 |
Percent selection MDS∗ | 0.68, 0.57, 0.76, and 0.19 | 0.69, 0.6, 0.77, and 0.17 | n/a | n/a |
BIN, input matrix based on 0 and 1 encoding any variant presence /absence in a gene; VAF, input matrix based on maximum variant allele frequency.
Percent selection of subjects with MDS diagnosed by the outer model as “myeloid malignancy = yes,” which was then used as input for the inner model.
Contrasting ROC curves with associated 95% CIs between VAF and binary mutational profiles based on bootstrap resampling demonstrated improved performance using VAF instead of binary mutational profiles for predicting MDS vs non-MDS myeloid malignancy, showing an upward shift of the median ROC with an overall higher AUROC (0.73 vs 0.69) and a narrower 95% bootstrap confidence band for this metric (Figure 3). Slightly narrower 95% bootstrap confidence bands for the ROC were also observed for VAF compared with the binary mutational profiles for myeloid vs no myeloid malignancy performance, but the AUROC for both were very similar (0.85). When inspecting the individual metrics, the gain in performance for VAF vs binary mutational profiles for MDS vs non-MDS myeloid malignancy was most attributable to a gain in specificity (0.66 vs 0.58), followed by PPV (0.71 vs 0.68).
The best performing outer model based on VAF retained 17 genes as sufficient to discriminate a diagnosis of myeloid malignancy vs no myeloid malignancy (supplemental Table 6A). This included, based on descending absolute model coefficient, the following: SF3B1, TP53, ASXL1, U2AF1, JAK2, NPM1, DDX41, STAG2, RUNX1, SRSF2, IDH2, PHF6, TET2, DNMT3A, BCOR, IDH1, and ZRSR2. For all of these genes, a higher VAF favored a diagnosis of myeloid malignancy. Among these genes, all but the last 3, BCOR, IDH1, and ZRSR2, had a selection frequency ≥95% among the 1000 bootstrap samples, indicating a high degree of internal stability of these selections. SF3B1 and TP53 were consistently selected as the top 2 genes based on the median rank of their absolute model coefficient across bootstrap samples. The best inner model selected 10 genes to differentiate between MDS and non-MDS myeloid malignancy (supplemental Table 6B). This included, based on descending absolute coefficient, the following: JAK2, STAG2, SF3B1, TET2, TP53, ASXL1, SRSF2, PHF6, IDH2, and CBL. The top 4 genes (JAK2, STAG2, SF3B1, and TET2) had a selection frequency ≥95% among bootstrap samples. Among these, higher VAFs for STAG2, SF3B1, and TP53 increased the likelihood of a MDS diagnosis, whereas higher VAFs for JAK2, TET2, ASXL1, SRSF2, PHF6, IDH2, and CBL reduced it. All predictive genes of the inner model, except for CBL, were also part of the outer model, resulting in 18 unique genes that are being used by the classifier.
Predictive value of the 2-stage conditional model to enhance pathology assessments
Next, we assessed how well our chosen approach could resolve the disagreement between local and central pathology, using the final diagnosis by the tertiary pathology review to assess the performance of our model. To achieve this in an unbiased way, we retrained the same 2-stage conditional model on a subset of 1108 of the 1298 patients (85%) for whom the local and central pathology were in agreement without conflicting the tertiary pathology diagnosis (for example, we excluded cases with local = myeloid malignancy, central = myeloid malignancy, and tertiary = no myeloid malignancy). This approach allowed us to assess the resolution on pathology disagreement without overestimation of true performance. If we would have reused the model that was programmed using all data, the resolution on disagreement estimates would have been biased owing to the inclusion of all data, including the discordant cases in the training data. This training data set of 1108 patients included 261 patients (24%) with agreeing MDS diagnoses (supplemental Figure 1).
We then evaluated the performance on those cases that were in disagreement between local and central pathology (supplemental Table 7). This cohort included 139 patient diagnoses with disagreement for myeloid malignancy vs no myeloid malignancy and 25 with disagreement for MDS vs non-MDS myeloid malignancy. The latter were restricted to those for whom the local and central pathology review showed disagreement for MDS vs non-MDS diagnoses but showed agreement for myeloid malignancy.
Among the 139 patients with a discordant myeloid malignancy diagnosis, the VAF-based model reassigned 99 patients (accuracy of 71%) to the final diagnosis given by the tertiary pathology reviewer (considered the internal gold standard here) based on a subset of 16 predictive genes (supplemental Table 7). This included the resolution of 40 of 72 myeloid malignancy cases (56%) that had been classified as having no myeloid malignancy by local or central pathology (PPV of 0.83) and 59 of 67 no myeloid malignancy cases (88%) that were misclassified as having a myeloid malignancy by the local or central pathologist (NPV of 0.65).
Of the 25 patients with disagreeing MDS diagnoses, 19 patients (76%) were classified by the outer model as having a myeloid malignancy. The inner model correctly resolved the diagnoses for 11 of 19 cases (58%) based on 7 predictive genes (supplemental Table 7). This included 4 of 7 MDS cases (57%) that had been classified as non-MDS myeloid malignancy by local or central pathology (PPV of 0.44) and 7 of 12 non-MDS myeloid malignancy cases (58%) that were misclassified as MDS by the local or central pathologist (NPV of 0.70).
The reassignment ability of the model is useful to adjudicate discordant local and central pathology classification when used solely without morphological information. In clinical practice, histopathologic data would be available with sequencing results when providing diagnostic categorization. To demonstrate the additional clinical relevance of using the 2-stage conditional model to diagnose myeloid malignancy, we calculated the accuracy of the diagnosis with morphology alone conducted by 2 reviewers compared with the diagnosis with morphology plus using the 2-stage classifier based on targeted exon sequencing to resolve discordant cases. For morphology alone, an accuracy of 0.89 was achieved between local and central pathology review for myeloid malignancy, which was increased to 0.97 when resolving 99 of the 139 disagreeing diagnoses using the VAF-based outer model.
Discussion
MDS develops as hematopoietic stem cells acquire mutations in genes involved in various cellular pathways, including epigenetic regulation, transcription, growth signaling, and the spliceosome.17 Mutations in the same genes also play a role in the pathophysiology of other related myeloid malignancies, such as MPNs or AML, but these conditions are treated differently, and, thus, diagnostic accuracy is clinically relevant. MDS remains challenging to diagnose, given that criteria for morphologic dysplasia and evaluation may be subject to high interobserver variability.18,19 In 2022, we also found ourselves in a unique situation that involved 2 histologic classification systems for MDS, both of which incorporate genetic classification.20,21 Consequently, there is a growing need to leverage genetic sequencing information for MDS diagnosis. The National MDS Study provides a unique opportunity to address the incorporation of gene mutation data in the diagnostic evaluation of patients with cytopenia. We developed a 2-stage model based on the VAFs of gene mutations to first predict whether a patient has a myeloid malignancy, initially classifying as either having myeloid or no myeloid malignancy. Next, we predicted the diagnosis of MDS vs non-MDS myeloid malignancy, when the first outcome favored a myeloid malignancy. This novel 2-stage conditional model achieved a PPV of 0.84 and NPV of 0.8, with an AUROC of 0.85, when classifying patients as having either myeloid vs no myeloid malignancy based on the VAFs of 17 genes, a PPV of 0.71, and NPV of 0.64 with an AUROC of 0.73 for MDS vs non-MDS malignancy classification based on 10 genes.
We attribute the reduced performance of discerning MDS from non-MDS myeloid malignancy to non-MDS myeloid diagnoses (AML with ≥30% blasts, AML with <30% blast, and MDS/MPN overlap) often harboring similar mutations in the 53 tested genes. This was evident by the more similar mutational profiles for patients with MDS vs non-MDS myeloid malignancy (Figure 2). The increased challenge of separating MDS from non-MDS myeloid malignancy was also reflected by the wider 95% bootstrap CIs of performance metrics compared with that of the myeloid vs no myeloid malignancy model (Table 2). Our conservative approach of valuing PPV (ie, precision) twofold more than sensitivity (ie, recall) (by optimizing for the F0.5 score), led to a higher PPV and reduced sensitivity for both models. Clinically, we prioritized PPV over sensitivity to avoid the potential of overdiagnosis of MDS and subsequent overtreatment, because this is usually more problematic than undertreatment early in MDS.
In this study, 316 individuals were classified with CCUS (idiopathic cytopenias of undetermined significance with a clonal mutation or abnormal karyotype) based on the presence of a gene variant or a non-MDS-defining clonal karyotype abnormality and lack of dysplasia, and these patients were classified as having no myeloid malignancy using histopathology in this study.22 Using gene mutations to discriminate between CCUS and MDS, in particular, low-grade MDS, remains a challenge because both diagnoses share similar clinical characteristics, gene mutations, and disease progression risks.2,22-24 Although inclusion of a large number of CCUS cases in this study likely reduced the performance of the model to discriminate myeloid vs no myeloid malignancies, their inclusion reflects the real-world evaluation of patients with cytopenia. Consistent with this caveat, MDS cases that were misclassified by the model as having no myeloid malignancy (N = 99) had lower International Prognostic Scoring System-R scores than that of MDS cases correctly classified as myeloid malignancy (N = 243; supplemental Figure 2). Although the utility of gene-panel sequencing results in discriminating patients with a myeloid malignancy vs CCUS has been reported,8,23,25,26 our study focuses on improving the diagnosis of MDS, particularly when the histopathologic diagnosis is not clear. Future studies from the National MDS Study, including having a larger number of patients, could provide a refined gene-panel model to improve classification of CCUS from myeloid malignancy and further improve the model’s ability to discriminate MDS from non-MDS myeloid malignancy cases.
Different clinical laboratories report the presence or absence of mutations with or without mutation VAFs. Here we demonstrated that the inclusion of VAF did improve the model's performance for discerning MDS from non-MDS malignancy, particularly by improving specificity, reducing the number of false positive MDS calls. The results for myeloid vs no myeloid malignancy were similar for VAF and binary mutational profiles. This suggests that gathering VAF information in the clinic is more beneficial for a MDS diagnosis and that binary profiles may be sufficient for assessing myeloid malignancy in general.
Our results show that the 2-stage conditional modeling approach, using adjudication from a tertiary pathology reviewer, was able to resolve 71% of cases (PPV of 0.83 and NPV of 0.65) for which myeloid malignancy diagnosis was discordant between local and central pathology. Overall, the accuracy between local and central pathology had increased from 0.89 to 0.97 when using the classifier as an additional step to resolve disagreeing myeloid malignancy vs no myeloid malignancy diagnoses. In a field in which second and even third and fourth opinions are common for some patients, using results from this classifier could provide additional reassurance in the clinic.
The relevance of the maximum VAFs in 17 genes for discerning myeloid vs no myeloid malignancy (SF3B1, TP53, ASXL1, U2AF1, JAK2, NPM1, DDX41, STAG2, RUNX1, SRSF2, IDH2, PHF6, TET2, DNMT3A, BCOR, IDH1, and ZRSR2) in MDS is well-established.2,23,25 For all of these genes, an increase in VAF increased the likelihood of patients being diagnosed with a myeloid malignancy by the classifier. A subset of 9 of these genes (JAK2, STAG2, SF3B1, TET2, TP53, ASXL1, SRSF2, PHF6, and IDH2) also helped discriminate between MDS vs non-MDS myeloid malignancy. Among these, higher VAFs in STAG2, SF3B1, and TP53 increased, whereas higher VAFs in JAK2, TET2, ASXL1, SRSF2, PHF6, and IDH2 decreased the likelihood of MDS. Mutations in the latter set of genes are more prevalent in MPNs, consistent with these findings. Using JAK2 as an example, it is likely that the reason for this relationship between VAF and diagnosis is the higher frequency of these genes in MPN rather the VAF itself. Interestingly, of the 18 mutated genes found diagnostically relevant here for MDS specifically, 13 genes were also present in the Bernard et al prognostic study for the latest iteration of the International Prognostic Scoring System-M.10 This suggests consistency across diagnostic and prognostic studies of myeloid diseases.
Despite the rigor of ensuring correct diagnosis (up to 3 pathologists) and using a consistent sequencing protocol and robust statistical approach, some limitations remain. Mutations in genes that were not included in the list of 53 genes that we sequenced and reviewed may contain important diagnostic information. Cytogenetics were also not incorporated in the model; however, it is less likely there would be diagnostic uncertainty in patients with abnormal karyotypes. Inclusion of patients up to 12 months after diagnosis may be relevant, potentially biasing the spectrum of somatic mutations. However, the genes of only a small fraction of previously diagnosed patients were analyzed, making it unlikely to affect the results. We note the higher agreement between the central and the tertiary pathology review. Among the 139 cases in which local and central pathologists disagreed, the tertiary review agreed with the central reviewer in 106 cases (76%) compared with 33 cases (24%) with the local reviewer. We attribute this bias to the cumulative nature of the data review process because the tertiary pathology had access to the central pathology report. In addition, it could reflect a difference in the volume of cases reviewed (lower volume at local vs higher volume of reviews at central/tertiary academic centers where the central review or tertiary review occurred). Finally, our model cannot discern diagnoses for patients who have no variants reported. Overall, 533 of the 1298 patients analyzed (41%), including 58 of the 509 with a myeloid malignancy (11%, including 43 of 342 with MDS [13%]) and 475 of the 789 with a no myeloid malignancy diagnoses (60%), did not report any variants for the 53 curated genes. Certain minority populations have also been shown to have differences in clinical outcomes and genomic characteristics related to MDS.27 Therefore, given the limited number of minority patients in our cohort, future studies will be needed to assess the predictive ability of our model for a more diverse population.
In conclusion, the new 2-stage classifier based on 18 genes can be applied alone or in combination with morphologic review to predict a diagnosis of myeloid malignancy and MDS for a patient with cytopenia, especially when the bone marrow morphology is less definitive. Moving toward that goal, we made an online version of the presented 2-stage diagnostic classifier available at https://thenationalmdsstudy.net for clinical use. Ultimately, building future classifiers using a larger set of genes, the karyotype, and other relevant information may improve our ability to predict for MDS vs non-MDS myeloid malignancy. Also, developing a classifier to test peripheral blood samples from a patient with cytopenia before performing a bone marrow biopsy may delay or even defer a marrow assessment if the classifier returns a low-likelihood of MDS result, prompting further evaluation of other causes first. Ultimately, integrating genetic information into MDS classification schemes is paramount to help establish the appropriate therapeutic interventions to alter the natural history of MDS.
Acknowledgments
The authors thank the patients and participating centers. Additionally, the authors are grateful to Andy Brunner at the Massachusetts General Hospital for the helpful scientific discussions.
The National MDS Natural History Study has been supported by US Federal Government Contracts HHSN268201400003I and HHSN268201400002I from the National Heart, Lung, and Blood Institute and additional funding by the National Cancer Institute toward NCI Community Oncology Research Program and the participating Clinical Centers.
The content is solely the responsibility of the authors and does not represent the policy of the National Institutes of Health or the Department of Health and Human Services.
M.J.W. is supported by the Edward P. Evans Foundation.
Authorship
Contribution: J.B.G., A.E.D., and M.J.W. designed the study and wrote the manuscript; J.B.G. and T.G. performed the statistical analyses; D.H., M.A.S., N.L.D., and S.H.W. contributed to the study design and interpretation of the data; L.Z. and S.H.K. collected clinical data and samples and contributed to study design and interpretation of the data; M.J.W., R.C.L., H.R., and R.B. processed and reviewed the sequencing data; J.D., S.G., T.A.B., J.M., J.L., E.P., R.K., W.S., G.A., A.H., H.R., and R.F. contributed to study design and interpretation of the data; and all authors contributed to preparation of the manuscript and approved its content.
Conflict-of-interest disclosure: A.E.D. reports consultancy and membership on a board or advisory committee for Taiho, Novartis, and Bristol Myers Squibb and membership on a board or advisory committee for Takeda. R.C.L. reports consultancy for Takeda, bluebird bio, and Thermo Fisher Scientific. R.B. reports employment and stock in private company of Aptose Biosciences; has a chair of data safety monitoring committee in Gilead and Epizyme; has membership on a board or advisory committee in Silence Therapeutics; and receives research funding from Takeda. T.A.B. has stock in private company and membership on a board or advisory committee in Bristol Myers Squibb; has stock in private company in AstraZeneca, Epizyme, and Heron Therapeutics; and has membership on a board or advisory committee in MorphoSys, Karyopharm, and Cardinal Health. J.M. has membership on a board or advisory committee in bone marrow failure. E.P. receives research funding from Bristol Myers Squibb, Kura, and Incyte and receives honoraria from Taiho and Blueprint. R.K. receives honoraria from Novartis, Geron, and Acceleron, and receives honoraria from and is on the speakers bureau of Agios, AbbVie, Jazz, and Bristol Myers Squibb. M.A.S. reports membership on a board or advisory committee for Novartis, Takeda/Millenium, and Bristol Myers Squibb. The remaining authors declare no competing financial interests.
Correspondence: Amy E. DeZern, Oncology and Hematology, Comprehensive Cancer Center at Johns Hopkins, 1650 Orleans St, CRBI Room 3M87, Baltimore, MD 21287-0013; e-mail: adezern1@jhmi.edu.
References
Author notes
∗A.E.D. and J.B.G. contributed equally to this study.
The sequencing data is available in dbGaP at phs002714.v2.p1.
An online version of the classifier that can be used with either VAFs or binary mutation profiles is available at https://thenationalmdsstudy.net.
Data are available on request from the corresponding author, Amy E. DeZern (adezern1@jhmi.edu).
The full-text version of this article contains a data supplement.