Validation and refinement of the Disease Risk Index for allogeneic stem cell transplantation

Although many factors influence the outcome of allogeneic hematopoietic cell transplantation (HCT), disease type and disease status at the time of transplantation are the strongest determinants of post-HCT survival. It is therefore essential to categorize patients by disease risk in HCT studies. This is already feasible for other variables, such as donor HLA match¹  or comorbidity burden.²  The Disease Risk Index (DRI) was developed as a tool to assign patients into one of 4 overall survival (OS) risk groups based on disease type and status at the time of transplantation.³  This index was developed by using a single-institution patient cohort and has since been successfully applied in other studies.^4,5  The primary goals of this study were to examine the validity of the DRI in a multicenter setting and to refine the DRI by using a much larger data set from the Center for International Blood and Marrow Transplant Research (CIBMTR). The refinement was intended to create a conditioning regimen–independent index, to categorize Burkitt lymphoma (BL), which we could not do in the original study, to perform a finer categorization of some diseases (eg, myelodysplastic syndromes [MDSs] and acute lymphoblastic leukemia [ALL]), and to use the increased statistical power to appropriately classify less common disease type-disease status combinations. Ultimately, the aim of the DRI is to provide a robust tool that can be used for prognostication, for the analysis and interpretation of retrospective data, whether conducted in single-center, multicenter, or registry settings or within the context of the federally mandated center outcome reporting. The DRI can also be used for the stratification of patients entering prospective HCT clinical trials.

The CIBMTR is a combined research program of the Medical College of Wisconsin and the National Marrow Donor Program. CIBMTR comprises a voluntary network of more than 450 transplantation centers worldwide that contribute detailed data on consecutive allogeneic and autologous HCTs to a centralized statistical center. Observational studies conducted by the CIBMTR are performed in compliance with all applicable federal regulations pertaining to the protection of human research participants. Protected health information used in the performance of such research is collected and maintained in CIBMTR’s capacity as a public health authority under the Health Insurance Portability and Accountability Act Privacy Rule. Additional details regarding the data source are described elsewhere.⁶ 

The study cohort consisted of patients reported to the CIBMTR who underwent HCT between 2008 and 2010, excluding autologous and syngeneic transplantations. Among the 17 223 patients in this data set, we excluded 2361 patients with missing disease type, disease subtype, or disease status information; patients transplanted for benign or rare disorders (eg, histiocytic disorders, large granular lymphocyte or natural killer cell leukemias); and 1731 pediatric (age <18 years) patients. For the remaining 13 131 patients, the baseline characteristics at HCT, disease information, transplantation type, and OS outcomes were obtained from the CIBMTR database. We categorized cytogenetics for patients with acute myeloid leukemia (AML)⁷  and MDSs⁸  according to previously proposed risk schemes that were specifically derived from HCT patient cohorts. For AML, t(8;21), inv(16), and t(15;17) were classified as favorable, complex karyotype (4 or more abnormalities) as adverse, and all others as intermediate. For MDSs, abnormal chromosome 7 and complex karyotype (≥ 4 abnormalities) were classified as adverse and all others as intermediate. For patients with ALL, cytogenetics were classified into 3 groups: Philadelphia chromosome–positive (Ph+; defined by t(9;22) on karyotypic analysis or BCR-ABL translocation by fluorescence in situ hybridization, where available); t(4;11); and all others. Patients with nodular lymphocyte–predominant Hodgkin lymphoma were grouped with Hodgkin lymphoma (HL); patients with small lymphocytic lymphoma were grouped with chronic lymphocytic leukemia.

Patient baseline characteristics were reported descriptively. The primary end point was OS. No secondary HCT outcome was examined for this study. OS was defined as the time from stem cell infusion to death from any cause. Patients who were alive or lost to follow-up were censored at the time last seen alive. The 1-year follow-up completeness index was 99% and 83% overall. OS was calculated by using the Kaplan-Meier method. The log-rank test was used for comparisons of Kaplan-Meier curves. Potential prognostic factors for OS, including disease risk, age, conditioning intensity, donor and recipient gender, cytomegalovirus (CMV) serostatus, graft source, donor HLA type, comorbidity index, and Karnofsky performance status (KPS) at HCT were examined in the proportional hazards model. The proportional hazards assumption for each variable of interest was tested, and interaction terms were examined. Because conditioning intensity (myeloablative conditioning [MAC] vs reduced-intensity conditioning/nonmyeloablative [RIC/NMA]) did not meet the proportional hazards assumption, the Cox model was stratified by conditioning intensity. All P values were 2-sided at a significance level of .05. To refine the DRI, we randomly split the 13 131 patients into a training cohort comprising three-fourths of the patients (n = 9849) and a testing cohort comprising the remaining one-fourth of the patients (n = 3282). The risk groups were then recreated on the basis of the hazard ratios (HRs) for mortality of each individual disease type-disease status category in a multivariable model, with AML intermediate cytogenetics in first complete remission (CR1) as the reference category. We used thresholds of 2.0, 1.33, and 0.89 to separate the groups (separated by a 50% proportional increase in hazard for death). We then compared the performance of the refined DRI and various other risk scores in the testing cohort by using both the multivariable c-statistic⁹  (rcorr.cens procedure from Hmisc package in R; R Foundation for Statistical Computing, Vienna, Austria), which quantifies the predictive ability of a stratification scheme, as well as the assessment of model fit measured by the difference in the Akaike information criterion (AIC) between multivariable models built with and without the disease risk variable. For this comparison, we excluded patients with BL who were not classified in the original DRI.

Because the very-high-risk group represents only a small percentage of patients, we examined the behavior for various possible sample sizes of the 4-group index and a potential 3-group index, built by combining the very-high-risk and high-risk categories. We performed a simulation study with sample sizes of 100, 200, 300, 400, 500, 750, 1000, 2000, 3000, 4000, and 5000 patients. For each sample size, 100 random samples of that size were drawn from the entire 13 131 patients. We then fit Cox models on each sample by using a 3-group or 4-group DRI and pooled the outcomes from the 100 random samples. All calculations were performed by using SAS 9.3 (SAS Institute, Inc., Cary, NC), and R version 2.13.

The baseline characteristics of the 13 131 patients are shown in Table 1. The cohort included a broad representation of diseases, disease status, donor types, and graft sources; more than 5000 patients received transplantations for lymphoid malignancies, including nearly 2000 for non-Hodgkin lymphoma (NHL). Approximately half the patients were conditioned with an MAC regimen and half with an RIC/NMA regimen. The median follow-up for survivors was 24 months (range, 2 to 50 months).

To validate the original DRI, patients were assigned to a DRI category as previously described³  (Table 2). Because patients with NHL or HL in first partial remission (PR) and subsequent PR were grouped together in this data set, all patients with NHL or HL in PR were categorized as low-risk stage, regardless of conditioning intensity, for this validation. Patients with BL who were not included in the original DRI were excluded from the validation step but were included in the refinement step. The results are shown in Figure 1. The DRI provided a highly significant risk stratification, with 2-year OS of 64% (95% confidence interval [95% CI], 61% to 67%) for patients in the low-risk group (12% of the cohort), 51% (95% CI, 50% to 53%) in the intermediate-risk group (64% of patients), 34% (95% CI, 32% to 36%) in the high-risk group (23% of patients), and 24% (95% CI, 17% to 31%) in the very-high-risk group (1% of patients) (P < .0001 for all pairwise comparisons between groups). We also built a multivariable model for OS by using the DRI categories and other baseline variables. As shown in Table 3, the DRI was associated with highly significant differences in OS, with an HR for mortality of 2.97 (95% CI, 2.44 to 3.62) in the very-high-risk group compared with the low-risk group (P < .0001); in this cohort, the DRI was the variable most strongly associated with survival.

We built 17 distinct multivariable models for patient subgroups of interest: patients who were conditioned with MAC regimens, were conditioned with RIC/NMA regimens, received peripheral blood grafts, received bone marrow grafts, received umbilical cord blood grafts, were younger than age 40 years, were age 40 to 64 years, were age 65 years or older, were transplanted from a matched sibling, were transplanted from a well-matched unrelated donor,¹  were transplanted from a partially matched unrelated donor, were transplanted from a mismatched donor, had a KPS <90% at HCT, had a KPS of 90% to 100%, were CMV seropositive or were transplanted from a seropositive donor, were CMV seronegative or were transplanted from a seronegative donor, and patients with any disease except for AML. In each of these 17 distinct subgroups, the 4 DRI risk groups were significantly different (P < .05), with HRs of 1.3 to 1.7 for intermediate-risk, 1.9 to 2.8 for high-risk, and 2.5 to 4.7 for very-high-risk subgroups, all compared with low-risk DRI.

By using the multivariable models previously mentioned, we examined the impact of the AML and MDS cytogenetics grouping schemes used here on OS. In a model restricted to patients with AML, the HRs for mortality associated with adverse (n = 251) and favorable (n = 224) cytogenetics were 1.5 (95% CI, 1.3 to 1.8) and 0.6 (95% CI, 0.5 to 0.8), respectively, (both P < .0001) compared with intermediate-risk cytogenetics (n = 4838). In a model for patients with MDSs, the HR for adverse (n = 245) compared with intermediate-risk cytogenetics (n = 1294) was 1.2 (95% CI, 1.0 to 1.4; P = .050). We also specifically examined the outcome of patients with t(8;21) AML (n = 95) who were categorized as low-risk in the original HCT cytogenetics study¹⁰  and intermediate-risk in a subsequent validation study.⁷  In this cohort, they were best classified as low risk (HR, 0.6 [95% CI, 0.5 to 0.9] compared with normal karyotype AML; P = .009). Cytogenetics in ALL (as described in “Methods”) were not associated with outcome: HR, 0.9 (95% CI, 0.7 to 1.1) for Ph-positive ALL (n = 184 patients) compared with Ph-negative/non t(4;11) (P = .4). To ensure that the DRI could also be used with non-HCT–specific cytogenetics assignment, we repeated the validation using the cytogenetics risk grouping scheme from the Medical Research Council/National Cancer Research Institute¹¹  for AML and the International Prognostic Scoring System (IPSS)¹²  for MDSs. The HRs for mortality in the multivariable models for the intermediate-risk (1.4; 95% CI, 1.2 to 1.5), high-risk (2.0; 95% CI,1.9 to 2.2), and very-high-risk (2.6; 95% CI, 2.2 to 3.1) groups compared with the low-risk group remained highly significant (all pairwise P < .005), although this decreased the stratification ability of the DRI slightly (c-statistic, 0.631 using the Medical Research Council and IPSS cytogenetics schemes vs 0.637 using the schemes from the original DRI).

Among the 2381 patients with lymphoma, 889 (37%) had undergone a prior autologous hematopoietic cell transplantation (61% of patients with NHL and 39% of those with HL). This did not affect survival in the multivariable model (HR,1.1; 95% CI, 1.0 to 1.2; P = .2), even in models restricted to NHL or to HL. Moreover, there was no significant interaction between DRI assignment and prior autologous hematopoietic cell transplantation. For all subtypes of lymphoma examined, patients with second complete remission (CR2+) had outcomes indistinguishable from those in first CR (CR1), and better than those in any PR. Finally, we re-examined the assignment of risk categories for T-cell NHL and found in this study that only enteropathy-associated T-cell lymphoma and extranodal natural killer cell T-cell lymphoma appeared to be at significantly higher risk than others when adjusted for disease status at HCT; however, because this was based on a small number of patients, all patients with T-cell NHL were grouped together for subsequent analyses.

We used the large size of this cohort to entirely rebuild the DRI in order to confirm the previous assignments, especially for rarer disease type-disease status combinations. For this analysis, we randomly split the cohort into a training set (three-fourths of the patients) and a testing set (the remaining one-fourth of the patients). In the training cohort, we used the multivariable model described earlier but included separate terms for every possible disease type-disease status combination. We did not make any assumptions about disease groupings in the model; however, categories with similar outcomes were regrouped for presentation of the final results. In this case, we did not build separate models for MAC and RIC/NMA patients, as we had for the original DRI.³  We assumed that a DRI built on the entire cohort would be independent of conditioning intensity and not subject to the possible selection bias associated with the choice of conditioning regimen (see “Discussion”). Risk groups were established on the basis of proportional increases in the HR for mortality, centered on AML intermediate cytogenetics in CR1. The results are shown in Table 4, which also shows the original DRI assignment for comparison. We then tested the performance of this refined DRI in the testing cohort compared with the original DRI and with other proposed disease-risk stratification schemes. The multivariable c-statistics (with higher values denoting improved predictive ability) were 0.635 for the scheme used in the Patient Activation Measure score,¹³  0.626 for the stratification scheme used in Blood and Marrow Transplant Clinical Trials Network trials,¹⁴  0.637 for the original DRI, and 0.643 for the refined DRI. Similarly, the reduction in AIC (with a larger reduction implying better model fit) for the refined DRI was 126 compared with 81 for the Patient Activation Measure score, 37 for the Blood and Marrow Transplant Clinical Trials Network scheme, and 86 for the original DRI. The survival of patients in the entire cohort stratified by using the refined DRI is shown in Figure 2A; Figure 2B shows the survival of patients in both the training and testing cohorts stratified by the refined DRI.

In multivariable models using this refined DRI, there were no significant interactions between the risk group assignment and any of the other model variables. The new DRI stratified patients regardless of conditioning intensity (Figure 2C). Nonetheless, we repeated the refinement process in the MAC and RIC/NMA cohorts separately. The resultant risk groups were identical in the 2 conditioning groups, with one exception. For lymphoma, patients in PR who underwent RIC HCT had an outcome similar to those in CR, but patients in PR who underwent MAC HCT had a worse outcome than those in CR. Despite this, and as discussed further, we chose to keep the refined DRI independent of conditioning intensity, as shown in Table 4.

For small size cohorts, the very-high-risk group, which will often comprise only a small proportion of patients, can be merged with the high-risk group, and the DRI can be used as a 3-group breakdown. To approximate what cohort size would work best in a data set like this one, we performed simulations by randomly selecting subcohorts of various sample sizes from our entire cohort (see “Methods”). The results (not shown) indicate that a 3-group breakdown (merging the very-high-risk and high-risk groups) is preferable for a cohort size of 300 or smaller, and a 4-group breakdown is preferable for cohort sizes larger than 300.

To validate and refine the original DRI, we used a cohort of more than 13 000 patients. The original DRI, which was built on training and validation groups of patients completely separate from the one used in this study, performed very well in this cohort (Figure 1). It stratified patients into 4 groups with very different OS and provided the strongest prognostic information in this cohort. It was also very stable across age groups, conditioning intensity, graft source, and donor type. As discussed in the original proposal for the DRI, we envision that this tool should not be fixed but should instead be refined by the transplant community as new information becomes available. Thus, we anticipate that in the future, molecular information (eg, FLT-3 internal tandem duplication status for AML and MYC/BCL-2 status for aggressive B-cell NHL) will be incorporated and will alter the risk assignments. We therefore used the large sample size of this cohort to begin this refinement process. For example, we added BL to the DRI; we were also able to clarify the assignment of some of the patients with rarer diseases in HCT cohorts, such as HL or mantle cell lymphoma (Table 4). We also found a similar survival for ALL patients regardless of Ph status, as preliminarily suggested in other studies.^15,16  Moreover, we show that patients with MDSs are best stratified based on the combination of blast percentage, cytogenetics, and response to prior therapy; this breakdown provided better discrimination than the IPSS¹²  in our cohort (data not shown). Despite these changes, the OS rates in each group of this refined DRI are similar to those in the original DRI. In this large data set, we were also able to refine the DRI and confirm that this refined DRI performed slightly better than the original one in a testing cohort separate from the training cohort. The differences in c-statistics between the schemes compared here are relatively small. This reflects in part the use of a multivariable c-statistic, in which the difference in predictive power of the disease risk variable is further diluted by the predictive power of other variables in the model. Indeed, univariable c-statistic calculations showed the same ranking of predictive ability but with larger differences (data not shown). The differences in model fit as assessed with the AIC yielded the same ranking. Overall, the purpose of those analyses was only to compare different schemes in an independent cohort, and the results suggest that the refined DRI appears to outperform existing schemes, likely because it was derived by using the largest cohort of patients of all 4 schemes with no preexisting assumptions.

The DRI should be collapsed to a 3-group system for smaller studies, with the very-high-risk and the high-risk groups merged. The choice between using the 3- or 4-group index, or collapsing further, should depend on the characteristics of the particular cohort under analysis, considering such factors as duration of follow-up, number of events, and survival distribution; the choice involves a trade-off between getting more accurate estimates of risk with more groups and the additional variability in estimation resulting from using smaller groups. In general, one rule of thumb may be that each category requires at least 10 events, where events are patient deaths if the end point is OS. Even with 10 events in each category, if the HRs are similar between two categories, it would be preferable to merge these two categories to increase power. In our simulation, a 3-group system appeared optimal for cohorts of fewer than ∼300 patients, but we emphasize that the choice of a 3- or 4-group breakdown is best dictated by the characteristics of the cohort under study and that the numbers provided here are to be used only as a general guide. When using the DRI for stratified randomization in a clinical trial, the 3-group version will likely be most useful, especially if other stratification variables are used.

The issue of conditioning intensity is noteworthy. As described previously, and in a remarkable concordance between the original DRI and this study, the risk assignment of patients with lymphoma in PR seemed to differ according to conditioning intensity, with a higher risk (relative to patients in CR) only for patients receiving MAC. This runs somewhat contrary to the common assumption that patients with more chemotherapy-refractory disease benefit from higher conditioning intensity. There are at least 2 possible explanations for this finding. One is that it reflects selection bias, because patients with more aggressive disease, as assessed clinically but not in a way that can be captured with the variables available to us, may be preferentially treated with MAC HCT. The other is that, in fact, such patients, because they are more chemotherapy refractory, may not benefit from the additional cytotoxicity of a MAC regimen and may suffer from the additional toxicity of those regimens without a commensurate benefit in disease control. Unfortunately, the nature of our studies does not allow us to distinguish those possibilities, which must be left to further prospective trials. For this reason, and also to keep the DRI separate from the choice of transplantation method, we chose to keep the assignments independent of conditioning intensity, unlike in our original study. More broadly, we emphasize that this study was not meant to compare the outcomes of MAC and RIC hematopoietic stem cell transplantation, or to suggest an optimal conditioning regimen for any group of patients. The strong likelihood of selection bias in the choice of conditioning regimen would preclude this type of analysis in a retrospective study like ours. In this respect, the salient conclusion of this study is therefore not that conditioning intensity does not have an impact on hematopoietic stem cell transplantation outcome, but that the DRI applies to patients regardless of conditioning intensity.

The derivation of the DRI was based solely on OS and did not directly take into consideration relapse risk. Relapse and time to relapse are more difficult to reliably ascertain in multicenter studies, given the different possible definitions of relapse (eg, molecular, cytogenetic, hematologic, radiologic, clinical). Obtaining such data without a detailed analysis of relapse time and ascertainment method for each patient would be unreliable, and obtaining it with the appropriate quality control for a cohort this size would be prohibitively time-consuming. Moreover, not only the risk but also the tempo of relapse may be quite different between different diseases. It is therefore questionable whether the type of analysis performed for this study would be appropriate in a cross-disease analysis. Finally, OS is the least subjective HCT outcome and is usually very closely associated with progression-free survival anyway, reflecting the poor outcomes of patients who relapse after HCT.⁵  For those reasons, we chose to focus exclusively on OS.

It is also conceivable that longer follow-up could alter the assignments but, again, this is unlikely in an HCT cohort because disease-related variables meet the proportional hazards assumption so well in this cohort and in general in HCT studies. The chosen cohort allowed us to validate and refine the DRI for patients transplanted in the most recent timeframe that allows collection of 2-year survival outcome, which is particularly useful because year of transplantation has been clearly associated with HCT outcome.¹⁷ 

We propose this refined DRI as a system for risk-stratifying heterogeneous populations of patients undergoing HCT, based on OS. This index applies regardless of age, conditioning regimen, donor type, and graft source. We anticipate that the DRI will be used by clinicians for prognostication and by researchers to improve the quality of retrospective studies and prospective clinical trials that enroll patients across disease type and disease status categories. We also hope that this work encourages the performance of HCT trials that are not disease specific, at least when the success of the investigational treatment is not likely to be disease specific, by removing the concerns over outcome variability introduced by disease type and disease status heterogeneity.

This work was supported by an American Society of Clinical Oncology Career Development Award (P.A.), by grant U19 AI 29530 from the National Institute of Allergy and Infectious Diseases (NIAID), and grant PO1 HL 070149 from the National Heart, Lung and Blood Institute (NHLBI). The Center for International Blood and Marrow Transplant Research is supported by Public Health Service grant/cooperative Agreement U24-CA76518 from the National Cancer Institute (NCI), the NHLBI, and the NIAID, by grant/cooperative agreement 5U01HL069294 from NHLBI and NCI, by contract HHSH234200637015C with Health Resources and Services Administration/Department of Health and Human Services), by grants N00014-06-1-0704 and N00014-08-1-0058 from the Office of Naval Research, and by grants from Allos, Inc.; Amgen, Inc.; Angioblast; an anonymous donation to the Medical College of Wisconsin; Ariad; Be the Match Foundation; Blue Cross and Blue Shield Association; Buchanan Family Foundation; CaridianBCT; Celgene Corporation; CellGenix, GmbH; Children’s Leukemia Research Association; Fresenius-Biotech North America, Inc.; Gamida Cell Teva Joint Venture Ltd.; Genentech, Inc.; Genzyme Corporation; GlaxoSmithKline; Kiadis Pharma; The Leukemia & Lymphoma Society; The Medical College of Wisconsin; Millennium Pharmaceuticals, Inc.; Milliman USA, Inc.; Miltenyi Biotec, Inc.; National Marrow Donor Program; Optum Healthcare Solutions, Inc.; Otsuka America Pharmaceutical, Inc.; Seattle Genetics; Sigma-Tau Pharmaceuticals; Soligenix, Inc.; Swedish Orphan Biovitrum; Therakos, Inc.; and Wellpoint, Inc.

Contribution: P.A. designed the research, analyzed the data, and wrote the paper; H.T.K. designed the research, analyzed the data, and edited the paper; B.R.L. designed the research, collected and analyzed the data, and edited the paper; Z.W. collected and analyzed the data and edited the paper; E.P.A., M.E.K., and R.T.M. edited the paper and served as cochairs of the Center for International Blood and Marrow Transplant Research Chronic Leukemia Working Committee; J.H.A., R.J.S., D.J.W., J.D.R., and M.M.H. designed the research and edited the paper; and W.S. designed the research, analyzed the data, and edited the paper.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Philippe Armand, Dana-Farber Cancer Institute, 450 Brookline Ave, Boston MA 02215; e-mail: parmand@partners.org.