Multi-Institutional Validation of the Predictive Power of the Hematopoietic Cell Transplantation Comorbidity Index (HCT-CI) for HCT Outcomes

Abstract 145

In 2005, the HCT-CI was introduced as a weighted scoring system to predict mortality risk following allogeneic HCT. Since then, not all investigators were able to validate the HCT-CI after testing in their respective institutions. In 2007, a collaborative multi-institutional study was initiated to investigate 1) whether the HCT-CI was predictive of outcomes across different institutions, 2) the degree of homogeneity of outcome prediction, and 3) the reasons for lack of agreement among investigators. To this end, data were collected from 3347 consecutive patients (pts) treated with allogeneic HCT between 2000 and 2006 from HLA-matched related or unrelated donors at 5 institutions. All data were collected by a single investigator, blinded from the final outcomes of pts, to ensure consistent comorbidity coding. Numbers of pts, percentages of available comorbidity data, and other transplant and pt characteristics were statistically significantly different among institutions (Table 1). Pts missing comorbidity or other covariate data were excluded from further analyses, yielding a final sample size of 2523.

Overall, pts with HCT-CI scores of 0 vs. 1–2 vs. ≥3 had 2-year non-relapse mortality (NRM) rates of 14%, 23%, and 39% (p <0.0001), respectively, and 2-year overall survival (OS) rates of 74%, 61%, and 39% (p <0.0001), respectively. Proportional hazards models were used to estimate the hazard ratio (HR) for NRM and OS associated with HCT-CI scores in each of the 5 institutions (Table 2). The models were adjusted for covariates in Table 1. Increased HCT-CI scores were associated with increases in the HR for NRM and OS across all 5 institutions and these increases were highly statistically significant except for institution E, which had the smallest sample size. Of note, the magnitudes of increases in HRs were not entirely comparable across institutions. In a unified model including all institutions, we found a statistically significant lack of homogeneity across institutions for the HRs associated with scores 1–2 (p=0.03) and ≥3 (p=0.04) for NRM and with scores ≥3 (p=0.01) for OS but not with scores 1–2 for OS (p=0.18). We also found a statistically significant, independent impact of institution on NRM (p=0.001) and OS (p<0.001).

We then assessed, among 80 pts from institution A, the inter-observer variability in scoring comorbidity between two individual investigators and between each of them and unknown individuals from a pool of other evaluators. Weighted kappa statistics were highest (0.59) between two single evaluators and lowest between each and multiple evaluators (0.43 and 0.55, respectively). The principal investigator then developed a comprehensive guideline to code comorbidities and used it to train the other single investigator in a single session. Additional evaluation of inter-observer agreement demonstrated marked improvement of the weighted kappa statistic to 0.78. The reported disagreements on the validity of the HCT-CI may be explained by different institutional experiences in managing transplant pts, small number of pts at some institutions, and inter-observer variability in score assignment.

The HCT-CI is valid to discriminate relative risks of mortalities after HCT across different institutions and should be used regularly for counseling pts and clinical trial design. Efforts to improve methods for coding comorbidity are in progress.