Identification of poor prognostic features among patients requiring mechanical ventilation after hematopoietic stem cell transplantation

Autologous and allogeneic hematopoietic stem cell transplantations (HSCTs) are frequently performed procedures with a variety of indications. Unfortunately, 15% to 30% of HSCT patients develop respiratory failure that requires mechanical ventilation (MV). Once this complication occurs, death ensues in 85% to 90% of the cases.1-5 Because HSCT is often performed on young patients with curative intent, advance directives are frequently not available when respiratory failure develops. In these circumstances, patients and family members agonize over whether to withhold or withdraw life-sustaining measures, and physicians are often asked to guide these difficult decisions.

A number of investigators have supplemented this decision-making process by identifying prognostic features that are associated with a very high likelihood of death. The benefits of these investigations are apparent for both providers and families: the knowledge that death is inevitable, or even highly likely, makes the irrevocable decision to withdraw support easier.6,7 A few researchers have also extended this argument to suggest that the decision to withdraw care can be made unilaterally in patients with certain of these prognostic features.8-10 

At present, none of the features that have been reported to be associated with a poor prognosis have been prospectively validated. There are reasons why these prognostic features may not be as strongly predictive of mortality when applied to a naı̈ve cohort as they are in the study cohorts from whence they were originally derived—that is, they may not be transportable to other settings.11-15First, patients who undergo HSCT are heterogenous, whereas patients in a particular study may not be typical of all HSCT patients—a problem referred to as lack of generalizability. Second, in analyzing cohorts, investigators sometimes examine multiple possible predictors and then report only the few that are statistically associated with a particular outcome (ie, death). As more predictors are considered, the possibility that chance associations rather than true associations are uncovered (and reported) rises—a problem arising from lack of validation.16 Lastly, new technologies and treatments may alter outcomes.

In this paper, we adopt a Bayesian approach to answering the 2 questions that form the cornerstones of this discussion.17First, can we identify a subset of patients who are virtually certain to die based on the presence of certain poor prognostic features? Second, what is the duration of survival of patients who do recover from respiratory failure? In other words, if we were to withdraw support from someone who would have survived if given maximal treatment, how egregious would be our error?

To answer the first question, we systematically reviewed all studies that assess the outcomes of patients who require MV after HSCT, and abstracted the reported survival statistics and poor prognostic features. We then developed a measure of the predictive power of each prognostic feature using Bayesian principles, which are based on the observation that diagnostic tests are imperfect: not all patients who die have poor prognostic features (there are false- negative results), and some patients who live have poor prognostic features (there are false-positive results). According to Bayes theorem, the probability of death in the presence of a particular poor prognostic feature depends both on the test characteristics of the prognostic feature and the probability that any patient requiring MV after HSCT will die. To estimate the chance of long-term survival, we summarized results from published reports and assessed the survival of patients in our “validation cohort” of 226 patients treated at 5 centers.

We searched MEDLINE, EMBASE, and HealthStar using the MeSH browser and “exploded” terms “bone marrow transplantation” and “ventilators, mechanical.” Two investigators (P.B.B., J.S.G.), working independently, determined eligibility of the articles retrieved. The searches included English-language articles published between January 1, 1966, and August 1, 2000. Studies were included if they enrolled patients who, for the treatment of any malignancy, received a hematopoietic stem cell transplant (allogeneic or autologous) and subsequently required endotracheal intubation and MV. We excluded letters, case reports, case series (ie, fewer than 10 patients studied), studies that only addressed more general intensive care unit (ICU) populations, review articles, editorials, studies that only addressed specific clinical syndromes or treatments, and studies that focused on children.

From the retrieved studies, we abstracted demographic data, treatment details, outcomes, and reported poor prognostic features for patients who required MV after HSCT for any indication other than postsurgical care. In several studies, demographic data were only available aggregated across larger cohorts, such as all observed HSCT patients or all ICU patients, and so we report those data. Outcomes and prognostic feature data are only reported if presented specifically for patients requiring MV after HSCT. We report short-term survival as defined within each study, and long-term survival as survival to 6 months after achievement of this endpoint.

Within each report, we sought poor prognostic features that had been assessed for their association with short-term mortality. We specifically sought analyses of categorical predictors (alone, or in combination) that had been reported in a manner enabling calculation of standard descriptive statistics, such as false-positive rate.18 We then identified prognostic features that warranted further evaluation based on 3 criteria. First, the feature had to be statistically associated with death in the original report (ie, 2-sided; P < .05). Second, the feature had to be relevant to at least 25% of patients. Third, the feature had to be objectively determined, and defined in a manner that would allow for the feature to be assessed in other settings.

We report the following attributes determined from the original studies: the proportion of patients who had the prognostic feature (theprevalence), and the proportion of patients with the prognostic feature who died (positive predictive value). Some investigators presented only percentages, some presented only graphical data, and one presented data in the form of a nested case-control study. In these cases, we determined the relevant attributes using appropriate techniques. In addition, in some reports individual patients appeared multiple times within a cohort—thus, it was not always possible to disaggregate these entries, and so they are analyzed as presented, with some patients contributing more than one outcome. We also report, when available, the number of prognostic features assessed within each study so that readers can determine the likelihood that associations were uncovered by chance.

The data collection procedure and study population have been previously described.19 In brief, data were obtained on all patients who required intensive care within one year of receiving an infusion of hematopoietic progenitor cells harvested either from bone marrow or from peripheral blood after receiving myeloablative treatment (chemotherapy and/or radiation). These patients received care at one of 5 tertiary care hospitals during the time period July 1, 1994, to June 23, 1997. The study was approved by the institutional review boards of all participating sites: Memorial Sloan-Kettering Cancer Center, New York, NY; City of Hope National Medical Center, Duarte, CA; The University of Texas M. D. Anderson Cancer Center, Houston, TX; Mount Sinai Medical Center, New York, NY; and The Johns Hopkins Oncology Center, Baltimore, MD.

Demographic, clinical, laboratory, and physiologic variables were obtained within one hour, at 24 hours, and at 72 hours after ICU admission. Vital status at discharge from both the ICU and the hospital were recorded. Events after hospital discharge were ascertained from medical records at each institution.

The probability of death in the presence of each of the poor prognostic features (the posttest probability), is determined through combining the overall probability of death (theprior probability) and the incremental probability of death conferred by having the poor prognostic feature (the likelihood ratio). The likelihood ratio equals the true-positive ratedivided by the false positive rate. In this study, we determined the posttest probability by combining the pretestmortality probabilities from the systematic review with the likelihood ratio for each prognostic feature as determined from the validation cohort. Although Bayes formula requires transforming the probabilities into odds (as shown below), we present all of our results as probabilities for clarity.20 

In estimating likelihood ratios using the validation cohort,only the initial admission requiring MV for each patient was analyzed. In most cases, the reported prognostic features could be tested using the exact criteria suggested by the authors. We constructed definitions that closely matched those that could not be tested directly: the features described by Rubenfeld et al3and Denardo et al,21 and the Acute Physiology and Chronic Health Evaluation II score (APACHE II)22 (Table1). We assessed all other plausible definitions in these cases as well, and for none did we see substantively different results. The poor prognostic features presented by Price et al were derived on a subset of patients at one of the centers also involved in our multicenter study (The University of Texas M. D. Anderson Cancer Center, Houston, TX).23 These features were assessed only on patients from the other 4 centers. If patients had insufficient data to assess a predictor, they were eliminated from the analysis. In estimating the pretest probability of death, we used a range rather than a point estimate in order to obtain a generalizable result. The range of prior probabilities used in this article reflects the span of reported mortality rates from all retrieved studies with 2 caveats: we excluded the most extreme result at each end of the distribution, and we collapsed the results of Price et al's study into the mortality estimate from the validation cohort for the reasons cited above.23 

Date of last follow-up in the medical record was chosen as the censoring date for the validation cohort. Follow-up was complete on all patients for a minimum of 200 days. Therefore, the 6-month survival estimate is exact; the median survival estimate uses the Kaplan-Meier method.24 Prognostic features reported as associated with mortality in the original studies were assessed using the Fisher exact statistic.25 Likelihood ratios and prior probabilities are combined using the odds-likelihood form of Bayes formula as described above.20 All P values quoted are 2-sided, and a level less than .05 is considered significant. All analyses were performed using Stata 6.0 (Statacorp, College Station, TX).

The initial search of the literature yielded 61 studies, and a search of their reference lists yielded 2 additional articles. We evaluated these studies and eliminated letters,2 case reports and case series,10 studies that only addressed more general ICU populations,6 review articles or editorials,7 studies that only addressed specific clinical syndromes or treatments,14 and studies that focused on children,9 leaving 15 studies. The characteristics of patients included in these studies are listed in Table2. Across these studies there is considerable heterogeneity in age and sex distribution, in the type of HSCT studied, and in the time interval between HSCT and the requirement for MV. The majority of the studies focus on patients undergoing HSCT for a hematologic malignancy or disorder, although many include small numbers of patients undergoing HSCT as a component of treatment for a solid tumor. Not shown in the table are that 26 (54%) of the patients in Price et al's study, and 8 (47%) of the patients in Shorr et al's study received stem cells harvested from peripheral blood, rather than from bone marrow.23 33 

The 226 patients in the validation cohort drawn from 5 medical centers are described in Table 3. These patients have a somewhat higher mean age than the patients in the prior studies. The cohort contains both patients who underwent allogeneic (71%) and autologous (29%) HSCT. In the vast majority of the cases (88%), the indication for HSCT was for a hematologic malignancy or disorder, similar to that seen in the prior studies.

Fifteen studies reported (and we calculated for the validation cohort) short-term survival of patients requiring MV after HSCT. Survival rates ranged from 40% to 2% (Table4). Based on these results, we discarded one value at each end of the extreme, and incorporated the estimate from Price et al's23 subset of the validation cohort into the validation set, yielding an estimate of the pretest probability of death ranging from 82% to 96%. This range includes the point estimates from 12 of the 15 studies, the mortality rate of 86% that was observed in the validation cohort, and the mortality rate reported by Staudinger et al.26 

Investigators reported a variety of features as predictive of a poor prognosis. The features that were reported with sufficient detail to calculate test characteristics are shown in Table5. Investigators reported 3 clinical variables as strongly predictive of mortality: requiring mechanical ventilation for 4 days or longer,21 requiring mechanical ventilation more than 30 days after HSCT,27 and requiring mechanical ventilation less than 90 days after HSCT. This latter predictor was reported in 2 studies, one with calculable statistics8 and one without.28 Two studies reported that age group was predictive of mortality—aged 21 or older in one study,2 and aged 40 or older in another study.8 Age was not predictive of mortality in the studies of Denardo et al,21 Afessa et al,29 Paz et al,30 or Huaringa et al.27 Examining a subset of patients from the validation cohort, Price et al reported that receipt of stem cells of bone marrow origin rather than peripheral origin was associated with a poor prognosis, as was having a high respiratory rate at the time of admission.23 

Five studies focused on physiologic measures. Jackson et al31 and Rubenfeld and Crawford3 both reported that combined measures of hepatic, renal, and circulatory dysfunction were predictive of death. Jackson et al31 presented only a multivariable analysis, whereas Rubenfeld and Crawford3presented data sufficient for calculation (Table 5). Paz et al30 and Shorr et al33 using APACHE II, and Epner et al32 using APACHE III, reported an association between these measures of physiologic derangement and mortality. Shorr reported that patients with an APACHE II score ≥ 29 had a 100% mortality. The Apache II score was not predictive of mortality in the study by Huaringa et al.27 

Using the likelihood ratios as determined on the validation cohort (Table 6), we found that the presence of simultaneous hepatic dysfunction (bilirubin > 68 μmol/L or 4 mg/dL) and renal dysfunction (serum creatinine > 177 μmol/L or 2 mg/dL) as defined by Rubenfeld and Crawford3 was associated with the highest posttest probability of death. In our validation cohort, all 51 patients with this prognostic feature died, yielding an infinite likelihood ratio and an incalculable posttest probability. Given this limitation, we assessed this characteristic as if it were a little less predictive—that is, that one of the patients with this characteristic had actually survived. This approach led to a likelihood ratio of 9.48, and a posttest probability of death ranging from 98% (when the baseline mortality rate was 82%) to 100% (when the baseline mortality rate was 96%).

We reanalyzed this result after incorporating the information that was lost from the 19 patients (8% of validation cohort) who had missing data, and who were therefore eliminated from the primary analysis. All 19 of these patients died, ruling out the possibility that these patients may have increased the false positive rate. Considering the most extreme alternative—that all of these 19 patients represented false negatives (ie, they died without having the poor prognostic feature)—the estimated likelihood ratio fell only to 8.55, marginally lowering the range of posttest probabilities from the current estimate of 97.7% to 99.6%, to a range of 97.5% to 99.5% (data not shown).

Some of the other reported poor prognostic features are also associated with an increased probability of death, such as requiring MV for more than 4 days—a feature that confers a posttest probability of death of 88% to 97% (Table 6). Features such as age, time from HSCT to MV, source of stem cells for transplant, and other measures of physiologic derangement are not reliably predictive of death.

Seven studies reported the 6-month survival rates for the minority of patients who recovered from respiratory failure, with results ranging from 27% to 88% (Table 4). Of the 32 patients in the validation cohort who survived to discharge from the hospital, 20 (63%) were alive at 6 months (9% of entire cohort). Median survival for those who recovered was reported by Rubenfeld and colleagues3 as 634 days (21 months), and median survival for recovering patients in the validation cohort was 451 days (15 months). One study we found reported that 6 of 7 patients who recovered and survived more than one year rated their quality of life as “good.”34 

The vast majority of patients who develop respiratory failure after hematopoietic stem cell transplantation do not survive. In this study, we asked 2 questions in order to determine when it might be appropriate for physicians to recommend withdrawing life-sustaining care from these patients. First, based on available clinical information, are there patients for whom we can know that death is virtually certain, thus allowing us to use clinical features to guide end-of-life decisions? Second, what is the survival rate of those patients who recover from respiratory failure? We found that the presence of simultaneous elevation of bilirubin (> 68 μmol/L or 4 mg/dL) and serum creatinine (> 177 μmol/L or 2 mg/dL) is highly predictive of death in patients requiring MV after HSCT. After assuming a false positive rate for this characteristic, we estimated that the probability of death in the presence of this characteristic is 98% to 100%. Supporting this finding is the observation that approximately 383 (98%) of the 391 patients with this characteristic in Rubenfeld and Crawford's original study died.3 We did not find that any of the other 10 reported prognostic features were as reliably associated with death.

In order to make a recommendation based on this finding, we considered the possible consequences of withdrawing support from a patient who would have survived otherwise. If the consequences of premature withdrawal of MV are minimal because prognosis is poor regardless, clinicians and family members can agonize less about these decisions. On the other hand, if those patients who do recover from respiratory failure have reasonable long-term survival, the recommendation to withdraw MV should only be made at a very high threshold of certainty about death. We found that for those subjects who do recover, survival is considerable, and quality of life may be good. This finding underscores the importance of our first aim, identifying prognostic features that are highly predictive of death.

There are several limitations to our assessment of the predictive power of these prognostic features. First, we did not collect data in precisely the same manner as the authors of some other studies, limiting our ability to evaluate their findings. Second, we cannot determine whether the limited predictive ability of some of the previously published prognostic features is due to differences between cohorts (ie, heterogeneity), inaccurate estimates of predictive power due to multiple analyses, or changes in critical care technologies. Third, we constrained ourselves to the cutpoints of continuous variables presented by other authors. In some cases, the same variables when dichotomized differently might have been more strongly predictive of mortality. Lastly, the predictive power of the features that we assessed may change with advances in conditioning regimens, stem cell preparation, prophylaxis against infection, and supportive care technologies.

There are also some limitations to our assessment of long-term survival. Although we report that survival after recovery is substantial, due to sample size limitations we are unable to reliably estimate the long-term survival of patients with the poor prognostic features, leaving open the possibility that these patients experienced substantially shorter survival after recovery than did patients without these features. In addition, we cannot determine the quality of life experienced by those surviving, nor do we have data on what proportion of surviving patients were free of relapse.

Several authors have advocated using certain poor prognostic features to identify those patients for whom the recommendation to withdraw or withhold support should be made. Others, after evaluating the outcomes of these patients, have suggested that the available evidence may support unilaterally limiting or withdrawing life-sustaining care in certain identifiable subsets of patients.3 6-9 The reasoning behind these recommendations is understandable: ICU beds are typically scarce, lengths of stay for this condition are protracted and costly, and prolonged critical illness is emotionally exhausting for patients and their family members.

Due to the irrevocable nature of the decision to withdraw support, and the considerable survival we demonstrated for those patients who recovered, we conclude that most of the prognostic features that we review here are not predictive enough to support their use in end-of-life decision-making. The criterion of combined hepatic and renal dysfunction proposed by Rubenfeld and colleagues3 is the notable exception, and appears to identify patients with a very high probability of death—98% to 100% based on our analyses. Although the decision to withdraw life support must take into account the values of individual patients and their families, the degree of predictive power associated with this prognostic feature may justify its inclusion in the decision-making process.

We are indebted to Alvin I. Mushlin, MD, and Ennapadam S. Venkatraman, PhD, for their recommendations and assistance with the analyses, and to the research team led by Kristen J. Price, MD, and Peter F. Thall, PhD, at the University of Texas, M. D. Anderson Cancer Center, for their contribution of subjects and their review of earlier versions of this manuscript.