https://orcid.org/0000-0001-5571-2775
TO THE EDITOR:
Allogeneic hematopoietic cell transplantation (HCT) is a potentially curative therapy for blood disorders. However, the efficacy of this procedure has been impeded by early endothelial dysfunction, which can lead to a severe and potentially lethal complication called veno-occlusive disease (VOD)/sinusoidal obstructive syndrome (SOS). Despite less aggressive conditioning regimens leading to a decrease in incidence in recent years, VOD/SOS that evolves into multiorgan dysfunction syndrome has a high mortality rate.1 As highlighted by the Pediatric Acute Lung Injury and Sepsis Investigators and Pediatric Blood and Marrow Transplantation Consortium Joint Working Committee consensus, there is high variability in pediatric management of VOD/SOS, which may contribute to the increased morbidity and mortality.2
Defibrotide received Food and Drug Administration approval for VOD/SOS treatment,3-5 and its safety profile is excellent.6 In a randomized trial, its prophylactic administration in a pediatric population resulted in decreased VOD/SOS incidence from 20% to 12%.7 This was not confirmed in a trial including adults and children.8 The observed difference could be due to different risk profiles between 2 populations indicating the need for developing individualized care for the management of VOD/SOS.
In a prospective cohort of 80 pediatric HCT, a model based on 3 biomarkers (L-ficolin, hyaluronic acid, and stimulation 2) accurately stratified these patients in high (>30%) vs low (<5%) risk groups for developing VOD/SOS.9
However, improved risk stratification for VOD/SOS does not necessarily help guide the administration of preemptive therapy. Fast-and-frugal (FFT) decision tree–derived generalized decision curve analysis (gDCA) can help tailor therapy toward individual risk characteristics.10 FFT represents a simple but highly effective problem-solving and decision-making strategy composed of sequentially ordered cues (tests) and binary (yes/no) decisions formulated via a series of if-then statements.11-16 If the condition is met, the decision can be made, and the FFT is exited. If the condition is not met, the FFT considers other cues, 1 after another, until the exit condition of the cue is met. Note that it is the very exit structure of FFTs that determines the ratio between false negatives and false positives.
We used the data set from the prospective evaluation of the VOD/SOS predictive model that accrued 80 pediatric HCT patients (ClinicalTrials.gov identifier: NCT03132337).9 The study was approved by the institutional review boards of all institutional participating centers. Informed consent was obtained from all patients or their legal guardians. First, we have reformulated the previously published Cox-based model9 as an FFT model. In our case, “cues” consisted of 3 biomarkers laboratory values. The order was defined based on the optimal algorithm previously defined.9 We applied Youden index to determine the optimal cutoff point for each biomarker. Briefly, every cue in an FFT can correctly or incorrectly classify the signal and noise. The exit structure (and order of cues) of the FFTs determines its overall classification accuracy. FFTyes/yes has a high hit rate (sensitivity) and the expense of a large rate of false positives. FFTyes/yes maximizes the avoidance of false negatives. FFTno/no has a low rate of false positives, at the expense of a large rate of false negatives. FFTno/no maximizes the avoidance of false positives. FFTyes/no and FFTno/yes have intermediate sensitivities, specificities, and predicative classification accuracy (supplemental Figure 1).15 In addition to standard versions of FFT, which aim to classify a condition of interest (ie, whether the patient has VOD/SOS or not), we also applied a version of FFT with the threshold which did show the same data with both methods (supplemental Table 1).15 FFT with the threshold incorporates benefits and harms of treatment at each exit of the tree to indicate treatment if the probability of VOD/SOS is greater than the calculated threshold probability of VOD/SOS below which treatment should not be offered.14 supplemental Figure 2 shows the discrimination and calibration properties of the original Cox-based VOD/SOS model9 after performing bootstrap (n = 50) internal validation. Harrell C was consistently between 0.75 and 0.77, indicating acceptable discrimination on par with most predictive models published in the medical literature. Likewise, calibration of the model was also acceptable, with the intercept and slope not statistically significantly different from 0 and 1, respectively. We then compared our recently published categorical model9, in which the 3 biomarkers were classified as high and low (coded as 1/0) and incorporated into Cox proportional hazards regression to obtain the β estimates for each biomarker. Then, a score was defined as β × x for each patient, where x is the covariate. Finally, we created 2 biomarker groups: a 3-biomarker positive score and a 3-biomarker negative score, which were formed according to a score >0 and equal to 0, respectively. When defined like this, the Cox-derived model exhibited the feature of an FFT decision tree, a feature of the model not originally recognized (supplemental Table 2; supplemental Figure 3).
The next step was to test the clinical utility of the VOD/SOS model by integrating (benefits and harms) of preemptive treatment with defibrotide. We applied gDCA to evaluate several treatment options to help tailor therapy toward individual risk characteristics in which 3 strategies were compared with calculate Net Benefit (NB): (1) do not provide treatment to any patient, (2) administer prophylaxis to all patients, and (3) use the FFT-driven VOD/SOS model to guide defibrotide preemptive treatment. The strategy with the highest NB represents the best management strategy, regardless of statistical significance. gDCA is an extension of the threshold model according to which we act if the model predicted probability is above the treatment threshold (T) calculated as T = AE/RRR, where AE refers to the harm (expressed as absolute adverse events between 2 treatments) and RRR defines relative risk reduction in patients receiving defibrotide compared with placebo.17 The threshold model helps tailor treatment toward individual risks, as patients with a probability of VOD/SOS greater than T should be given preemptive treatment, otherwise not. We performed gDCA analysis using data from a pediatric randomized trial comparing defibrotide and placebo7 with 2.5% of AE and 40% of RRR. Thus, we calculated the treatment threshold as follows: T = 2.5%/40% = 6.25%. The analysis further assumed that the patients did not receive treatment before experiencing a VOD/SOS event. Figure 1 shows the NB for the RRR ranges. For a very large RRR (>65%), offering prophylaxis to all patients represents the best management strategy. For RRR <2.5%, no prophylaxis should be offered. For RRR values between 2.5% and 65%, the VOD/SOS model is the best strategy and almost perfectly individualizes the use of preemptive defibrotide. Based on the calculated threshold (T = 6.25%), we should offer preemptive treatment with defibrotide to patients who according to the Cox or FFT model have a predictive probability of VOD/SOS >6.25%. Figure 2 shows that this is the case at each “yes” exit in the FFT tree. Note that P(D+|T+) indicates a positive predictive value or postprobability of disease (VOD/SOS), which is always higher than T = 6.25%, and P(D+|T−) refers to the false omission rate (1-negative predictive value).
Representative curves for the NB on the y-axis vs RRR on the x-axis for 5 possible management strategies. The NBs of each management strategy are plotted against the efficacy of prophylaxis with defibrotide, expressed as RRR. Three strategies are evaluated: treat none, treat all, and use the Cox/FFT model to guide the administration of defibrotide. The adverse events associated with defibrotide were assumed to be 2.5%, whereas the efficacy varied from 5% (pessimistic estimates) to 65% (optimistic estimate), with the best estimate at 40%. The best strategy is the one with the highest NBs. Under the assumptions of 5% < RRR < 65%, the use of the Cox/FFT model to guide the administration of defibrotide was superior to the other 2 strategies (treat none and treat all with defibrotide). Defib-optim, defibrotide-optimistic; Defib-pesim, defibrotide-pessimistic; H, harms (ie, adverse effects of treatment).
Representative curves for the NB on the y-axis vs RRR on the x-axis for 5 possible management strategies. The NBs of each management strategy are plotted against the efficacy of prophylaxis with defibrotide, expressed as RRR. Three strategies are evaluated: treat none, treat all, and use the Cox/FFT model to guide the administration of defibrotide. The adverse events associated with defibrotide were assumed to be 2.5%, whereas the efficacy varied from 5% (pessimistic estimates) to 65% (optimistic estimate), with the best estimate at 40%. The best strategy is the one with the highest NBs. Under the assumptions of 5% < RRR < 65%, the use of the Cox/FFT model to guide the administration of defibrotide was superior to the other 2 strategies (treat none and treat all with defibrotide). Defib-optim, defibrotide-optimistic; Defib-pesim, defibrotide-pessimistic; H, harms (ie, adverse effects of treatment).
VOD/SOS biomarker–based model stratification assuming baseline treatment values at AE = 2.5% and RRR = 40%. Preemptive treatment with defibrotide should be offered to patients who, according to the Cox or FFT model, have a predictive probability of SOS >6.25%. This occurs for each marker exceeding its cutoff∗ that is considered “positive” (see text for details). ∗The cutoffs for the markers are determined as 1100 ng/mL for L-ficolin, 200 ng/mL for hyaluronic acid, and 45 ng/mL for stimulation 2.
VOD/SOS biomarker–based model stratification assuming baseline treatment values at AE = 2.5% and RRR = 40%. Preemptive treatment with defibrotide should be offered to patients who, according to the Cox or FFT model, have a predictive probability of SOS >6.25%. This occurs for each marker exceeding its cutoff∗ that is considered “positive” (see text for details). ∗The cutoffs for the markers are determined as 1100 ng/mL for L-ficolin, 200 ng/mL for hyaluronic acid, and 45 ng/mL for stimulation 2.
To our knowledge, this is the only prospective pediatric study conducted to assess VOD/SOS risk biomarkers that were previously identified with a discovery proteomic study.18 The published categorical Cox (yes/no) and FFT models have good discrimination and calibration performance and were similar even though they were developed using different theoretical assumptions. Using 3 biomarkers according to FFT and categorical VOD/SOS (yes/no) models represents the optimal management strategy for VOD/SOS prophylaxis with defibrotide as compared with prophylaxis for all or no defibrotide at all. Because defibrotide prophylaxis in all pediatric HCT recipients has been reported to not be cost-effective regarding treatment-related mortality or length of stay,19 FFT modeling would treat only a small proportion of high-risk patients, potentially rendering the treatment cost-effective.
This study has its limitations. Only 10 patients developed VOD/SOS, resulting in a 12.5% incidence which is <20% reported in the placebo group from the defibrotide prophylaxis study.7 The sample size of 80 was calculated at the initiation of the study in 2017 based on the placebo group in the pediatric prophylactic study.7 However, VOD/SOS incidence has since decreased, although not as much as in adults.20,21 The cases of pediatric VOD/SOS that evolve into multiorgan dysfunction syndrome continue to have a high mortality rate and be a major pediatric clinical problem after HCT.22-24 Although the FFT model is promising and established in a prospective cohort, by virtue of this study, its validity will require further demonstration in other settings, and particularly in implementation, especially if a preemptive trial significantly decreases VOD/SOS incidence.
We conclude that the VOD/SOS biomarker–based FFT model offers a potential method to guide risk-adapted preemptive therapy for VOD/SOS.
Acknowledgments: The authors thank the clinicians at all institutions that helped accrue samples, all the data managers for excellent management of the database, biobank, and institutional review board, and all members of the S.P. laboratory for help in generating biomarker values.
This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development and National Cancer Institute (grants P50HD090215 [Jamie Rembarger, Project 2: S.P.], R01CA168814 [S.P.], and R01HD074587 [S.P.]).
Contribution: S.P. designed and oversaw the prospective biomarker study, analyzed data, and ensured data integrity; I.H. and B.D. generated fast-and-fungal and generalized decision curve analyses; and all authors wrote, reviewed, provided feedback on the interpreted results, and edited and approved the final version of the manuscript.
Conflict-of-interest disclosure: S.P. holds a patent on “Methods for detecting sinusoidal obstructive syndrome (SOS)” (US patent no. 11,193,945 B2). The remaining authors declare no competing financial interests.
Correspondence: Sophie Paczesny, Medical University of South Carolina, 86 Jonathan Lucas, Rm HO612E, Charleston, SC 29425; email: paczesns@musc.edu; and Benjamin Djulbegovic, Medical University of South Carolina, 86 Jonathan Lucas, 3rd floor, Charleston, SC 29425; email: djulbegov@musc.edu.
