External Validation of a Risk Assesment Model for Bleeding Risk in Medical Patients: Findings from the Improve Investigators in a Tertiary Health System

Background: The risk of venous thromboembolism (VTE) in hospitalized medical patients remains high and pharmacologic thromboprophylaxis remains suboptimal. Current national guidelines call for physicians to assess an individual patientÕs VTE and bleeding risk, prior to administering VTE prophylaxis. Therefore, Risk Assessment Models (RAMs) are needed to identify those patients at high risk for a bleed. The International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) bleed RAM remains the only evidence-derived and weighted bleed risk model in this population using 11 clinical and laboratory factors (Table 1). However, this RAM has not undergone external validation. The aim of our study is to validate the IMPROVE bleed RAM for hospitalized medical patients in a large tertiary health system.

Methods A retrospective chart review was conducted, utilizing data from our electronic health records (EHR) and administrative data systems, between October 1^st 2012 and July 31^st 2014. After applying the inclusion and exclusion criteria as outlined in the IMPROVE study, and accounting for missing EHR data, we identified 12082 subjects. Missing data included INR, platelet count, accurate weights and gender. Within this population we identified each of the eleven IMPROVE RAM risk factors utilizing administrative (ICD9) and EMR data. We identified outcomes, major (MB) or non-major bleeds (NMB), using ICD9 billing codes. To validate the IMPROVE bleed RAM, we then applied the point scoring system to compute risk scores for each patient in the validation sample. Lastly, we dichotomized the patients into those with a score <7 (low risk) vs. >=7 (high risk), as outlined in the original study, and compared the frequencies/rates of bleeding in the derivation and validation samples at these two different levels. All analyses were carried out in SAS Version 9.3 (Cary, NC).

Results: Among the 12082 subjects, 2.6% had a bleed within 14 days (compared with 2.1% in the original IMPROVE inception cohort). In our population, overall bleeding rates were 2.2% in those with a score <7 and 4.7% with a score of >=7. NMB occurred in 1.29% of patients with a score <7 and 3.1% of patients with a score of >=7, p<0.0001. MB rates were 1.51% of patients with a score of <7 and 3.23% with a score of >=7, (p<0.0001). The calculated sensitivity and specificity from the derivation sample were 35.86% and 90.85%. For the validation sample, these estimates were 33.97% and 81.47%. The PPV and NPV for the derivation sample were 10.5% and 98.2% and 4.7% and 97.9% for the validation sample. Using only MB as an outcome resulted in a sensitivity of 33.33%, specificity of 81.33%, PPV of 3.23% and NPV of 98.49%.

Conclusion: This study represents the first external validation of an evidence-derived and weighted bleed RAM, the IMPROVE bleed RAM, in a hospitalized medically ill patient population. A cut-point score of 7 or above was able to identify a high-risk patient group for MB and NMB, with a doubling of the bleed rates, consistent with the original derivation score. The high NPV for any bleed and MB are evidence that the RAM is also useful in the clinical setting to predict those patients at low risk of a bleed. However, a limitation of this study was the retrospective study design and reliance on claims data, likely resulting in an underestimation of NMB and the magnitude of effect the score has on bleed risk. The IMPROVE bleed RAM has the potential to allow for more tailored approaches to pharmacologic thromboprophylaxis in hospitalized medical patients.