Risk-assessment models for VTE and bleeding in hospitalized medical patients: an overview of systematic reviews

The clinical and economic burden of venous thromboembolism (VTE), including deep vein thrombosis (DVT) and pulmonary embolism (PE), is immense.¹  VTE is a complex multifactorial disease, influenced by acquired or inherited predispositions to thrombosis (eg, thrombophilia), environmental exposures (eg, clinical risk factors), and the interaction between them.²  The annual incidence of VTE in the adult populations is ∼1 in a 1000.^3,4  Approximately 50% of all VTE events occur during or shortly after hospitalization for surgery (24%) or acute medical illness (22%).^4-6  Patients hospitalized for an acute medical illness have an eightfold increased risk of VTE compared with the general population.^7,8  Medical costs in the United States are estimated at approximately $17 000 dollars more for patients who have a VTE event during or after a recent hospitalization compared with their hospitalized counterparts who do not experience a VTE event.¹ 

The use of pharmacological thromboprophylaxis reduces the incidence of VTE in a cost-effective manner in many patient populations but increases the risk of bleeding.^6,9  To inform optimal management, risk-assessment models (RAMs) can be used to aid in stratifying risk of developing a VTE or bleeding event for individual patients. A RAM is defined as a formal combination of multiple predictors from which risks of a specific end point can be calculated for individual patients. A RAM may also be called a prognostic model, risk (or clinical) prediction model, or predictive model.¹⁰  A RAM undergoes 3 main phases: model development (including internal validation), external validation, and investigations of impact on decision-making and patient outcomes.¹⁰ 

In 2018, the American Society of Hematology (ASH) together with the Michael G. DeGroote and MacGRADE Centres at McMaster University developed clinical practice guidelines for the prevention of VTE in hospitalized medical patients.⁴  The methods used to develop these guidelines were based on the Guidelines International Network (GIN)-McMaster Guideline Development Checklist,¹¹  the GRADE approach,¹²  and the Cochrane Handbook for systematic review methodology.¹³  We conducted an overview of systematic reviews to identify and describe RAMs and their clinical utility for VTE and bleeding in hospitalized medical patients to inform the ASH guidelines.⁴ 

We conducted an overview of reviews to identify systematic reviews that report on RAM development, validation, or impact studies for VTE and bleeding in hospitalized medical patients. We developed a protocol that was reviewed and revised by the authors, but we did not register it.

We initially searched Epistemonikos, Cochrane Database of Systematic Reviews, Medline, and Embase from 2005 through June 2017 with the help of an information specialist and conducted an update of the search in January 2020. Supplemental eTable 1 provides detailed descriptions of the search strategies. Our search included both Medical Subject Headings (MesH) and text-word terms that combined VTE-related terms with prognosis terms. We added a systematic review filter when using Medline and Embase. We used no language restrictions.

Prior to starting the screening process, 4 teams of 2 reviewers participated in training and calibration exercises. Teams of 2 reviewers screened, independently and in duplicate, the titles and abstracts of all identified citations. They then retrieved full texts of all citations judged as potentially eligible by at least 1 of the reviewers on each team. The reviewers screened the full texts independently and in duplicate and compared results. A third, senior reviewer resolved disagreements when necessary. Reviewers used standardized screening forms throughout the process. The eligibility criteria for study selection included the following characteristics.

We included studies that explicitly stated the use of the “systematic review” methodology, with or without conducting a meta-analysis, in the title or abstract. Also, the study must have reported conducting a search for individual studies in at least 1 database. We included systematic reviews that reported on development, validation, or impact studies of multivariable prediction/RAMs, tools, or scores, proposed for individual risk estimation of any future VTE or bleeding outcome in hospitalized medical patients.

We included systematic reviews that addressed adult patients hospitalized for an acute, critical, or chronic medical illness. An acute medical illness is defined as an illness that requires urgent, nonoperative care such as heart failure, respiratory insufficiency, stroke, and infectious or inflammatory diseases.⁴  A critical illness is one that presents as an immediately life-threatening condition that requires care in an intensive or critical care unit.⁴  A chronic medical illness is defined as an acute exacerbation of a chronic medical condition that requires hospitalization.⁴ 

We investigated all RAMs that assessed risk of VTE or bleeding in adults hospitalized for medical illness and were reported in the eligible systematic reviews.

Standard care without the use of RAMs or a different RAM than the one used in the intervention.

We evaluated the outcomes of VTE and bleeding. We defined VTE as any symptomatic or asymptomatic DVT or PE from hospital admission up to 90 days postdischarge. We considered bleeding as any major or nonmajor but clinically significant bleeding up to 90 days postdischarge.¹⁴ 

We included systematic reviews that reported on studies in which the patients were admitted to an inpatient ward or intensive care unit for medical illness.

We conducted calibration exercises and piloting of the data-extraction form prior to the start of the process. Using a standardized form, a team of 2 reviewers (A.J.D. and R.C.) extracted data independently and in duplicate from all eligible studies and compared results. They consulted a third reviewer (H.J.S.) in case of any disagreement.

For the identified RAM systematic reviews, the reviewers abstracted data on the following characteristics.

• Main elements of the search strategy (including databases searched, date of search).

• Approach used to synthesize findings (narrative synthesis, meta-analysis).

• Risk-of-bias tool used to assess individual studies and results.

• Authors’ assessment and conclusions.

• Type of prognostic model studies included (ie, development, validation, or impact).

• Population.

• Outcomes.

• Setting.

• Time frame of prognostic measurement.

• Results (predictive performance).

We assessed the risk of bias in systematic reviews using the Risk of Bias in Systematic Reviews [ROBIS] tool. The tool includes 3 phases: assessing relevance (optional), identifying concerns with the review process and judging risk of bias.¹⁵  We focused on phases 2 and 3. Phase 2 considers the following 4 domains: study eligibility criteria, identification and selection of studies, data collection and study appraisal, and synthesis and findings. The signaling questions in each of the domains are judged as yes, probably yes, probably no, no, or no information. In phase 3, we rated the overall risk of bias as low, high, or unclear depending on the rating of the individual domains. For the individual studies included in the systematic reviews, we reported on the risk-of-bias tool used and the judgments made by the authors when available.

We used a narrative synthesis of included systematic reviews to summarize our findings. We presented the findings of any qualitative or quantitative syntheses conducted by the authors of the reviews. We focused on identifying and describing the RAMs, their performance, and gaps in their development, validation, or assessment of impact. If a meta-analysis was presented in 1 of the included systematic reviews, we presented the results and the relevant methodological aspects (eg, types of data, effects measured, heterogeneity, sensitivity analysis). If different information regarding the same RAM was provided in the reviews, we narratively described all findings to provide a comprehensive description of that model. We did not perform a quantitative synthesis of the RAMs, as the main aim of an overview of reviews is to provide a summary of existing research synthesis and not resynthesizing evidence.¹⁶ 

Figure 1 illustrates the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.¹⁷  In our original search, we identified 6095 citations, of which we included 144 studies for full text assessment. From those, we included 2 systematic reviews. Both reviews evaluated RAMs for VTE in hospitalized medical patients.^18,19  We did not identify any systematic review that evaluated RAMs for bleeding. When we conducted an update of our search in January 2020, we identified an additional 4122 citations, none of which fulfilled our inclusion criteria after full evaluation.

The included systematic reviews aimed to identify RAMs developed to calculate the risk of VTE in hospitalized nonsurgical patients and to evaluate their generalizability, validity, and utility.^18,19  Stuck et al focused only on acutely ill medical patients and searched for English, German, French, or Italian studies in only 1 database (Medline), from inception until 30 May 2016.¹⁹  They then performed an additional search to identify impact analysis studies that may have been missed.¹⁹  The authors did not conduct a quality appraisal of the included studies.¹⁹  Huang et al included a comprehensive search for English articles across 4 databases from inception until December 2011.¹⁸  The authors of that review appraised the evidence by using a modified Downs and Black checklist for the RAMs developed using individual patient data, and the modified Appraisal of Guidelines for Research and Evaluation (AGREE) instrument for the RAMs generated based on consensus approaches, published data, and clinical expertise.¹⁸  The reviews included 19 unique primary studies that reported on 15 RAMs. Of those, 3 studies and 4 RAMs were common in both systematic reviews.^20-38  Stuck et al identified 4 additional impact studies from their supplementary search.¹⁹  Neither systematic review conducted a meta-analysis of the results.^18,19  Huang et al highlighted that pooling was not possible due to variability in the methods to develop the RAMs, in the outcome measurements, and in the number, type, and strength of associations of the included VTE risk factors.¹⁸  Authors of both reviews concluded that there is a lack of generalizability and adequate validation of the published RAMs, which hinders their use in clinical practice.^18,19  However, Stuck et al encouraged the implementation of any of the available RAMs to improve the consistency of use of thromboprophylaxis until further evidence is available.¹⁹  Characteristics of the systematic reviews are detailed in Table 1.

We rated the overall risk of bias for the systematic review by Huang et al as low,¹⁸  and the review by Stuck et al as high.¹⁹  We judged both reviews as low risk in terms of study eligibility criteria (domain 1). However, we originally rated both systematic reviews at high risk for their identification and selection of studies for inclusion (domain 2), but each review came with different concerns. Stuck et al searched only 1 database,¹⁹  and did not conduct a sufficiently sensitive search to identify all potentially eligible studies.¹⁹  On the other hand, Huang et al made no mention of conducting an independent and duplicate screening process,¹⁸  or using standardized screening forms throughout the screening and abstraction processes.¹⁸  Also, both reviews placed restrictions on language that may have led to missing studies.^18,19  Despite originally rating Huang et al as high risk of bias for the domain identification and selection of studies for inclusion, we considered it low risk of bias in our overall judgement.¹⁸  We made this decision because our results revealed that all relevant studies were captured within the search date, from inception through May 2011, despite the methodological limitations noted herein.¹⁸  Regarding data collection and study appraisal (domain 3), we rated the review by Huang et al as low risk of bias for appraising the included studies and the review by Stuck et al as high risk of bias for not conducting a critical appraisal.^18,19  Neither of the reviews conducted a meta-analysis, which made it challenging to assess their methods of synthesis and presentation of findings domain (domain 4) using ROBIS. However, our rationale for rating Huang et al as low risk was due to the authors stating that the results were too heterogeneous and could not be pooled.¹⁸  We rated the review by Stuck et al as high risk because the authors did not provide a reasoning for not pooling the results.¹⁹  Supplemental eTable 2 provides the risk-of-bias assessment of the systematic reviews by phase and domain.

Huang et al described that the quality assessment, using the modified Downs and Black checklist, ranged between 55% and 88% for the RAMs derived using individual patient data.¹⁸  The quality appraisal score ranged between 48% and 77% for the RAMs developed by consensus based on the modified AGREE instrument.¹⁸ Tables 2 and 3 report the quality-assessment score for each of the RAMs.

Table 2 provides a detailed description of the included RAMs. From the 15 included RAMs, 7 were derived by identifying risk factors with predictive power using individual patient data mainly from medical records. These include the RAMs by Alikhan et al, Weill-Engerer et al, Yale et al, Spyropoulos et al (IMPROVE RAM), Rothberg et al (Multivariable RAM), and Woller et al (4-Element and the full logistic RAMs).^22,29,31-34  The individual studies included cohort studies (n = 3),^29,31,33  case-control studies (n = 2),^32,34  and a study based on a randomized controlled trial (RCT).²²  The studies were all multicenter with 2 being multinational,^22,31  and sample sizes ranged between 380³⁴  to ∼243 000 patients.²⁹  The eligibility criteria of individual studies differed mainly in the age cutoffs, ranging from >18 years to >65 years, the inclusion of history of surgery or trauma, length of hospital stay, known thrombophilia, and the exclusion of patients on pharmacologic thromboprophylaxis or anticoagulation. The proportion of cancer patients included in the derivation cohorts also varied widely ranging between 9%³²  and 44%.³³  Two studies reported on DVT alone (n = 2),^32,34  whereas the rest of the studies used VTE (DVT, PE, and both) as their primary end point. All but 1 study considered symptomatic events only²² ; however, the definitions and the methods of diagnosis of the outcomes varied. Two studies used International Clinical Diagnosis Code version 9 (ICD-9) to define VTE.^29,33  One of these 2 studies included codes for upper-extremity, superficial, and chronic DVT that were excluded from other studies, and did not use diagnostic test results and evidence of VTE treatment to validate the outcome.³³  The other study validated the outcomes identified using ICD-9.²⁹  Only 4 studies described their methods for diagnosis of DVT (ultrasonography or venography) or PE (lung scan, pulmonary angiography, or spiral computed tomography scanning).^22,29,31,32  The follow-up time ranged from the index hospitalization up to 90 days postdischarge. The methods for selection of candidate risk factors also varied among the RAMs, with only 1 study using Kaplan-Meier and Cox multiple regression analyses to adjust for timing of events³¹  and another considering hospital clustering as a factor.²⁹  The number of risk factors ranged between 4 and 14 in all the RAMs except the full logistic model, which included 86 risk factors.³³  The measure of discrimination of the RAMs was reported in terms of the area under the curve (AUC) and ranged between 0.65 and 0.89.^18,19  VTE prophylaxis administered in the hospital was considered as a potential confounder in only 2 of the included studies.^29,31  Both studies reported no statistically significant impact of VTE prophylaxis on the outcome, and only 1 study included it as a prognostic factor in the RAM.²⁹ Table 3 details the individual study characteristics related to these RAMs.

The other 8 RAMs included in the systematic review were empirically developed based on consensus approaches, risk factors identified from a literature review, and input from clinical experts (ranging between 3 and 24 members). These include the RAMs described by Caprini et al (Caprini RAM), Cohen et al, Samama et al, Rocha et al, McCaffrey et al, Kucher et al, Barbar et al (Padua RAM), and Nendaz et al (Geneva RAM).^{23-27,30,37,39}  Only 1 of the 8 RAMs conducted a comprehensive systematic review of the literature to identify all potential risk factors prior to developing the model.³⁷  One study focused on symptomatic DVT as the primary end point.²⁶  Another study considered VTE as an outcome, but it was unclear whether it was symptomatic VTE only or symptomatic and asymptomatic VTE.²³  The rest of the studies only considered symptomatic VTE as their primary end point. The number of risk factors included ranged between 8 and 39 for all except 4 RAMs whose risk factors were not described in the systematic reviews.^25,26,30,37 Table 4 describes the individual study characteristics related to these RAMs.

The systematic reviews identified 5 RAMs that underwent internal validation without external validation. The RAMs by Woller et al (4-Element RAM and full logistic RAM) and the RAM by Rothberg et al (multivariable RAM) were developed based on individual patient data where risk factors were included based on their predictive ability identified in a regression analysis and compared in 1 retrospective study to one another.^29,33  Comparing the 2 RAMs by Woller et al showed that the full logistic RAM reported a slightly higher AUC (0.86) compared with the 4-Element RAM (0.84).³³  The study that assessed the RAM described by Rothberg et al (Multivariable RAM) in a validation cohort reported an AUC of 0.75.³³  The other 2 RAMs developed empirically by Samama et al and McCaffrey et al were internally validated using clinical cases.^18,26,30  Samama et al reported a 70% agreement between the levels of risk and judgments by clinicians in a validation effort.³⁰  McCaffrey et al found that the mean total risk score in the VTE group was significantly higher than in the non-VTE group.²⁶ 

Only Stuck et al reported on external validation efforts of the RAMs.¹⁹  The authors of the systematic review reported the results of the validation studies conducted for the RAMs by Caprini et al (Caprini RAM), Nendaz et al (Geneva RAM), Spyropoulos et al (IMPROVE RAM), Kucher et al, and Barbar et al (Padua RAM), all of which were multicenter except for the Caprini RAM.^{20,21,27,28,31,33,35,36}  The Caprini RAM was validated in 3 studies. The first study describes the validation of the Caprini RAM in a population including cancer patients and reported that the incidence rate of VTE in low-risk patients was 0% compared with 4.2% in high-risk patients.²¹  The second study compared the Caprini RAM to the RAMs by Kucher et al and Barbar et al (Padua RAM) and found that the Caprini RAM assigned more VTE patients into high- or highest-risk groups compared with the other 2 RAMs.³⁶  The third study compared the cumulative risk of inpatients with VTE vs those without VTE using the Caprini and Padua RAMs. The authors reported that the high- to highest-risk group compared with the low- to moderate-risk group had similar odds ratios (ORs) using both RAMs with an OR of 3.01 for the Caprini RAM and 2.9 for the Padua RAM. However, 82.3% of VTE patients were found to be in the high- to highest-risk group according to the Caprini RAM whereas 30.1% of VTE patients were found to be in the high-risk group using the Padua RAM. A prospective multicenter study compared the RAM described by Nendaz et al (Geneva RAM) to the RAM by Barbar et al (Padua RAM) in ∼1500 patients with VTE and reported a favorable prediction of VTE and VTE-related mortality using the Geneva RAM.²⁷  The study also found that the Geneva RAM more accurately identified low-risk patients who do not require thromboprophylaxis with a negative likelihood ratio of 0.28 compared with 0.51 for the RAM by Barbar et al (Padua RAM). The internal validation of the RAM by Spyropoulos et al (IMPROVE VTE RAM) reported that during hospitalization, the observed VTE rate for an IMPROVE-RAM score of 2 or 3 points (1.5%) and 4 points (5.7%) correlated with predicted VTE risk with an AUC of 0.69.³¹  The IMPROVE RAM was then externally validated in 2 studies. The first study reported an AUC of 0.77 based on high-, moderate-, and low-risk groups.²⁰  The second study found good discrimination of low-risk and at-risk medical patients with an AUC of 0.70 despite using different cutoffs for risk classification, with more than two-thirds in the low-risk group not requiring prophylaxis.²⁸  The RAM by Kucher et al was compared with the RAMs by Woller et al (4-Element and full logistic RAMs) in 1 study and was found to have the lowest AUC (0.76) among the 3 RAMs. In the internal validation cohort, the RAM by Barbar et al (Padua RAM) identified a 32-fold increased risk of VTE in the group of patients not on prophylaxis with a high score compared with a low score. It was then externally validated in 4 studies, 3 of which were described previously, including the study that compared the RAM by Barbar et al (Padua RAM) to the RAMs by Caprini et al (Caprini RAM) and Kucher et al, the study that compared Padua to the Caprini RAM alone, and the third that compared Padua to the Geneva RAM. A fourth study assessed the Padua RAM alone and found a correlation between the risk groups and in-hospital mortality but not with the incidence of VTE.³⁸  The authors suggested that the Padua RAM is better used as a general comorbidity and disease severity index score than a VTE RAM.³⁸ Table 5 describes the findings from comparative studies of risk assessment models identified in the systematic reviews.

Stuck et al identified impact studies.¹⁹  The authors found that 3 RAMs described in the systematic reviews by Barbar et al (Padua RAM), Nendaz et al (Geneva RAM), and Kucher et al have been assessed in terms of thromboprophylaxis rates or clinical outcomes. The impact of the RAM by Barbar et al (Padua RAM) was assessed in a single-center study and was found to improve rates of adequate prophylaxis.²³  The RAM described in the systematic reviews by Nendaz et al (Geneva RAM) was included in an e-Alert system, as part of a multicenter trial, and showed that its use increased appropriate prophylaxis rates.²⁷  Two single-center studies tested the impact of the RAM described by Kucher et al. One was a randomized trial that included the RAM in a computer-alert program and showed an increase in the use of prophylaxis and a reduction in VTE rates among at-risk patients.³⁹  The second study confirmed that implementing a computer alert program using the RAM may increase prophylaxis rates.⁴⁰  No impact studies were conducted to assess the economic impact of the RAMs.

We conducted an overview of systematic reviews to identify RAMs for VTE and bleeding in hospitalized medical patients. We identified 15 unique RAMs for VTE, 7 of which were derived from individual participant data and 8 that were developed empirically using consensus approaches, risk factors identified from a literature review, and clinical expertise. The RAMs described in the systematic reviews by Caprini et al (Caprini RAM), Nendaz et al (Geneva RAM), Spyropoulos et al (IMPROVE RAM), Kucher et al, and Barbar et al (Padua RAM) have been externally validated. The RAMs described by Barbar et al (Padua RAM), Nendaz et al (Geneva RAM), and Kucher et al have been evaluated in terms of thromboprophylaxis rates or clinical outcomes and have been reported to increase appropriate prophylaxis rates. However, their economic impact has not been assessed.

Following the publication of the most recent systematic review by Stuck et al in 2017,¹⁹  additional validation and impact studies were conducted. One study used the Kucher RAM along with electronic alerts and performance audits and reported increased rates of thromboprophylaxis in high-risk patients and decreased 90-day VTE rates without an observed increase in adverse events.^44,45  Also, although our overview of systematic reviews did not identify any impact assessment of the Caprini RAM, we did identify a single-center study that used the Caprini RAM as part of a multifaceted quality-improvement initiative.⁴⁶  The study reported increased VTE prophylaxis rates and a reduction in hospital-acquired VTE rates with the use of the Caprini RAM.⁴⁶  Another prospective comparative study that was not included in the systematic reviews was conducted to assess the performance of the Geneva, Padua, and IMPROVE RAMs on thromboprophylaxis rates in acutely ill hospitalized medical patients.⁴⁷  The study reported comparable discrimination abilities with a 90-day AUC of 0.71 for the Geneva RAM and an AUC of 0.70 for both the Padua and IMPROVE RAMs in patients not on thromboprophylaxis.⁴⁷  The authors of the study highlighted that the IMPROVE RAM classified more patients as low risk (two-thirds of patients) compared with the Geneva RAM (one-third of patients), but with possibly lower sensitivity and greater VTE risks.⁴⁷  Also, a secondary analysis of a cohort of acutely ill hospitalized medical patients participating in a cluster RCT (The Prevention of Venous Thromboembolism Disease in Emergency Departments [PREVENU] study) was not included in the systematic reviews.⁴⁸  This study aimed to assess the Caprini, IMPROVE, and Padua RAMs and compared their performance to advanced age as a stand-alone predictor.⁴⁸  The study reported poor discriminative ability of the RAMs to identify non–critically ill inpatients at risk of VTE and found that the RAMs did not perform better in comparison with risk assessment using advanced age as a sole predictor.⁴⁸  Our search did not capture a systematic review reporting on bleeding RAMs in hospitalized medical patients. However, a primary study by Decousus et al in 2011 reported on the development of the IMPROVE bleeding RAM for in-hospital bleeding risk in acutely ill medical patients.¹⁴  Two studies externally validated the IMPROVE bleeding RAM and found similar results to the derivation effort in which hospitalized medical patients with a score >7 were shown to have over a twofold increased risk of any bleed or major bleed compared with those with lower bleeding-risk scores.^49,50 

Our study has several strengths, including the systematic and rigorous methods, a robust search strategy, the use of broad inclusion criteria, and our duplicate and independent screening and data-abstraction process. Another strength of our study is the applicability of our findings. The RAMs identified and the description of their clinical utility informed guideline developers. The RAMs can also aid health care practitioners and health care systems in selecting RAMs to optimize shared decision-making and provide appropriate prophylactic management.

The limitations of this study result from the included studies themselves. Huang et al applied language restrictions during the initial search¹⁸  and Stuck et al applied language restrictions during the full text evaluation.¹⁹  Stuck et al also conducted searches in only 1 database.¹⁹  These limitations may have led to missing relevant studies. Some of the studies that were missed in the overview of reviews include a study by Zakai et al in 2014 in which the authors empirically derived the Medical Inpatients and Thrombosis (MITH) RAM to assess risk of VTE in medical patients on admission.⁵¹  Another study that the overview of reviews did not capture was the IMPROVEDD RAM, in which the variable D-dimer was added to the IMPROVE RAM and showed an enhanced VTE risk discrimination in hospitalized medical patients compared with the IMPROVE RAM.⁵²  Also, we did not identify the external validation study by Greene et al in 2016. This study aimed to validate the Kucher, Padua, IMPROVE, and Intermountain RAMs.⁵³  The authors reported that all RAMs showed good calibration but uniformly poor discrimination.⁵³ 

The RAMs identified in our study can be used to estimate baseline risks of future health outcomes in people with a given disease or health condition⁵⁴  and to aid health care practitioners in identifying an individual patient’s risk of VTE based on their individual characteristics. Also, many of these RAMs can be readily embedded in clinical decision aids to individualize the population-based recommendations. However, numerous shortcomings have limited the interpretation and clinical utility of the developed RAMs. First, the inability to more accurately identify medically ill patients at low or high risk of VTE may lead to overuse or underuse of prophylaxis and increased adverse events such as bleeding or thrombosis.⁷  Second, the complexity of some of the current RAMs (due to the large number of risk factors, such as 39 variables in Caprini and 86 in the full logistic RAM) limit their use to computer-based calculations.^24,33  Third, the variability in the derivation and validation methods of the RAMs limit their comparability. Fourth, in the 2 systematic reviews, some RAMs were not found to be externally validated, limiting their use in clinical practice.^29,31  Also, other RAMs have been validated in very specific medical subpopulations such as those with sepsis or cancer as noted in the Padua and Caprini RAMs, thus limiting the generalizability of their results to the general medical population.^21,38  Fifth, there is a limited number of prospective comparative studies that assess the impact of applying different RAMs in clinical practice on outcomes.

In this study, we provide the original and the update of the overview of systematic reviews evaluating VTE and bleeding RAMs used in hospitalized medical patients that informed the ASH guidelines. Findings from the original search directed us to conduct several follow-up studies. First, our work led us to conduct a systematic review of prognostic studies to identify all potential risk factors for VTE and bleeding in hospitalized medical patients.⁵⁵  Second, we assessed the certainty of the evidence of the identified risk factors using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.⁵⁵  Third, we aimed to develop RAMs for VTE and bleeding that are accurate and usable in clinical practice.⁵⁶  This was done by using the results of the systematic review and the clinical and methodological input of an expert panel.^55,56  The expert panel made judgments on whether to include, potentially include, or exclude the identified risk factors from the final RAMs by considering GRADE evidence-to-decision framework criteria using the Delphi method.⁵⁶  Fourth, we standardized the definitions of the identified included and potentially included risk factors based on our systematic review to decrease variability in methods of measurement across settings and to provide more clarity to health care professionals when evaluating patients’ risks of VTE and bleeding.⁵⁶ 

We conducted an overview of systematic reviews of VTE RAMs in hospitalized medical patients to inform the 2018 ASH guidelines on VTE prophylaxis.⁴  Our findings can assist experts in selecting a RAM to integrate in their health care systems, although further effort should be made to enhance these existing RAMs. This will allow for standardized approaches to estimating patients’ risks of VTE and individualization of population-based guideline recommendations for appropriate prevention strategies.

This overview of systematic reviews was conducted to inform the American Society of Hematology (ASH) guidelines for VTE prophylaxis in hospitalized medical patients.

This work was supported by ASH.

Contribution: A.J.D., E.A.A., and H.J.S. conceived and designed the study; A.J.D., A.B.R., R.Z.M., R.C., I.E.-I., S.G.K., A.A., T.L., R.W., E.A.A., and H.J.S. acquired data; A.J.D., A.B.R., and H.J.S. analyzed data and drafted the manuscript; and A.J.D., A.B.R., F.A.S., R.Z.M., R.C., I.E.-I., K.A.B., A.E.B., M.C., F.D., S.R.K., S.M.R., N.A.Z., A.A., S.G.K., T.L., W.W., R.W., A.I., E.A.A., and H.J.S. interpreted results, critically revised the manuscript, and approved the final version of the manuscript.

Conflict-of-interest disclosure: All authors were members of the 2018 ASH guideline panel, members of the overview of systematic review team, or both. A.J.D., I.E.-I., A.I., E.A.A., and H.J.S. reported that they are members of the GRADE Working Group. H.J.S. is cochair of the GRADE Working Group. A.J.D., F.A.S., M.C., N.A.Z., A.A., S.G.K., A.I., E.A.A., and H.J.S. were authors on a manuscript that conceptualized and tested a novel approach to selecting prognostic factors for the development or update of a RAM that has been accepted in the journal Blood Advances. H.J.S. reported being cochair of the ASH 2018 guidelines for prophylaxis for hospitalized and nonhospitalized medical patients and reported grant funding from the Centers for Disease Control and Prevention (CDC) for the project on identifying and selecting VTE and bleeding prognostic factors for the development or update of a RAM using a novel approach. M.C. reported being a former board director (2013-2017) of the American Heart Association and chairing the ASH 2018 guidelines for management of VTE: prophylaxis for hospitalized and nonhospitalized medical patients. F.A.S., A.B.R., K.A.B., A.E.B., F.D., S.R.K., S.M.R., and N.A.Z. reported participating as panel members for the ASH 2018 guidelines for management of VTE: prophylaxis for hospitalized and nonhospitalized medical patients, for which they disclosed their conflicts that are provided in the supplement of the published guideline. N.A.Z. also reported receiving honoraria in 2017 from ASH for the Highlights of ASH 2017 meeting (Dallas, New York, Latin America). N.A.Z. and M.C. reported intellectual conflicts as leads in the group that derived and validated the MITH RAM for VTE risk assessment in hospitalized medical patients. The remaining authors declare no competing financial interests.

Correspondence: Holger J. Schünemann, Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St West; Hamilton, ON L8N 3Z5, Canada; e-mail: schuneh@mcmaster.ca.