VTE Risk Assessment and Prophylaxis: The Reliability of Clinical Assessment by Medical Housestaff,

Hospital acquired venous thromboembolism (VTE) is a significant cause of mortality in hospitalized patients. The incidence of VTE may be as high as 40% in medical inpatients and is preventable in 50–75%. However, only one-half of eligible hospitalized patients receive adequate thromboprophylaxis. In response, national quality organizations and expert panels recommend a VTE risk assessment and risk-based prophylaxis for every inpatient. Point scoring systems have been proposed for risk stratification but have not been prospectively validated, and may be misleading; a recent study showed that medical residents using a point system made errors in risk stratification and choice of VTE prophylaxis. Thus, the optimal method of assessing VTE risk and whether these assessments can have adequate inter-rater reliability remains unknown.

1. To compare the inter-rater reliability of VTE risk assessment by paired expert reviewers within the paired team and to the clinical team's assessment. 2. To evaluate the appropriateness of VTE prophylaxis administered by clinical teams compared to expert reviewer's determinations.

We performed a cross-sectional study at a 464-bed public teaching hospital. Medical patients were randomly selected and their charts abstracted by four expert housestaff reviewers (two teams of two reviewers) who had been trained through literature review, case discussion and participation in guideline development. Paired reviewers independently assessed VTE risk blinded to the other reviewer's determination using clinical data and a ‘3-bucket' model (low; moderate or high; very high). Appropriateness of prophylaxis was based on VTE risk as well as contraindications to prophylaxis. Reviewers also recorded the primary teams' VTE risk assessment and prophylaxis choices. Reviewer discrepancies were adjudicated through a third blinded review. We calculated the inter-rater reliability between paired reviewers and between reviewers and clinical teams using weighted Kappa scores (K). We recorded reasons for disagreement between reviewers and teams.

A total of 40 charts were reviewed and analyzed for agreement on VTE risk. 36 charts were analyzed for appropriateness of VTE prophylaxis; 4 patients on therapeutic anticoagulation were excluded from this analysis. Compared to expert reviewers (E), medical teams (M) significantly underestimated VTE risk, as follows: low risk (E, 2.5% vs M, 20%); moderate to high risk (E,85% vs M,75%); very high risk (E, 12.5% vs M, 5%); P=0.004. In 11 of 12 cases of disagreement, team's assessment of VTE risk was lower than that determined by reviewers. Compared to the inter-rater reliability between experts and clinical teams, reliability was significantly better for the paired experts both for VTE risk assessment (P<0.01) and choice of prophylaxis (P<0.01).

Among the 8 (22%) of patients for whom the reviewers determined VTE prophylaxis was suboptimal, for most (n=6) the method of prophylaxis was less intensive than recommended by the guidelines, and the most common reason was failure to restart prophylaxis after an invasive procedure or transfer of care.

Our study shows that expert reviewers can assess VTE risk with a high degree of reliability. The risk assessments by clinical teams during routine clinical evaluation did not correlate well with expert risk stratification and underestimated the risk of VTE in medical inpatients. Incorrect risk assessments were common but the most frequent reasons for underutilization of VTE prophylaxis were oversights in ordering prophylaxis during care transitions or after invasive procedures. Although we trained our experts to be highly reliable in risk assessment this training cannot be generalized to most provider groups. An optimal approach to improving VTE risk assessment in clinical settings involving trainees would include real time decision support for risk assessment with linked VTE prophylaxis choices appropriate to the level of risk at the point of care.

No relevant conflicts of interest to declare.