Diagnostic value of immunoassays for heparin-induced thrombocytopenia: a systematic review and meta-analysis

In clinical practice, immunoassays are pivotal for the workup of patients with suspected heparin-induced thrombocytopenia (HIT).^1-4  HIT is a life-threatening complication of heparin therapy that affects a significant number of patients.⁵  It is associated with a high morbidity and mortality because of a massive pro-coagulant state, with a high incidence of extensive venous and arterial thrombosis, limb loss, and even death.^6,7  Suspicion of HIT requires an immediate diagnostic workup to prevent severe complications.^1,8,9  Still, diagnosis of HIT is challenging.^1,10  Functional assays, such as the serotonin release assay (SRA) or the heparin-induced platelet activation assay (HIPA) are accepted as gold standard, but are rarely available in a timely manner.^2,3,11  Clinical assessment tools such as the thrombocytopenia, thrombosis, timing of decrease in platelet count, and other causes for thrombocytopenia (4Ts) score can exclude HIT in many patients if conducted by experienced observers.¹²  However, the positive predictive value is low,¹²  they are subject to a relevant inter-observer variability,¹³  and they are not adequately evaluated in all settings.¹⁴  To make a treatment decision at bedside, physicians are recommended by current guidelines and recent reviews to order an HIT immunoassay in all patients with an intermediate or high-risk 4Ts score.^1,8,10-12,15 

Many tests and several classes of assays have been developed by manufacturers. They recognize different classes of antibodies, use varying thresholds, and may apply test variations such as the high-dose heparin confirmation step. All tests are based on detecting antibodies targeting complexes of platelet factor 4 (PF4) bound to heparin (or other polyanions respectively).^16,17  A solid-phase enzyme-linked immunosorbent assay (ELISA) using a low optical-density threshold (OD) and targeting immunoglobulin G (IgG), IgM, and IgA antibodies was developed in the mid-1990s.^16,18  Later, other tests were introduced: IgG-specific assays,¹⁹  high- and intermediate-OD thresholds,^19,20  and the high-dose heparin confirmation step.²¹  Other classes of assays have been developed to enable short turnaround times and a 24-hour service: particle gel immunoassay (PaGIA), particle immunofiltration assay (PIFA), lateral flow immunoassay, latex agglutination assay, and chemiluminescent immunoassay (CLIA). Different antibody specificities are available for CLIA, and several thresholds have been proposed for CLIA and PaGIA. The corresponding diagnostic accuracy studies are hard to interpret because of varying results, imprecise estimates, and sometimes conflicting data. In clinical practice, it is difficult to choose the right assay because it remains unclear if and how test characteristics influence the diagnostic value. Several authors have raised concerns that the diagnostic value may differ relevantly among the individual tests.^2,22-24  As a result of this uncertainty, the application of assays varies greatly among laboratories.²⁵ 

The aim of the present investigation was to assess the diagnostic accuracy with regard to different classes of assays, antibody specificities, thresholds, manufacturers, application of a high-dose heparin confirmation step, and material used. In addition, we assessed whether different reference standards used in diagnostic accuracy studies have an impact on diagnostic accuracy. We retrieved all available data, applied strict inclusion criteria, systematically assessed methodological quality, and pooled diagnostic accuracy measures if possible.

Before starting the investigation, we developed a research protocol according to recommendations of the Cochrane Diagnostic Test Accuracy Working Group.²⁶ 

A search strategy was developed for the Medline, EMBASE, and Cochrane Collaboration databases to identify diagnostic accuracy studies of immunoassays for diagnosis of HIT (see supplemental data on the Blood Web site). Search strategy was refined using keywords of references found in a pilot search and after manual review of reference lists. Search strategy was tested in 12 index publications (100% sensitivity). The literature search was supplemented by manual review of reference lists, including the American College of Chest Physicians Evidence-Based Clinical Guidelines (9th edition).²  No restrictions or filters with regard to language, publication date, or age range were applied. The last search run was conducted on November 2, 2014. Records were screened by 2 investigators (M.N., A.t.C.-H.); all hits were assessed in full text for eligibility.

The following inclusion criteria were applied: (1) evaluation of a commercially available or fully described immunoassay to detect heparin/PF4 antibodies; (2) application of the index test to patients with suspected HIT; (3) application of a reference standard that is established or fully described; and (4) numbers of true positives, false positives, true negatives, and false negatives are reported or can be calculated. We considered a reference standard test to be established if it was evaluated in reasonably designed studies and implemented in routine practice. Studies were excluded in case of (1) application of the index test to patients without any suspicion for HIT (screening test), (2) evaluation of tests that are not immunoassays, (3) evaluation of confirmatory tests after application of a previous laboratory test, (4) reference standard not clearly defined, and (5) studies that compare diagnostic accuracy of 1 immunoassay with another 1 without using an established reference standard. Two reviewers assessed eligibility and consensus was achieved by discussion (M.N., A.t.C.-H.; raw agreement, 0.91; Cohen’s κ, 0.71).

Included studies were reviewed in duplicate and the following data were extracted: author; year of publication; study design; type of patient selection; characteristics of study population including cohort criteria, inclusion procedure, and setting; characteristics of index test including antibody specificity, thresholds, material used, and use of high-dose heparin confirmation test; application of index test; characteristics of reference standard; application of reference standard; sample size; disease prevalence; and numbers of true positives, false positives, true negatives, and false negatives (reported or calculated). Test results were categorized according to the “intention-to-diagnose” principle to avoid biased overestimation of diagnostic accuracy²⁷ : inconclusive results of the index test were classified as negative if the reference standard was positive and were rated as positive if the reference standard was negative.²⁷  Observations were excluded from analysis if the reference standard revealed inconclusive results. For example, in 1 study, an inconclusive PIFA result was classified as negative because SRA was positive, another inconclusive PIFA result was counted as positive because SRA was negative, and 2 observations were excluded because of a borderline SRA result in combination with a negative ELISA and a low 4T score.²⁸  Two-by-two tables were created for every evaluation study. Sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and pretest probability (prevalence) were calculated. To account for differences in the scale of the threshold among different assays (ELISA assays in particular), we categorized thresholds into low, intermediate, and high. For ELISA, ODs ≤0.7 were classified as low, OD between 0.8 and 1.4 as intermediate, and OD >1.4 as high. A positive/negative PaGIA result was ranked as low, and a titer of 2 or 3 as intermediate. LFI results were categorized as low (no studies using titration available). For CLIA, a threshold of 1.0 U/mL was classified as low, between 1.0 and 2.8 U/mL as intermediate, and above 2.8 U/mL as high. Threshold of latex agglutination assay (1.0 U/mL) was classified as low.

The methodological quality of the individual studies was assessed in terms of risk of bias and concerns regarding applicability using the revised tool for quality assessment of diagnostic accuracy studies (QUADAS-2). Details of this validated and widely accepted instrument are published elsewhere.²⁹  Then, methodological quality was assessed in 4 domains (patient selection, index test, reference standard, flow and timing) by using signaling questions. According to the published guidelines, we adjusted the signaling questions to our particular research questions and developed detailed decision criteria. In a pilot study, the adapted QUADAS-2 tool was applied to a subset of 7 studies by 2 investigators (M.N., A.t.C.-H.) with good agreement. Signaling questions and decision criteria were refined after these pilot studies to further improve reliability. Application of adapted QUADAS-2 tool to all included studies was done in duplicate and disagreement was solved by discussion between 2 investigators.

To assess the diagnostic accuracy of different classes of assays, we grouped observations by class, antibody specificity, threshold, and the use of the high-dose heparin confirmation step. In addition, we considered differences in ELISA assay designs by grouping according to manufacturer, antibody specificity, threshold, and use of the high-dose heparin confirmation step. To observe the impact of different reference standards used in evaluation studies, we conducted a stratified analysis in the assay class with the most accessible studies and performed a meta-regression analysis (see the following section). To maintain independency of observations, studies that were conducted in the same study population were included only once in each group. In these cases, we selected the study to be included according to the following priorities: (1) the largest number of observations and (2) type of manufacturer that was included less often within the particular category. Because heterogeneity is expected to be high in systematic reviews of diagnostic accuracy studies, we followed current recommendations and did not calculate an I² statistic.³⁰  Diagnostic accuracy measures were pooled with the use of the unified model,³¹  comprising a bivariate random-effects model³²  and a hierarchical summary receiver operating characteristics model.³³  However, it was not possible to fit these models in every category because at least 4 studies are required to perform the analysis. To give an approximation of the diagnostic accuracy in these tests, we fitted a fixed effect model according to Mantel and Haenszel.³⁰ 

Several measures were carried out to explore sources of heterogeneity and to recognize potential sources of bias. First, results of studies with a higher methodological quality were compared with estimates obtained from all studies. We selected the criterion of cohort studies (in contrast to case-control studies) because it is considered to have the highest impact on diagnostic accuracy study results.³⁴  Second, we compared measures between tests within the same study population using the same reference standard. Third, estimates obtained with accepted gold standards (SRA/HIPA) were compared with estimates from all studies. Finally, we conducted a random-effects meta-regression³⁵  using diagnostic odds ratio as outcome variables and important characteristics as predictors (type of reference standard [SRA/HIPA vs others], material used [plasma vs serum], setting [surgical vs mixed population; intensive care unit patients vs mixed population]).

All statistical analyses were conducted using Stata 13.1 (StataCorp, 2013, Stata Statistical Software, release 13; College Station, TX), figures of methodological quality were generated using Review Manager (version 5.3, Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).

Literature search yielded 2716 records, including 2 publications identified by manual review (Figure 1). After removal of duplicates, title and abstract were screened in 2261 records. We selected 258 publications for full-text review. Of these, 37 were excluded because of duplicate publication (conference abstract as well as original article), 78 studies did not evaluate an immunoassay, 24 investigations were not conducted in patients with suspected HIT, 39 articles because of insufficient data for analysis, and the reference standard was not clearly defined in 31 publications. Finally, we included 49 studies comprising 128 test evaluations in 15 199 patients.^{16,19-21,28,36-79} 

Detailed characteristics of all 128 diagnostic accuracy studies are given in supplemental Table 1. Six different classes of assays were evaluated: ELISA (79 evaluation studies),^{16,19-21,28,39-41,43-55,59-61,63-79}  PaGIA (14),^{20,36,42,45,46,53,58,64,65,68,70,72,75,79}  PIFA (1),²⁸  lateral flow immunoassay (9),^{54-56,58,66,70,72}  latex agglutination assay (1),³⁷  and CLIA (24).^{37,38,57,61,62,77}  Available assays, antibody specificities, thresholds, test variations, and manufacturers are illustrated in Table 1. Often, more than 1 evaluation study was reported in a publication (range, 1-13), and more than 1 assay was evaluated in the same study population (range, 1-11). Low thresholds were studied in 96 cases,^{16,19-21,28,36-64,66-79}  intermediate in 21,^{19-21,28,47,61,65,77}  and high in 11.^{19,37,39,47,61,62,77}  Assays targeting polyspecific antibodies were used in 78 studies,^{16,19-21,28,36-39,41-43,45-54,57-65,67-75,77-79}  and IgG-specific antibodies in 49 investigations^{19-21,37,40,44,54-58,61,63,64,66,69-72,77}  (unclear in 1 study⁷⁶ ). A high-dose heparin confirmation step (ELISA) was studied in 9 cases.^{21,40,67,73,76}  The following reference standard tests have been used: SRA (43 evaluation studies),^{19,28,36,38,39,41-43,47,50,53,63,64,67-69,73,74,76}  HIPA (42),^{16,20,21,37,40,48,49,51,52,54-56,70,72}  combination of HIPA/SRA with clinical criteria (10),^46,57-59,78  flow cytometry (12),^44,45,66,77  heparin-induced platelet aggregation test (4),^60,65  and others (17).^{61,62,71,75,79} 

A summary of the methodological quality is shown in Figure 2; the quality of the individual studies is reported in supplemental Figure 1. A low risk of bias in all 4 domains was observed in 2 of 128 evaluation studies only (1.6%).⁶⁸  The median number of low-risk ratings was 1 (25%). A high risk of bias was assessed in 125 cases (24.4%). Most high-risk ratings were assigned in the domain patient selection (57) and the fewest in index test (11). Most low-risk ratings were found in the reference standard domain (90). In contrast, low concerns regarding applicability were assessed in 276 cases (71.9%). Methodological criteria that were frequently inadequately addressed were (1) prospectively enrolled patients, (2) specified cohort criteria, (3) numbers and reasons for excluded patients, (4) interpretation of reference standard without knowledge of the index test, and (5) clearly described sequence of testing.

We were able to calculate diagnostic accuracy measures according to the unified model for 9 classes of assays: polyspecific ELISA with low and intermediate threshold, IgG-specific ELISA with low and intermediate threshold, PaGIA, lateral flow immunoassay, polyspecific CLIA with low and high threshold, and IgG-specific CLIA with low threshold. Detailed results are reported in Table 2; Figure 3 illustrates diagnostic accuracy as a hierarchical summary receiver operating characteristics (HSROC) curve. Among the categories that could be analyzed with the unified model, sensitivity was >95% for polyspecific ELISA with low and intermediate threshold, IgG-specific ELISA with low threshold, PaGIA, lateral flow immunoassay, polyspecific CLIA with low and high threshold, and IgG-specific CLIA with low threshold. Specificity was >90% in polyspecific ELISA with intermediate threshold, IgG-specific ELISA with intermediate threshold, PaGIA, lateral flow immunoassay, polyspecific CLIA with high threshold, and IgG-specific CLIA with low threshold. Both criteria were fulfilled in 5 assay classes (polyspecific ELISA with intermediate threshold, PaGIA, lateral flow immunoassay, polyspecific CLIA with high threshold, IgG-specific CLIA with low threshold). A forest plot was generated to compare diagnostic accuracy measures (Figure 4). To illustrate the diagnostic value of the tests at different pretest probabilities (prevalence or probability according to clinical scoring tools), we show posttest probabilities of positive and negative test results in Table 2 and Figure 5 (assuming constant likelihood ratios).

Because of a limited number of studies, we were unable to calculate diagnostic accuracy measures using the unified model in 9 tests: polyspecific ELISA and IgG-specific ELISA with high threshold, polyspecific ELISA and IgG-specific ELISA with high-dose heparin confirmation step (low and intermediate threshold), PaGIA with intermediate threshold, polyspecific CLIA with intermediate threshold, and IgG-specific CLIA with intermediate and high threshold. To give an impression of the diagnostic accuracy in these tests, we calculated diagnostic accuracy measures using a fixed-effects model (if possible) and report the results in Table 2. Sensitivity was below 95% in all of the previously mentioned tests except polyspecific ELISA with low threshold and high heparin dose confirmation step.

We were able to calculate diagnostic accuracy measures for the individual ELISA manufacturers by means of the unified model in 8 assays and by the use of a fixed-effects model in 2 assays. No pooling was possible in 4 assays with a unique evaluation study. Results are reported in Table 3. In most of the assays, diagnostic accuracy corresponds to the general characteristic of the class (sensitivity above or below 95%; specificity above or below 90%). However, some assays appear to be inferior (eg, IgG-specific Asserachrom assay with a low threshold). Diagnostic accuracy was superior to corresponding class in case of IgG-specific Genetic Testing Institute-ELISA with low threshold (sensitivity, 99.6%, 95% confidence interval [CI] 22.7-100.0; specificity, 89.9%, 95% CI 86.2-92.6).

For the first sensitivity analysis, calculations were repeated in studies with cohort designs only (lower risk of bias). Only minor differences were observed (range, 0.0%-0.5% for sensitivity; 0.0%-3.6% for specificity), the results are shown in supplemental Table 2. In the second sensitivity analysis, data of studies that used SRA/HIPA as a reference standard were pooled only (supplemental Table 5) and compared with results from all studies. Differences ranged between 0.2% and 1.7% for sensitivity and 0.7% and 1.5% for specificity. Third, we compared data of different tests obtained within the same study population using the same reference standard. Supplemental Table 4 shows the results of the 2 populations with the most tests evaluated. The results of the individual studies and the differences between different categories (thresholds, antibody specificity, class of assays) correspond very well with the pooled data. Supplemental Table 3 shows results of a meta-regression analysis exploring the impact of the reference standard used, setting, and specimen material on diagnostic accuracy. No significant results were observed for any of the previously mentioned variables.

In general, neither sensitivity analyses nor meta-regression recognized any characteristic of study design as possible source of heterogeneity and bias.

Important differences exist between classes of immunoassays, type of antibodies, thresholds, and application of confirmation step; only 5 tests appear optimal for clinical use. Relevant variations were found even between manufacturers of ELISA assays. In general, methodological quality of primary studies was moderate. Nevertheless, the results were consistent and no differences were found in sensitivity analyses.

The present investigation is the first systematic review and meta-analysis focusing on immunoassays for diagnosis of HIT. We conducted a comprehensive literature search to retrieve the published evidence, applied strict inclusion criteria, assessed the methodological quality of the studies systematically, and used an appropriate meta-analytic technique to pool the existing data. Thus, we were able to compare diagnostic accuracy measures between classes of assays, manufacturers, antibody specificities, thresholds, and the application of high-dose heparin confirmation step.

Our study has several limitations. First, methodological quality of primary studies was only moderate in general. Even though we cannot fully exclude that this might have influenced our results, we estimate the risk to be low: (1) applying strict inclusion criteria, we excluded 94 publications because of low quality; (2) sensitivity analysis did not show relevant differences if compared with high-quality publications only; and (3) the results were consistent across subgroups. Second, the number of available studies was limited in certain categories of tests, restricting the analysis to a fixed-effects meta-analytic model only (Tables 2 and 3). The results of this analysis must be interpreted with caution because it is not accepted as a valid statistical technique in this situation. The numbers of studies and cases are even more limited in certain categories. For example, the favorable sensitivity of polyspecific ELISA with low threshold and high heparin dose confirmation step largely depends on 1 study in which the only 1 HIT case was detected (corresponding to a sensitivity of 100%).⁶⁷  The number of studies and patients is limited even for the tests most investigated, leading to a restricted precision with wide CIs. For example, expected differences between polyspecific and IgG-specific ELISA in terms of sensitivity and specificity could not be reproduced in every subgroup (Table 2). And third, several different reference standard tests have been used in evaluation studies and we cannot fully preclude that this might have influenced the results of our investigation. However, we expect the risk to be low for several reasons: (1) a sensitivity analysis comparing diagnostic accuracy measures obtained with 1 of the accepted gold standard SRA/HIPA only, and all results did not find any difference; (2) meta-regression analysis revealed no difference in diagnostic odds ratio if SRA/HIPA were compared with other defined reference standard tests; (3) we applied strict inclusion criteria requiring a clear definition of reference standard tests; and (4) in the majority of studies (95/128), either SRA/HIPA or a combination of SRA/HIPA with clinical criteria was used.

One may argue that a systematic review of diagnostic tests for HIT should focus on sensitivities and specificities. Indeed, sensitivity and specificity are well-known (even though often misinterpreted) measures of diagnostic accuracy, and reporting them is essential. However, the important question in clinical practice is how a particular test result predicts disease, and sensitivities/specificities cannot answer this question. Predictive values are often used, but these measures depend on the prevalence of the disease, and generalization beyond a study population is difficult.⁸⁰  Likelihood ratios are powerful diagnostic accuracy measures because they are assumed to be independent from prevalence, they express how many times more (or less) likely a test result is in patients with the disease in contrast to patients without the disease, and they can be used to calculate posttest probabilities. In addition, differences in likelihood ratios between tests can be seen clearly in a forest plot. In contrast, it is hard to read a forest plot with estimates of sensitivities or specificities along with CIs when most data are between 90% and 100% (ceiling effect). Following these considerations, we decided to report all sensitivities/specificities in a table and likelihood ratios as a forest plot.

With the present investigation, we provide aggregated diagnostic accuracy measures for all frequently applied immunoassays used to diagnose HIT. Our results will raise clinicians’ awareness for the characteristics of tests they are confronted with. Using the posttest probabilities provided in Table 2 and Figure 5, clinicians may be able to estimate the probability of HIT in individual patients. If used in combination with clinical scoring systems (which assess the pretest probability), they might substantially improve care in patients with suspected HIT. In addition, laboratory managers can use our results to compare test characteristics directly and base decisions about which test to implement on a sound scientific background.

No antibody test is both 100% sensitive and specific, and such a test is not expected in the near future. Functional assays are rarely available in a timely fashion. Clinical scoring systems such as the 4Ts score could help identifying patients with a low pretest probability and might reduce the number of “overtreated” patients.¹²  However, there are several unsolved issues associated with the 4Ts score.^12-14  Thus, we will always miss a fraction of patients with HIT, or treat a number of non-HIT patients. Both situations are dangerous. A promising concept to overcome these limitations might be a subsequent or parallel determination of different tests and estimation of posttest probabilities using nomograms. We suggest researchers focus on diagnostic algorithms to improve diagnostic procedures in this vulnerable group of patients.

Important differences exist between immunoassays for diagnosis of HIT, in particular with regard to different classes of assays, antibody specificities, manufacturers, thresholds, and the application of the high-dose heparin confirmation step. Only 5 tests have a high sensitivity combined with a high specificity: polyspecific ELISA with an intermediate threshold (Genetic Testing Institute, Asserachrom), PaGIA, lateral flow immunoassay, polyspecific CLIA with a high threshold, and IgG-specific CLIA with a low threshold. Diagnostic accuracy appears to be inadequate in tests with high thresholds (ELISA, IgG-specific CLIA), combination of IgG specificity and intermediate thresholds (ELISA, CLIA), high-dose heparin confirmation step (ELISA), PIFA, and IgG-specific Asserachrom assay at low threshold. Clinicians should be aware of the pretest probability as well as the diagnostic characteristics of the tests they are confronted with. Investigations focused on diagnostic algorithms combining different tests might further improve care in patients with suspected HIT.

We thank Prof Greinacher, Greifswald (Germany) for his valuable comments on an early version of this manuscript.

This study was supported by Research Fund Haematology Luzerner Kantonsspital.

Contribution: M.N. developed the protocol, conducted the literature search, assessed methodological quality, performed data extraction, conducted the statistical analysis, and wrote the manuscript; L.M.B. developed the analysis plan and reviewed the analysis; H.t.C. reviewed the protocol and the manuscript; A.t.C.-H. reviewed the protocol, assessed methodological quality, performed data extraction, and reviewed the manuscript. All authors approved the final version of the manuscript.

Conflict-of-interest disclosure: M.N. has received research grants or lecture fees from Pfizer, Bayer, CSL Behring, Sanofi-Aventis, Roche Diagnostics, and Axonlab. H.t.C. is a Fellow of the Gutenberg Forschungscolleg, Gutenberg University, Mainz, Germany. The remaining authors declare no competing financial interests.

Correspondence: Michael Nagler, Department of Haematology and Central Haematology Laboratory, Inselspital University Hospital, 3010 Bern, Switzerland; e-mail: michael.nagler@insel.ch