DISPLACE study shows poor quality of transcranial doppler ultrasound for stroke risk screening in sickle cell anemia

Children with sickle cell anemia (SCA) are at significantly increased risk of stroke when compared with their age-based counterparts, with up to an 11% chance of an overt stroke before the age of 20 in the prechronic transfusion era.¹ The Stroke Prevention Trial in Sickle Cell Anemia (STOP), completed between 1995 and 1997, demonstrated that with the use of transcranial Doppler ultrasound (TCD; Sickle Stroke Screen), children at the highest risk for stroke could be identified and started on chronic red cell transfusion therapy (CRCT), thus reducing the risk of stroke by over 90%.² This led to the adoption of the STOP protocol as standard of care, first announced in 1997 by the National Institutes of Health in a clinical alert,³ in which it is recommended that patients with SCA (genotypes HbSS and HbSβ0-thalassemia) between the ages of 2 and 16 years undergo routine yearly TCD screening and patients with abnormal findings should start chronic transfusion therapy. This guideline has been reaffirmed in several subsequent reports from the National Heart, Lung, and Blood Institute (NHLBI)⁴ and the American Society of Hematology (ASH).⁵ 

In defining the use of the TCD for SCA, the STOP protocol required measurement of the time–averaged mean maximum velocity (TAMMV) in the distal internal carotid artery (dICA) and proximal middle cerebral artery (MCA) with the following classifications: “normal,” if all TAMMV are <170 cm/sec; “conditional,” if there is at least 1 TAMMV of 170 to 199 cm/sec; and “abnormal,” if there is at least 1 TAMMV ≥200 cm/sec.² CRCT to prevent stroke is indicated for abnormal TCD on 2 occasions or 1 TCD with TAMMV of ≥220 cm/sec. Now, more than 20 years after the publication of these findings, implementation of TCD is inconsistent across the United States and lacks standardization. The NHLBI recommended that facilities “do studies to compare their current equipment with STOP trial TCD equipment.”³ To meet the need for training in correct TCD use, the STOP investigators provided national trainings and workshops; however, these are no longer available. A standard nonimaging TCD as used in STOP was not available at all centers, therefore, some centers started to use the transcranial Doppler imaging (TCDi) technique. An early study assessing for differences between TCD and TCDi velocities published in 2001, demonstrated for the MCA that, TCDi velocities were about 10% lower than those measured with TCD,⁶ which has subsequently been confirmed in some studies^7,8 but not in others.^9-11 In fact, the French National Authority for Health recommended use of the same thresholds for TCD and TCDi due to concern for potential over-transfusion in patients screened by TCDi.¹² In some centers, angle correction, or adjusting the velocity based on the angle between the transducer and the vessel, is performed when using TCDi.¹³ The precise correlation between TCD and TCDi velocities is not clear and likely highly dependent on technique. The ASH guidelines published in 2020 continue to support the cutoff values defined in the initial STOP trial, but added recommendations for TCDi, citing the velocity used in the Silent Cerebral Infarct Transfusion (SIT) trial: mean velocity ≥185cm/sec is abnormal.^5,14 

Since the routine use of TCD for stroke risk screening in SCA was instituted in 1997, there have been several studies evaluating site-level adherence to the recommendation to obtain annual TCD assessments.^15-17 These studies used a variety of techniques for both examining barriers to TCD and facilitating improvement, including the use of personalized reminders¹⁸ and tracking patients overdue for imaging.¹⁹ One European study identified a major barrier to routine TCD screening, which was the lack of trained personnel to perform the procedure. To overcome this barrier, they recruited a variety of practitioners, including clinicians with ultrasound experience, surgeons, pediatricians, and nurses, from 3 centers to complete a TCD/TCDi training program. They noted that before this training, there was significant variation in the percentage of scans classified as abnormal among the institutions, while following training, there were no differences in the distribution of classifications.²⁰ Each of these studies reports on barriers to appropriate and accurate stroke risk screening for children with SCA. However, none of these studies specifically addressed the quality of the reports themselves, assessed for the correct interpretation of measurements, or insured ongoing quality assurance. Only 1 study assessed the accuracy of the measurements themselves²⁰ and no recent reports have demonstrated multi-institutional assessments of TCD quality.

The recent DISPLACE (Dissemination and Implementation of Stroke Prevention Looking at the Care Environment) study was a 28-site consortium funded by the NHLBI to evaluate barriers to TCD screening implementation and test strategies to improve adherence to TCD guidelines in SCA in the US. DISPLACE demonstrated that less than 50% of children at participating SCA centers had annual TCD screening during the baseline period (2012-2016).²¹ This current project is a substudy using the DISPLACE database. In this study, we hypothesized that there would be both high variability in TCD/TCDi technique and that sites using TCDi would use inconsistent definitions to classify scans as “abnormal.” In addition, we wanted to determine if sites using TCDi were making treatment decisions regarding chronic transfusion therapy based on criteria other than those established by the STOP protocol.

The DISPLACE study has been described previously.^21,22 Briefly, DISPLACE was a dissemination and implementation study performed to improve TCD stroke risk screening in children with sickle cell anemia in the US. The initial phase of the study was an in-depth retrospective chart review that required each participating site to identify all children with SCA treated at their sites from 2012 to 2016 and upload multiple laboratory and radiographic reports (including TCDs) from each child from all available years to determine site-level adherence to TCD screening. These results from Part 1 demonstrated that more than 50% of children with SCA were not getting appropriate TCD screening and also identified updated findings regarding the decreased frequency of abnormal TCDs that coincided with the increased early initiation of hydroxyurea.²¹ During Part 1, over 12 443 TCD reports were collected from 28 institutions and uploaded into a customized database. An institutional review board (IRB) waiver was obtained and data from these reports were used for this DISPLACE substudy.

To facilitate evaluation, a computer-generated algorithm was used to randomly select 400 TCD/TCDi reports for this substudy. The algorithm ensured that reports were included from all 28 sites across all different years and patient age groups. The initial hypothesis of this substudy was that there would be increased variability in the interpretation of TCDi when compared with TCD. Consequently, it was determined that for an alpha of 0.05, ∼400 reports would be needed, for a 95% confidence interval with the precision of 0.1, assuming a sample proportion of 0.5 (most conservative assumption).

Data were manually extracted from the TCD/TCDi reports for patients aged 2 to 8 years at the time of their study. This age group was targeted, given the highest prevalence of stroke for patients with SCA is in the first decade of life, ²³ in addition to the previous DISPLACE data showing the highest rate of first abnormal TCD in the 4 to 8 year age range.²¹ Data collected from each report included the institution, year, blood vessel(s) assessed, whether numerical values for TCD velocities were recorded, the presence of low values, if peak systolic velocities were assessed, the interpretation of the TCD study, if the interpretation was based on TAMMV or another measurement, and if follow-up recommendations were provided in the report. Because most reports did not specifically state whether TCD or TCDi was used, these data were separately extracted from the information manually entered by each participating site directly into the DISPLACE study database. K.A.D. and R.E.M. reviewed these data.

Additionally, a REDCap²⁴ survey (version 11.2.1) of DISPLACE study principal investigators (PIs) was conducted. PIs were asked to recall information from the study period (2012-2016). Questions included the use of TCD vs TCDi, the type of machine, how technicians were trained, who reads the TCDs, velocity cutoffs used for classification of normal, abnormal or conditional, and vessels included in interpretation. A similar survey had been previously performed,²⁵ but was deidentified, so answers could not be directly linked to a specific institution’s TCD reports.

Microsoft Excel was used for statistics. Counts and frequencies were calculated for categorical variables. Measures of central tendency were calculated for continuous variables.

An IRB waiver from the Nemours Children's Hospital IRB was obtained.

A total of 391 TCD reports were reviewed from 26 different institutions (28 were included in the DISPLACE study; however, 2 sites did not upload usable reports). Due to variability in the number of reports uploaded to the DISPLACE database by each institution, the number of reports reviewed for this substudy from each institution varied. Within this subcohort of 391 reports (9 uploaded reports were outside of the age range or were not TCD reports), the median age of patients at the time of their TCD was 5 years (range, 2-8 years) and the median year the studies were completed was 2013 (range, 2000-2016). The reports evaluated included both TCD and TCDi (47%, 183/391 and 53%, 207/391 respectively), which is in a different proportion than the entire DISPLACE study (66.2% TCD vs 32.8% TCDi).²¹ This difference was intentional because the goal for this substudy was to compare equal numbers of TCD and TCDi reports.

After the initial review of the TCD and TCDi reports, there was such substantial variation across all institutions (in the content of both TCD and TCDi reports) that a conclusion regarding how the reports were categorized as abnormal, could not be determined. Instead, the decision was made to focus the analyses on the quality and completeness of the information included in the reports, the accurate and consistent description of what measurements were taken, and the interpretation of the results. The majority of TCD/TCDi reports were classified as normal (67%, 262/391) with 13% (52/391) conditional and 4% (14/391) abnormal, consistent with the original findings from the DISPLACE Part 1 cohort. In addition, 1% (5/391) were documented as inadequate and 15% (58/391) were unclassified or not interpreted using STOP-defined terminology. Further information regarding the TCD reports is summarized in Table 1.

A review of these 391 TCD reports identified deficiencies in reporting TCD modality (TCD vs TCDi), technique, and vessels examined. Upon data entry into the DISPLACE study database, sites were required to select if a report represented a TCD or TCDi. However, of those that were listed as being performed with TCDi, 96% (199/207) did not state that a TCDi was used in the radiology report itself. Second, of the 391 TCD/TCDi reports reviewed, 6% (24/391) of the reports did not include numerical velocities, whereas 8% (32/391) had some numerical values but not for all vessels that the body of the report listed as being evaluated. Furthermore, over half of the reports (52%, 200/391) did not clearly identify the velocity measurement as the TAMMV, the key variable per STOP criteria. About 30% (116/391) of the reports did not assess or did not report on the dICA velocity.

There were also deficiencies in reporting study interpretation clearly. Only 42% (162/391) of the reports clearly stated that the classification of normal vs abnormal results was based on the TAMMV, and only 32% of the reports (123/391) clearly reported which vessels were used to classify the study as normal/abnormal. Notably, some reports included verbiage saying classification was based on STOP criteria but did not give sufficient details in the report to determine if the correct vessels were used.

The surveys to assess the TCD screening practices from 2012 to 2016 are summarized in Table 2. Responses were received from 23 of the 26 institutions for whom TCD reports were included. About 57% (13/23) reported using TCDi and 1 institution reported using both TCD and TCDi. Of those sites using TCDi, 43% (6/14) used angle correction. There was substantial variability regarding which vessels were examined for both TCD and TCDi, ranging from 1 to 5 vessels per hemisphere, with 3 institutions reporting that it was “whatever the radiologist decides.” Radiologists interpreted TCDs at most sites (70%) and neurologists and hematologists were identified as the responsible physicians in the remainder. When asked how ultrasound technicians were trained in TCD assessment, 39% (9/23 sites) noted using peer-to-peer training, whereas 35% (8 sites) underwent formal STOP training for at least some of their technicians. Further details are provided in Table 2.

As part of this survey, site PIs were also asked what velocity measurements were used to classify TCD/TCDi scans as normal/abnormal/conditional. Of those using nonimaging TCDs, the classifications were consistent with those used for the STOP study for 80% (8/10) of respondents. Five of the 10 institutions also had defined “low” values that were not included as a STOP study outcome measure. For the 14 sites using TCDi, there was substantial variability in the lower velocity classification used to define a “conditional” result which varied from 150 to 170 cm/sec, and for the lower limit velocity for “abnormal” ranging from 180 to 201 cm/sec.

The STOP study revolutionized care for pediatric patients with SCA, identifying a noninvasive method to monitor children at high risk of stroke and identifying a life-saving intervention (CRCT).² However, despite this significant finding, several subsequent studies have shown poor implementation of TCD screening for a variety of reasons, including missed opportunities for referral and inconsistent technique.^16,20 Furthermore, there is no consistent means of ensuring that centers perform TCDs accurately. Only 1 European study performed a quality assessment of the technical capabilities of people performing stroke risk screening using TCD.²⁰ This study initially sought to evaluate the quality of TCD reports across multiple institutions in the United States, with the hypothesis that there would be more variability in the interpretation of TCDi than TCDs. Instead, this study identified a startling lack of standardization across all sites including which vessels were measured, how the measurements were performed (with or without angle correction for TCDi), how the reports were interpreted, and who was reading and interpreting the report. The reports were so discrepant that it was not possible to assess whether measurements were accurately interpreted.

TCD results have critical implications for medical decision making for children with SCA and our study reveals a significant lack of quality assurance and consistency in the reporting of stroke risk screening, using both TCD and TCDi. Most notably, many reports did not clearly identify which blood vessels were used to assess stroke risk, did not report on the correct velocity measurement (or lacked the information to determine if the correct velocity was measured), or made interpretations based on velocities other than TAMMV, the measurement validated in the STOP trial. Additionally, most reports reviewed in this study did not make note of whether a TCD or TCDi was performed, and for TCDi, it was frequently unclear if angle correction was performed which can affect cutoffs for normal, conditional, and abnormal. Corroborating these findings, survey data from the participating DISPLACE sites showed significant variation in the training methods used for those performing TCD. Though radiologists were the most common physicians interpreting results, some sites noted neurologists or hematologists assuming this role. The training of the interpreting physician was not assessed.

This study revealed a significant lack of quality assurance in TCD technique and interpretation across 26 pediatric SCA centers in the United States, suggesting variability of result quality that could result in missed opportunities to prevent strokes. To address this, we have several recommendations for improved clarity of TCD reporting at pediatric SCA centers. First, we recommend the creation of a standardized template for TCD/TCDi reports to be used across institutions which includes the following key data: 1) specific type of TCD (TCD or TCDi) being used, 2) defined measurement (TAMMV, also abbreviated as TAMMX) for each vessel examined (MCA, dICA, anterior cerebral artery, etc), 3) numerical values noted for the TAMMV of each vessel that is assessed, 4) a clear impression statement indicating whether the TCD/TCDi is normal, abnormal, or conditional, with a clear definition of what values were used to categorize the results as such, and 5) a statement regarding the adequacy of the study. We additionally recommend consideration for the inclusion of comments about asymmetry²⁶ and low values.²⁷ Because there is some evidence to suggest utility in measuring peak systolic velocities,²⁸ for sites measuring them we recommend inclusion in the body of the report but exclusion from the impression to prevent misinterpretation. Similarly, the assessment of additional vessels such as the anterior cerebral artery²⁹ or the external portion of the ICA (eICA)^30,31 deserves further study. The format of these reports should be standardized in all electronic health records using a standardized data dictionary (as used by the National Alliance of Sickle Cell Centers) to facilitate both longitudinal assessment of individual patients and intra- and intercenter comparative quality assessment. Data should also be entered into the electronic health record in an easily extractable format to facilitate multi-institution reviews. Our recommended standardized template is provided in Figure 1. The standardization of measurements and methods used for interpretations is critical to allow comparison of data for use in research to advance the field, especially as novel medications and transformative therapies are being developed.

Regarding TCDi measurements, it has been shown in several studies that one can expect the velocities to be about 10% lower than TCD. However, this has not been confirmed, and thus, there are no established guidelines regarding interpretation. Sites using TCDi varied significantly in their definitions of conditional and abnormal velocities with or without angle correction. In the US, we recommend compliance with the ASH guidelines for TCDi, using a TAMMV ≥185 cm/sec to define abnormal⁵ until further studies or internal quality assurance suggests another threshold should be used as discussed below.

We also recommend choosing 1 brand of TCD or TCDi that is used for all patients with SCA being cared for at an institution, to allow consistency within the institution as well as internal quality assurance. Most ideally, the same machine would be used for all patients undergoing TCD/TCDi for stroke risk screening at an institution, given the variability even among different equipment from the same manufacturer. While calibration of each machine for the performance of stroke risk screening for children with SCA would be optimal, there is not a recognized, guideline-approved method for this practice at this time. The equipment should undergo recalibration as recommended by the manufacturer to maintain the best precision. Triggers to evaluate the process and thresholds used within an institution could include a lack of abnormal scans (as this suggests the threshold for abnormal may be too high), a change in the percentage of abnormal results, and a clinical review of every patient who has a stroke that had undergone a TCD. The percentage of abnormal scans should be compared continuously both within and between SCA centers as the rates of abnormal TCD and frequency of stroke have changed as treatments have evolved. Hematologists, sonographers, and radiologists at SCA centers should work together to evaluate their practices and outcomes to improve local performance and outcomes.

Finally, without successful and ongoing TCD/TCDi training for technologists and interpreting physicians, results from these scans may be unreliable. Though some institutions in DISPLACE reported having technologists trained in formal STOP training courses, most training is peer to peer and currently, there are no formal sickle cell–specific training opportunities. The authors feel these trainings should be reintroduced by either the National Alliance of Sickle Cell Centers or through a professional diagnostic medical sonography organization. A standardized training curriculum would improve both the technical performance of TCD/TCDi and its interpretation. Physicians interpreting TCD/TCDi should have specialized training in the use of TCD/TCDi for stroke risk screening. Hematology organizations and radiology organizations should work together to improve mutual understanding of TCD performance standards and their critical clinical significance. Recommendations are summarized in Table 3.

Although this study is multi-institutional and included the review of nearly 400 TCD reports, there are still limitations. The sites included in DISPLACE represent ∼30% of all pediatric SCA centers across the United States and did not include international centers. The TCDs included were performed between 2000 and 2016, and institutional practices may have changed since these were completed; though notably, there were no relevant differences in the quality of reports from the early time points compared with the later ones. When surveys were sent to DISPLACE PIs, they were requested to recall what their institution was doing during the study period (2012-2016); however, recall bias may have impacted the results toward reporting what the facilities are currently using. Evidence against recall bias, however, is previously published; original deidentified survey results from DISPLACE similarly showed that there was significant variation in cutoffs used for TCDi and vessels used in classification.²⁵ 

In conclusion, though the STOP study clearly defined the importance of annual stroke risk screening with TCD for children with SCA, there continue to be barriers to implementation. Appropriate interpretation relies on accurate and consistent study performance, and continuous quality assessment is necessary. Consequently, this study serves as a call to action for immediate recalibration of TCD/TCDi assessment and reporting, including the need for standardized templates for electronic health records, reinstatement of formal training for both those performing and interpreting scans, and ongoing comparative quality analysis. We also recommend the inclusion of TCD quality assurance in future definitions of Pediatric Sickle Cell Disease Centers of Excellence, by the National Alliance of Sickle Cell Centers and consideration for inclusion of TCD–related quality assessment in US News and World Report rankings to incentivize institutions to invest in improvement. Accomplishing these goals will require engagement of the relevant stakeholders, identification of barriers to implementation, and funding. With these interventions, we can continue to work toward meeting the recommended screening and interventions instituted by the NHLBI over 25 years ago and optimize outcomes.

The DISPLACE study is funded by the National Heart, Lung, and Blood Institute (5R01HL133896-05).

Contribution: K.A.D., R.E.M., S.M.P., M.M., and J.K. contributed to the design of the study; K.A.D. and R.E.M. reviewed the data; K.A.D. drafted the manuscript; and all authors contributed to the discussion points and recommendations, reviewed the manuscript, and approved it in its final form.

Conflict-of-interest disclosure: M.L.H. is a consultant for bluebird bio; her institution receives research funding from Novo Nordisk; is on the scientific advisory board for Pfizer; and her spouse is employed by Pfizer. L.H. is a consultant for Hilton Publishing, Abt, and Aruvant, and his institution receives research funding from NHLBI, Health Resources and Services Administration, Asklepion Pharmaceuticals, and Vertex. R.J.A. is a consultant for Pfizer and Novo Nordisk. J.K. is a consultant for Guide Point Global, Gerson Lehrman Group, Novartis, bluebird bio, Fulcrum, GlaxoSmithKline, Ecor1, and Vertex; receives research funding from NHLBI, Health Resources and Services Administration, and the Centers for Disease Control and Prevention; and is a member of scientific advisory committees for Novartis, Oric, Bausch, and Glycomimetics. The remaining authors declare no competing financial interests.

Correspondence: Kimberly Davidow, Lisa Dean Moseley Foundation Institute for Cancer and Blood Disorders, Nemours Children’s Hospital, Delaware, 1600 Rockland Rd, Wilmington, DE 19803; email: kimberly.davidow@nemours.org.