Development of Quality Metrics to Evaluate Pediatric Hematologic Oncology Care in the Outpatient Setting

Introduction:

Systems to quantify and incentivize quality of care (QoC) have been developed in multiple healthcare settings. In pediatric oncology, lists of QoC metrics or recommendations have been procured through consensus methodologies such as the Delphi process. To date, no QoC metrics have been developed for outpatient pediatric oncology.

Objectives:

The aim of this study was to develop a list of QoC metrics for the leukeumia-lymphoma (LL) clinic at the Hospital for Sick Children in Toronto, using a consensus process that could be adapted to other clinic settings.

Methods:

A modified Delphi process following the American Society of Clinical Oncology (ASCO) guidelines was used to generate consensus on a list of QoC metrics (Loblaw et al., 2012). A Medline-Ovid search was conducted for quality indicators, metrics and recommendations relevant to pediatric oncology. Results were screened for (a) system-level metrics that could be translated to a clinic level and (b) clinic-level recommendations that could be converted to measurable quantities. Additional metrics outside the literature search were considered. A provisional list was compiled and circulated electronically to local stakeholders, including medical and nursing staff (n=10). Stakeholders ranked each metric on a 5-point Likert scale based on importance and feasibility of measurement (round 1). Stakeholders provided feedback on the metrics and suggested additional metrics. Median, interquartile range and full ranges were calculated for each metric. A metric was considered to reach consensus if the percent of respondents ranking within two consecutive scores was ≥70%. Results and comments from round 1 were re-circulated to stakeholders in personalized reports. This allowed each stakeholder to compare his or her previous scores with overall scores for each metric. Stakeholders were asked to re-rank each metric (round 2).

Results:

The literature search yielded 2 relevant publications from which a provisional list of 27 metrics was generated. Metrics were grouped into 7 categories (Table 1). In round 1, 19/27 (70%) metrics reached consensus. Stakeholders’ comments resulted in 4 new metrics and edits to 8 original metrics. All metrics were included in round 2 for a total of 31. Twenty-four of 31 (77%) metrics reached consensus after round 2 (Table 1). Thirteen were chosen for the final list based on highest consensus scores, highest interquartile and full ranges, and minimizing redundancy.

Conclusion:

This study demonstrates the feasibility of using a modified Delphi process to generate QoC metrics for a pediatric hematology oncology clinic, and provides a model other clinics may employ for local use. The final metrics will be used to evaluate the quality of care in the LL clinic, and to identify areas for improvement in clinic function.

Figure 1

View largeDownload slide

Figure 1

View largeDownload slide

Close modal