The ASH Practice Improvement Module in Non-Hodgkin Lymphoma: Assessing the Feasibility, Reliability and Usefulness of a New Quality Improvement Tool

Background:The American Society of Hematology Practice Improvement Modules (ASH PIMs) are online tools designed for clinicians to monitor the quality of care in their practice. The ASH PIM for non-Hodgkin lymphoma (NHL) was designed by a committee of NHL experts and was recently released in the ASH Academy. The ASH PIM for NHL defines quality metrics in six areas: pathological diagnosis, staging, Hepatitis B testing, use of growth factors, vaccination, and fertility counseling.

Objectives:1) Use the ASH PIM for NHL to measure quality of care at 4 cancer centers in the Greater Toronto Area. 2) Assess the feasibility, reliability, and usefulness of the ASH PIM for NHL.

Methods:To measure quality of care, 78 patients undergoing first line chemotherapy for NHL were reviewed at 4 cancer centers (3 academic centers and 1 community center) near Toronto, Canada. Two hematology fellows independently scored each patient chart for the 6 quality metrics in the ASH PIM. After data collection, interviews (using structured questionnaires) were conducted with the chart reviewers as well as with physicians experienced at treating NHL.

Results: Three out of the 4 cancer centers had high performances (>90%) in pathological diagnosis and staging. Two of the 4 centers had high performances for Hepatitis B testing. Zero of the 4 centers had high performances for documenting growth factor use, vaccinations and fertility counseling.

A feasibility questionnaire revealed that each chart required 15 minutes for review. Reviewers noted that the ASH PIM for NHL was clear for how to score pathological diagnosis, staging, and hepatitis B testing (mean score >4 out of 5 for clarity), but unclear for how to score the use of vaccinations (mean 2.3/5). Reviewers were able to accurately score pathological diagnosis, staging, and hepatitis B (mean >4/5 for perceived accuracy), but were unable to accurately score vaccination and fertility counseling (mean <3/5). Interviews revealed concern that practices around vaccination and fertility counseling were not being well documented in the medical record. Inter-rater reliability was high across all 6 metrics (Gwet’s first order agreement coefficient 0.81 – 0.97).

Of the 6 metrics, experienced NHL physicians rated pathological diagnosis and staging as most important, with vaccination and growth factor use rated as least important. When provided with the performance data for their own center, the results were perceived to accurately reflect patient care in 5 out of 6 of the metrics. Performance on fertility counseling was thought to be under-estimated due to poor documentation. Overall, the ASH PIM for NHL was perceived to capture the quality of patient care moderately well. Areas not captured by the ASH PIM, but perceived to be important to NHL patient care included: the reduction of delays between lymphoma suspicion and first treatment, patient satisfaction, and advanced care planning.

Conclusions: The ASH PIM for NHL is feasible, reliable, and measures a number of important aspects of patient care. Some metrics could be made clearer by including more explicit definitions. Performance on the metrics relying on clinical documentation was not as high as for those relying on objective testing. The performance of cancer centers in the Greater Toronto Area on the ASH PIM for NHL suggests specific areas for quality improvement at each center.