Comparison of self-report and electronic monitoring of 6MP intake in childhood ALL: a Children’s Oncology Group study

Children with acute lymphoblastic leukemia (ALL) require adequate exposure to oral 6-mercaptopurine (6MP) during the maintenance phase of therapy to sustain durable remissions.¹  Prior studies of adherence to oral 6MP in children with ALL have reported adherence rates ranging from 70% to 95% by using a variety of subjective and objective measures.^2-9  We have previously shown that inadequate systemic exposure to 6MP because of poor 6MP adherence (<95% adherence rate, objectively measured by the Medication Event Monitoring System [MEMS; WestRock Healthcare, Sion, Switzerland]) is associated with increased risk of relapse.^2,10  Accurate assessment of 6MP intake is therefore critical to ensure timely interventions for patients with poor adherence.

Self-report is a convenient and inexpensive method for monitoring 6MP intake in the clinic, but literature in the non-oncology setting indicates that self-report is subject to overreporting.^11-14  The accuracy of self-reported 6MP intake during maintenance therapy for childhood ALL is not known. In this study, we address this issue by directly comparing self-report to electronic monitoring of 6MP intake and identifying predictors of overreporting of 6MP intake in a racially and geographically diverse cohort of children with ALL during the maintenance phase of therapy.

Participants were enrolled on the Children’s Oncology Group (COG) Study to Assess Compliance With Long-Term Mercaptopurine Treatment in Young Patients With Acute Lymphoblastic Leukemia in Remission (NCT00268528; AALL03N1) by 87 participating institutions (supplemental Table 1, available on the Blood Web site) after obtaining approval for the study from local institutional review boards. Written informed consent or assent was obtained from all patients and/or parents or legal guardians. Patients and their parents or caregivers did not receive incentives for study participation. Eligibility criteria included a diagnosis of ALL at age 21 years or younger, being in first remission, belonging to 1 of 4 self-reported racial/ethnic groups (Asian, African American, Hispanic, and non-Hispanic white), and receiving maintenance chemotherapy that included oral 6MP administered by the patient or by the parent or caregiver. Although the participation rate of study participants across the 87 participating institutions is not available, we have previously shown that the AALL03N1 study participants were comparable to patients enrolled on the relevant parent COG therapeutic first-line ALL protocols.¹⁵  Only patients who had both evaluable MEMS and self-report data were included in this analysis.

Self-report of 6MP intake was assessed at 4 study time points (day 29, study month 1; day 57, study month 2; day 113, study month 4; and day 141, study month 5; see study schema [Figure 1]) by using a questionnaire that elicited the number of days that 6MP was taken during the past 4 weeks (ie, “During the past 28 days [4 weeks], how many days did you [your child] take 6MP? [Please fill in the blank]: I [My child] took 6MP on _ days of the past 28 days.”) For the patients for whom the reported number of days of 6MP intake was <28, the questionnaire did not elicit the number of days that 6MP was held for physician-directed holds or for patient- or parent-directed reasons. Thus, the self-report questionnaire assessed the number of days of 6MP intake (irrespective of reasons for 6MP holds, if any) rather than adherence to the prescribed regimen. The questionnaire was available in English, Spanish, and several Asian languages, and was completed by parents of patients younger than 12 years of age, by both parents and patients for patients between the ages of 12 and 17 years, and by patients 18 years of age or older at study participation.

6MP intake (irrespective of reasons for 6MP holds, if any) was monitored electronically by placing a MEMS TrackCap on the patient’s 6MP bottle. The MEMS TrackCap uses microelectronic technology to record the date/time of each bottle opening for the duration of the study (Figure 2). Patients and parents were informed about the purpose of MEMS-based assessment and were instructed to take all 6MP doses from the MEMS bottle throughout the study period. At the end of the study period, the MEMS data were downloaded (Figure 3).

Patients (18 years of age or older at study) or parents (of patients <18 years of age at study) completed a demographic questionnaire at study entry, providing information regarding patient race/ethnicity, parental education, and annual household income.

Monthly reports were completed by participating institutions for each patient, detailing the prescribed 6MP dose for each day of the preceding month and any dates when 6MP was held for toxicity or illness. This information was used to calculate 6MP dose intensity and the MEMS-based adherence rate for each patient. 6MP dose intensity was defined as the ratio of the 6MP dose actually prescribed to the planned protocol dose (75 mg/m²/d). The MEMS-based adherence rate was calculated as the ratio of the number of days with MEMS cap openings (X) to the number of days 6MP was prescribed (N), reported as a percentage (X/N × 100). Days when 6MP was withheld by the prescriber were removed from the denominator (N).

The electronic (MEMS) record of 6MP intake was compared with the self-report record for each of the 4 study months that elicited self-reported 6MP intake. For patients age 12 to 17 years, from whom both patient- and parent-reported 6MP intakes were collected, only patient report was used in the analysis, because the patients’ and parents’ reporting were found to be highly correlated (supplemental Table 2). Correlation between electronically monitored and self-reported days of 6MP intake was assessed by using the Pearson correlation coefficient¹⁶  and interpreted by Cohen’s convention.¹⁷ 

Mean days of 6MP intake (by self-report and electronic monitoring) over the study period were compared by using generalized estimating equation (GEE) analysis,¹⁸  adjusted for covariates. The covariates considered included sex, race/ethnicity, age at study entry, annual household income, maternal and paternal education, National Cancer Institute (NCI) risk group,¹⁹  6MP dose intensity, red cell thioguanine nucleotide (TGN) levels,²⁰  and thiopurine methyltransferase (TPMT) genotype.²¹  Backward step-wise procedure was used to eliminate nonsignificant variables until a parsimonious model consisting of variables with P < .1 was obtained.

Patients were classified as perfect reporters if their self-report matched their electronic records for each study month, overreporters if their self-reported days of 6MP intake exceeded their electronic record by 5 or more days for at least half of the study months, and underreporters if the number of days of self-reported 6MP intake was less than their electronic record in all study months. The remaining patients in the cohort were classified as others.

Multivariable logistic regression analysis was used to identify predictors of the overreporter phenotype. The following variables were examined using univariable analysis and were included in the multivariable model if their univariable P value was < .1: age at study entry, sex, race/ethnicity, annual household income, maternal and paternal education, NCI risk classification, 6MP dose intensity, red cell TGN levels, and 6MP nonadherence (MEMS-based adherence rate <95%).² TPMT genotype was retained in the model regardless of P value to account for frequent dosing fluctuations that may occur in non–wild-type patients.

MEMS-based 6MP intake was considered the gold standard (ie, indicator of true 6MP intake). By using electronically monitored 6MP intake records, patients were dichotomized into those with 6MP bottle openings on <95% of study days and those with 6MP bottle openings on ≥95% of study days (based on our previously reported findings that 6MP adherence rates <95% were associated with a significantly increased risk of relapse).²  The sensitivity of self-report in detecting true (ie, MEMS-based) 6MP intake <95% was defined as the proportion of patients with self-report <95% among the patients with MEMS report <95%. The specificity of self-report was defined as the proportion of patients with self-report ≥95% among the patients with MEMS report ≥95%.

PROCs CORR, GENMOD, and LOGISTIC modules of SAS software, version 9.4 (SAS Institute, Cary, NC) were used for analysis. Two-sided tests with P < .05 were considered statistically significant.

The cohort consisted of 416 patients who contributed a total of 1344 patient-months of self-report and MEMS data for this study that were collected over 4 study months per patient. Of the 416 study participants, 412 completed the study (99% retention rate). The clinical and sociodemographic characteristics of the cohort are summarized in Table 1. Median age at study participation was 6.0 years (range, 2-20 years), 66.6% were male, 35.6% were non-Hispanic white, 37.0% were Hispanic, 13.5% were Asian, 13.9% were African American, and 38.4% had high-risk disease by NCI criteria.

Overall, the cohort members self-reported taking 6MP for 92.6% of the total days of observation, whereas the MEMS cap records indicated 6MP bottle openings on 83.7% of the days. Correlation between the mean number of days of 6MP intake by self-report and electronic monitoring by study month was moderate (r, 0.36; 95% confidence interval [CI], 0.27-0.45; P < .001 to r, 0.58; 95% CI, 0.50-0.66; P < .001). GEE estimates of adjusted mean ± standard deviation days per month of 6MP intake for the 4 study months by self-report vs electronic monitoring were 25.8 ± 5.4 to 26.3 ± 3.7 vs 22.8 ± 6.6 to 25.3 ± 4.4, respectively (P < .001; Figure 4).

Comparing self-report to electronic monitoring for each patient, we found that 12.0% (50 of 416) of the patients were perfect reporters (self-report was the same as MEMS across all study months); 23.6% (98 of 416) of the patients were overreporters (self-report exceeded electronic monitoring by 5 or more days for half or more of the study months); 0.5% of the patients (2 of 416) were underreporters (self-report was less than MEMS across all study months), and the remaining 63.9% (266 of 416) of the patients were placed in the other category. For 95.1% (253 of 266) of the patients classified as other, self-reported 6MP intake exceeded electronic monitoring by 1 or more days in 1 or more study months. Thus, for 84.4% (351 of 416) of the entire cohort, self-report of 6MP intake exceeded the MEMS report at least some of the time.

Multivariable logistic regression modeling (adjusted for age at study entry, annual household income, NCI risk classification, 6MP dose intensity, and TPMT genotype) identified the following predictors of the overreporter phenotype (comparison group is all others): non-white race: Hispanic, OR, 2.4; 95% CI, 1.1 to 5.1; P = .02; Asian, OR, 3.1; 95% CI, 1.2 to 8.3; P = .02; African American, OR, 5.4; 95% CI, 2.3 to 12.8; P < .001 (referent group is non-Hispanic whites); paternal education less than college: OR, 1.4; 95% CI, 1.0 to 2.0; P = .05 (referent group is paternal education of college degree or higher); and 6MP nonadherence: OR, 9.4; 95% CI, 5.1 to 17.5; P < .001 (referent group is 6MP adherers) (Table 2). Although 77 (78.6%) of 98 of the overreporters had MEMS-based adherence rates <95% (nonadherers), none of the perfect reporters were nonadherers (P < .001). In the subset of 77 nonadherent patients who were overreporters, there was a negative correlation between mean adherence rate and number of days of overreporting (r, −0.81; P < .001).

GEE estimates of adjusted mean days of 6MP intake over the 4 study months by self-report vs electronic monitoring for the cohort are shown in Figure 5A-C, with findings from the analyses stratified by race/ethnicity (Hispanic, Asian, African American, and non-Hispanic white), by paternal education (less than a college degree and college degree or higher) and by adherence (nonadherers and adherers).

The sensitivity of self-report for detecting true (ie, MEMS-based) 6MP intake <95% was 52.7% (95% exact binomial CI, 46.8%-58.5%). The specificity of self-report for detecting MEMS-based intake ≥95% was 95.8% (95% exact binomial CI, 90.5%-98.6%). When the cohort was stratified by adherence status, sensitivity of self-report for detecting true 6MP intake was 61.8% among adherers and 45.4% among nonadherers.

To the best of our knowledge, this is the first study to directly compare self-report of 6MP intake with electronic monitoring in children with ALL during the maintenance phase of therapy. In this multiracial COG cohort drawn from 87 geographically diverse institutions, we found that overreporting of 6MP intake is common. The large majority (84.4%) of study participants self-reported 6MP intake in excess of the electronic report at least some of the time, and almost one-quarter (23.6%) of participants overreported 6MP intake by 5 or more days in half or more of the study months. We found that there was only modest correlation between self-report and electronic monitoring. Furthermore, nonadherers were more likely to overreport 6MP intake (47%) compared with adherers (8%).

Our finding that oral 6MP intake is overreported compared with electronically measured 6MP intake (92.6% by self-report vs 83.7% by electronic monitoring) during the maintenance phase of therapy in children with ALL is consistent with the literature in other pediatric chronic illness populations who require ongoing treatment with prescribed medications at home. Some examples include pediatric inflammatory bowel disease (intake of prescribed oral 6MP/azathioprine 90% by parent or patient interview vs 36% by pill count),¹¹  adolescent bariatric surgery (postoperative oral multivitamin 88.4% by self-report vs 37.4% by electronic monitoring),¹²  HIV-infected children or youth (antiretroviral medication 100% by self-report vs 75.4% by electronic monitoring),¹³  and pediatric asthma (inhaled steroid 80% by self-report or parent report vs 50% by electronic monitoring).¹⁴  Previous reports in pediatric patients with cancer have not directly compared self-report with electronic monitoring; however, reports of electronically monitored adherence rates have ranged from 71% to 95%,^2-4,9,10  whereas a single study that assessed adherence by a one-time self-reported interview reported an adherence rate of 70%.⁶ 

We found that patients of minority race/ethnicity and those from households with lower paternal education were more likely to overreport 6MP intake. This aligns with our previous studies, in which we found that children with ALL of Asian, Hispanic, or African American ancestry, as well as non-Hispanic white children from households with lower parental education were more likely to be nonadherent to the 6MP regimen.^2,10  Our findings are also consistent with reports in other pediatric chronic illness populations that have shown minority race/ethnicity and lower parental educational level to be factors associated with poor medication adherence.^22-24 

Use of self-report to assess medication intake is simple and inexpensive, and for this reason is often used in clinical settings. Conversely, the more sophisticated measures of medication adherence used in clinical trials (eg, MEMS, drug assays, prescription refill records) may not be practical or readily available to the clinician in real time.²⁵  However, despite the simplicity and convenience of self-report for assessing medication adherence in the clinical setting, our findings suggest that self-report may not be a reliable measure, and our findings are similar to those of other studies examining this issue in non-oncology settings.^26-29  Use of self-report to monitor 6MP intake may erroneously lead clinicians to believe that patients are doing better at taking their 6MP than they actually are.

We found that the sensitivity of self-reported 6MP intake was low (52.7%) but specificity was high (95.8%) during maintenance therapy for ALL. These findings are similar to studies focusing on other chronic illness populations, in which self-report was associated with low sensitivity (ie, patients who do not self-report their lack of intake of prescribed medications may go unrecognized) and high specificity (ie, patients who self-report not taking prescribed medications are generally not taking them),^29,30  and self-report was poorly correlated with objective measures of medication intake.^29,31,32  In this study, we found that only 12% of patients had perfect self-report records (ie, self-report and MEMS records matched exactly); thus, 88% of patients inaccurately reported their 6MP intake, despite the fact that they knew that their 6MP intake was being electronically monitored throughout the study. We found that the large majority of the inaccurate reporting was over- rather than underestimation of 6MP intake; and of particular concern was the finding that nonadherent patients were 9.4 times more likely to substantially overreport their 6MP intake (ie, by 5 or more days in ≥50% of study months) when compared with adherent patients; thus, it may be extremely difficult for clinicians to discriminate adherent from nonadherent patients among those who report no or few missed doses.

This study needs to be considered in the context of its limitations. Although MEMS is considered the gold standard for objective monitoring of medication intake,³³  this electronic monitoring system cannot determine whether the child actually swallowed their medication; we included red cell TGN levels in our models as an additional measure of chronic 6MP exposure. Self-report is also subject to bias resulting from parent or patient perception regarding responses considered acceptable or pleasing to the clinician (ie, social desirability bias).³⁴  This study was designed such that the self-report was collected as part of a research questionnaire rather than directly by the patient’s clinician; thus, social desirability bias may have played less of a role in this study than is typically seen in direct patient-clinician interactions. Nevertheless, it is possible that participants may have wished to please the study staff by providing favorable answers regarding adherence on the questionnaires, and thus the potential for social desirability bias must still be considered. In addition, participants were asked to report their (or their child’s) 6MP intake over the past 28 days, and thus it is possible that there was some recall bias. We also informed patients and parents that we were specifically monitoring adherence to 6MP, and it is possible that this knowledge could have altered their usual medication-taking behavior; however, previous research has shown that health behaviors tend to return to baseline shortly after initiation of monitoring,³⁵  and we followed patients over several months to account for this. Despite these limitations, this study had many strengths, including its longitudinal prospective design, the large and diverse cohort assembled across 87 institutions, and the collection of 1,344 patient-months of both subjective and objective data regarding 6MP intake in the cohort, which allowed for a determination of the relation between self-report and electronic (MEMS) monitoring of 6MP intake in children with ALL during maintenance therapy.

In conclusion, we found that overreporting of 6MP intake during maintenance therapy for childhood ALL is common, particularly in nonadherent patients. Furthermore, given that we have previously shown that 6MP nonadherence (as measured by MEMS) during maintenance therapy is associated with a significantly increased risk of relapse,^2,10  the poor sensitivity (52.7%) of self-reported 6MP intake is cause for concern, particularly since self-report is commonly used for adherence assessment in clinical settings. Because accurate assessment of 6MP intake is crucial for identifying nonadherent patients and ensuring timely intervention, our findings suggest that alternate methods for identifying nonadherent patients in the clinical setting are needed. Therefore, we are currently developing a prediction tool to help clinicians identify patients who are at increased risk for 6MP nonadherence, such that interventions can be targeted to the most vulnerable patients.

This study was supported in part by the Children's Oncology Group and National Institutes of Health grants from the National Cancer Institute (R01CA096670, U10CA098543, U10CA098413, U10CA095861, U10CA180886, U10CA180899, UG1CA189955, and P30CA21765) and National Institute of General Medical Sciences (P50GM115279).

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Contribution: S.B., W.L., and M.V.R. conceived and designed the study; S.B., W.L., L.H., B.C.B., J.N.C., D.S.D., K.W.M., L.M., A.K.R., A.M.T., and W.L.C. acquired the data; S.B., Y.C., F.L.W., W.L., H.K., W.E.E., and M.V.R. analyzed and interpreted the data; W.L., S.B., L.H., F.L.W., and Y.C. drafted the manuscript; S.B., W.L., Y.C., L.H., H.K., B.C.B., J.N.C., D.S.D., W.E.E., K.W.M., A.K.R., A.M.T., W.L.C., F.L.W., and M.V.R. revised the manuscript for important intellectual content; F.L.W., Y.C., H.K., W.L., and S.B. performed statistical analysis; S.B., W.L., L.H., and M.V.R. provided administrative, technical, or material support; and S.B., W.L., and L.H. supervised the study.

Conflict-of-interest disclosure: W.E.E. and M.V.R. received a portion of the income St. Jude receives from licensing patent rights related to testing for TPMT genetic polymorphisms. The remaining authors declare no competing financial interests.

Correspondence: Smita Bhatia, Institute for Cancer Outcomes and Survivorship, University of Alabama at Birmingham, 1600 7th Ave S, Birmingham, AL 35233; e-mail: sbhatia@peds.uab.edu.