Self-reported positive impact of mentored clinical research training is associated with academic success in hematology

Patient-oriented research is centered on patients and is situated between basic laboratory and population-level research.¹ The American Society of Hematology (ASH) developed the Clinical Research Training Institute (CRTI) in 2003 to promote the development of successful clinician researchers with a focus on patient-oriented research. Since its inception, the program has continued to train and mentor fellows and junior faculty in benign or malignant hematology in keeping with the mission of ASH.

A substantial number of resources, including monetary and time commitments, on behalf of faculty and trainees are required to sustain the program, so it is critical that evaluation of program outcomes is performed on a regular basis. In other words, it is important to evaluate whether CRTI contributes to the academic success of program participants. Although evaluation is hindered by challenges such as identifying ideal control groups,² describing factors associated with success can provide important insights. For example, in 2016, we identified a gender disparity among CRTI alumni, with men having more published research articles and greater percent effort in research.³ In a follow-up evaluation, we found that the gender disparity persisted and was not explained by caregiving responsibilities.⁴ 

Demographic characteristics such as gender, race, and ethnicity are variables previously associated with academic success. These fixed variables are not modified by training experiences. However, participants’ perceptions of how CRTI impacts research and career development are potentially malleable. These are attributes that we have measured throughout the CRTI program, and we recently expanded upon them in the last distributed survey. Consequently, the objective was to determine whether the self-reported impact of CRTI on research retention, career development, and connectedness to other hematology investigators was associated with academic success.

The CRTI program has previously been described in depth.³ To summarize, CRTI is a 1-year mentored clinical research training program. Participants must be senior fellows or junior faculty within the first 3 years of their first appointment and must intend to pursue a career that includes patient-oriented hematology research. Each year, 20 to 23 applicants are accepted to the program. The faculty consists of a similar number of experienced patient-oriented researchers, including 5 to 6 biostatisticians.

Each year, the program typically includes 3 face-to-face meetings that occur in August (1 week), December (1 day), and May (1 day). Training includes didactic, small-group, and one-on-one sessions with an increasing number of interactive workshops. The participants focus on a research proposal that is developed throughout the program. During the August workshop, they are paired with a CRTI mentor who interacts with them throughout the year.

In 2016, ASH established an evaluation plan that consisted of cross-sectional surveys of all CRTI alumni every 3 years. The 2020 survey focused on measuring self-reported impacts of CRTI on research retention, career development, and connectedness to investigators in hematology.

We included all alumni who participated in CRTI from its inception in 2003 to the 2019 program, completed all aspects of the program, were alive, and were in good professional standing with their academic institution or employer.

The survey was distributed via e-mail in January 2020 to all eligible CRTI participants. Data were collected and managed using REDCap electronic data capture tools.⁵ The survey included demographic characteristics, current academic status (including promotion within the past 3 years and percent effort in research), personal and family status, and self-reported perceptions of the impact of CRTI. These questions focused on the role of CRTI in the following: retention in hematology research, facilitation of career development in research, understanding of requirements to succeed in research, feelings of connectedness to investigators in hematology, and continued feelings of connectedness to others in hematology. These questions were asked on a 5-point Likert scale that ranged from strongly disagree to strongly agree. The exposure variables were those who agreed or strongly agreed vs those who were neutral, who disagreed, or who strongly disagreed with these statements.

In addition to the survey questions, participants were asked to submit their curriculum vitae (CV). Grants, publications, and invited lectures were abstracted from the CV from the previous 3 years (January 2017 to January 2020). Abstracted results were confirmed with participants who then had an opportunity to correct the reported accomplishments.

The outcomes were derived from our previous work in CRTI evaluation and consisted of the following within the previous 3 years: being a principal investigator on a federal grant, receiving a loan repayment award, the number of peer-reviewed publications (total, first, senior, and collaborator), number of invited lectures, number of promotions, and percent effort in research. American federal grant sources were those funded by the National Institutes of Health (R, K, U, P, and T mechanisms), the Agency for Healthcare Research and Quality, the Patient-Centered Outcomes Research Institute, and the Human Resources and Services Administration. Canadian federal grant sources were those funded by the Canadian Institutes of Health Research.

To compare those who agreed or strongly agreed that CRTI impacted retention in hematology research, facilitation of career development in research, understanding of requirements to succeed in research, and feelings of connectedness to investigators in hematology vs those who were neutral, disagreed, or strongly disagreed, dichotomous outcomes (principal investigator on federal grant, loan repayment award, or promoted) were compared using the χ² test, and continuous outcomes (number of publications and percent effort in research) were compared using the Wilcoxon rank sum test. Then we completed the same analyses for participants who are in academics and removed participants in private practice and industry. All tests were two-sided, and a P value < .05 defined statistical significance. All analyses were performed using R studio version 3.6.1 (The R Foundation for Statistical Computing, Vienna, Austria).

There were 341 potential CRTI alumni. Four were deceased, 2 were ineligible because of home institution–determined research misconduct, and 1 was ineligible because he or she did not complete the program, leaving 334 eligible respondents. Of these eligible respondents, 321 responded to the survey and submitted their CV, resulting in a response rate of 96.1%.

Table 1 shows demographics of the cohort; 42.7% were male, 5.9% were black or African American, and 5.0% were Hispanic. Table 2 shows current career and personal status and demonstrates that 84.4% were in an academic setting. Table 3 shows the self-reported impacts of CRTI. Of the 321 respondents, 250 (77.9%) agreed that CRTI was instrumental to research retention, 268 (83.5%) agreed that CRTI facilitated career development, 296 (92.2%) agreed that CRTI allowed for a better understanding of the requirements to succeed in research, and 289 (90.0%) agreed that CRTI increased feelings of connectedness to hematology investigators.

The baseline demographics by self-reported impacts of CRTI are provided in supplemental Appendix 1. The participants who agreed that CRTI was instrumental to retention in research, facilitated research career development, allowed better understanding of requirements to succeed in research, increased feelings of connectedness to hematology investigators, and continued feelings of connectedness were associated with more recent participation in the program and were more likely to focus on malignant vs benign hematology.

Table 4 shows the association between self-reported impacts of CRTI and academic outcomes. Those who agreed that CRTI was instrumental to retention in hematology research, agreed that CRTI facilitated their career development in research, and continued to feel connected to investigators in hematology were significantly more likely to have more total, first, senior, or collaborator author publications and had greater percent effort in research. Those who agreed that CRTI allowed a better understanding of what is required to succeed in research had significantly more first, senior, and collaborator author publications. Finally, those who agreed that CRTI enhanced their connectedness to other investigators in hematology research had more first author publications and greater percent effort in research.

In an effort to evaluate the self-reported impact of CRTI on physicians in academic settings, we completed a sub-analysis with the participants in private practice and industry removed. These results (provided in supplemental Appendix 2) demonstrate the same general trend as that for all of the CRTI participants. The median total publications among participants who felt like CRTI impacted their careers was slightly higher among the academic cohort than the entire cohort, which included all participants. Along these lines, the difference between the median of total publications and senior publications was no longer statistically significant when reflecting on whether CRTI facilitated a feeling of connectedness. The median percent effort in research remained significantly higher among those who endorsed continuing to feel connected to investigators in hematology because of CRTI.

In this cross-sectional evaluation of factors associated with academic success, we found that all evaluated factors, specifically the self-reported impacts of CRTI on research retention, career development, and connectedness to hematology investigators, were associated with greater academic success. Approximately 96% of CRTI alumni responded to the survey, and this level of engagement is a feat. Almost 85% of CRTI alumni remain in an academic setting. Investment in these early-career investigators in hematology is supported by these results.

Our results are consistent with those in other studies that have examined the impact of formal programs to enhance adult educators and researchers. Year-long training to enhance medical education skills was associated with significant changes in academic promotion, educational leadership, education committees, and education funding compared with academic medical faculty who did not participate in a formal training program in a single institution.⁶ Social connectedness to the field of education and to a mentor was associated with retention of early-career educators.⁷ In addition, junior faculty of American medical schools who had a mentor rated their research preparation and research skills higher than did faculty without mentors.⁸ However, these higher ratings were self-reported, without objective data to measure work products or accomplishments.⁸ 

Given that these data are observational, it is possible that there are important confounders when examining the self-reported impacts of CRTI on academic outcomes. Intrinsic personality features may be associated with the perception of CRTI impact and the ability to take full advantage of what CRTI can provide, as well as academic success. However, impacts of CRTI are likely intertwined, and even if CRTI has an impact on only one aspect of academic success, it may have indirect impacts on others. For example, if CRTI increases feelings of connectedness with hematology investigators, this may in turn improve a person’s ability to increase his or her understanding of what is required to succeed in research.

A strength of this study is the very high response rate, which reflects the positive attitude of alumni toward the program. Furthermore, this study builds upon previous surveys to identify potentially targetable factors that may impact academic success. However, our results must be interpreted in light of their limitations. Most importantly, it is possible that a positive sense of CRTI impact is driven by academic success or other confounding variables. We did not provide a free-text response option for participants to share other reasons that were not listed as factors outside of CRTI that may have contributed to their success. Understanding whether these attributes impact future success may help to clarify the role of these factors. In addition, measures such as loan repayment awards or promotion in faculty rank capture a short period in time for trainees who completed the program over variable time ranges from the time that they completed this survey. Thus, these measures may not represent sensitive positive outcomes, because some awards are typically received early in one’s career, and promotion to the highest faculty rank of professor, for example, will hit a ceiling for the trainees from the earlier years of CRTI. Despite these limitations, the trends for accomplishments were high for the cohort, particularly for those with a perceived positive experience.

In conclusion, self-reported perceptions that CRTI had an impact on research retention, career development, and connectedness to hematology investigators was significantly associated with more publications and greater percent effort in research. Clinical research training programs should identify and implement approaches to enhance these characteristics.

This work was supported by a grant from the National Cancer Institute, National Institutes of Health (2R25CA168526).

Contribution: A.A.K. designed the research, interpreted the data and wrote the article; S.K.V. designed the research, analyzed the data, and edited the article; G.D. collected the data, assisted with the analysis, and edited the article; C.C., A.C., W.S., and A.W. interpreted the data and edited the article; J.F. collected the data and edited the article; and L.S. designed the study, collected, interpreted, and analyzed the data, and wrote the article

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Allison A. King, Washington University School of Medicine, 4444 Forest Park Blvd, MSC 8505-66-1, St. Louis, MO 63108; e-mail: king_a@wustl.edu.

For data sharing, please contact Allison A. King via e-mail at king_a@wustl.edu.