Introduction
In recent decades, financial stresses have brought academic and health care institutions of all sizes under increasing pressure to accomplish more with fewer resources. Many have responded by increasing emphasis and reliance on revenue-generating activities, including faculty salary support and other funding through research grants [Reference Wiegers, Houser and Pearson1]. Indeed, pursuing and obtaining external research grant funding now plays an important role in the professional careers of faculty and other investigators at most universities and academic institutions [Reference Gordon, Osganian, Emans and Lovejoy2]. This funding brings resources, knowledge, and prestige to both individuals and their institutions [Reference Feeney and Welch3] and is recognized as an important element of academic appointments and promotions [Reference Svider, Lopez, Husain, Bhagat, Eloy and Langer4,Reference Zyzanski, Williams, Flocke, Acheson and Kelly5]. External research support is also considered an important indicator of intellectual achievement and is associated with publications and other measures of scholarly impact [Reference Colaco, Svider, Mauro, Eloy and Jackson-Rosario6-Reference Jacob and Lefgren9]. Most importantly research grants are the mechanism through which novel and innovative scientific ideas are recognized, supported, and translated into practice.
Previous research has investigated research grant productivity and factors potentially associated with it. This work has found differences in grant submissions and awards on the basis of gender [Reference Gordon, Osganian, Emans and Lovejoy2,Reference Blake and La Valle10-Reference Vydareny, Waldrop and Jackson14], race/ethnicity [Reference Ginther, Schaffer and Schnell15,Reference Kaplan, Raj, Carr, Terrin, Breeze and Freund16], age [Reference Levin and Stephan17], specialty [Reference Tamblyn, Girard, Qian and Hanley18], and faculty rank [Reference Colaco, Svider, Mauro, Eloy and Jackson-Rosario6,Reference Brocato and Mavis19-Reference Waisbren, Bowles and Hasan21]. Past grant writing productivity has also been examined and found to be an excellent indicator of future grant and other research productivity [Reference Allison and Stewart22-Reference Raj, Carr, Kaplan, Terrin, Breeze and Freund24], a finding supportive of accumulative advantage theory, which posits that already productive scientists are likely to be even more productive in the future [Reference Cole, Cole and Cole25].
In order to support innovation in Clinical And Translational Science, the Clinical and Translational Science Awards (CTSA) has established a network of hubs across the country that are intended to maximize research impact [Reference Woolf26,Reference Zerhouni and Alving27]. The aims of the hubs range from improving translational science workforce and education to moving research along the translational spectrum from bench to bedside and into the communities served. An integral part of this mission is to help faculty increase productivity. To date, almost 15 years into the CTSA program, it remains unclear as to whether or not it has been successful in doing so.
To address this question, we investigated the relationship between faculty receipt of support services from a CTSA-funded university center and subsequent grant-related outcomes. Specifically, we focused on external research grant productivity, including both the numbers of grants submitted and awarded among health researchers at the University of Illinois at Chicago (UIC), and examined the potential support role played by UIC’s Center for Clinical and Translational Science (CCTS) by comparing the grant outputs of UIC faculty who did and did not receive CCTS services. UIC’s CCTS was initially supported by the CTSA program in 2009 with the primary goal of improving population health in the Chicago Metropolitan Region, with an important emphasis on the health of minorities and under-served populations. Since its inception, the CCTS has offered a variety of support services to university-affiliated investigators. Specific services include biomedical informatics, biostatistics, multisite clinical trial support, clinical research support, community engagement, educational support, pilot grant funding, regulatory support, research navigation, translational research technologies and a KL2 scholar program, and collaborative project support. Below, we employ 5 years of CCTS service utilization data to prospectively examine the relationship between use of any of these services and investigator grant productivity.
Methods
Data were collected between 2015 and 2019 from multiple data sources at the UIC. These included the CCTS user database, which collects information regarding all research support services provided to faculty at UIC, the Office of the Vice Chancellor for Research Proposal Approval Forms (PAF) tracking database, which collects information, including final award status, for all grant applications submitted by UIC researchers, and UIC’s human resources database, which contains demographic information regarding all faculty. The CCTS user database is systematically collected by CCTS cores, which enter information regarding the users of each of their services. These services include all aspects of the CCTS, spanning services such as hour-long educational workshops, one-on-one consultations on topics such as biostatistical methods, regulatory and Institutional Review Board submissions, and intensive training and funding programs such as the KL2 scholars programs and the pilot award program. The PAF dataset identifies the principal investigators of all noninternal grants processed by the university including federal, private, and nonprofit foundation grants. These datasets were integrated for the analyses presented in this paper. This research was approved by the University of Illinois at Chicago’s Institutional Review Board #2 (Social & Behavioral Research).
Analyses were restricted to faculty with appointments in one of the following UIC colleges: School of Public Health, Health and Human Services, College of Dentistry, College of Social Work, Biological Sciences, College of Applied Health Sciences, College of Nursing, or the College of Medicine at Chicago, Rockford, or Peoria. These colleges represented those most likely to have researchers involved in clinical and translational research and to have historically been users of CCTS support services. Eligible faculty members included those with an appointment on campus during at least three consecutive years during the study period (2015–2019). Given that some measures were not normally distributed, initial descriptive comparisons between CCTS users and nonusers were examined using the Mann−Whitney U test.
The first set of outcomes examined as dependent variables, in separate negative binomial regression models [Reference Long28], were counts of total numbers of grants submitted as principal investigator by individual faculty during the calendar years 2017, 2018, and 2019. The independent variable in each model was a count of CCTS service use, modeled in 2-year time increments (e.g., 2015–2016, 2016–2017, and 2017–2018) prior to the grant submission year being examined (e.g., 2017, 2018, and 2019). In these models, a service use was defined as any interaction with a CCTS-provided service, which would include any form of assistance, such as biostatistical support, study subject recruitment support, pilot grant support, and educational services, among others. Additional covariates included gender, race/ethnicity (African American, Asian, Latinx, and White/other), faculty rank (assistant, associate, and full professor), appointment in the College of Medicine (vs. other health-related colleges), and prior productivity, defined as individual record of grant submissions during the preceding 2 years, in 2-year time increments (e.g., 2015–2016, 2016–2017, and 2017–2018). Interactions between faculty use of CCTS services and their (1) gender, (2) race, and (3) faculty rank were also examined to determine if CCTS use was particularly successful in supporting the research productivity of women, historically underrepresented persons, and junior faculty. When examining interaction terms, each service count measure involved was dichotomized (e.g., 1 = any receipt of CCTS services vs. 0 = no receipt of CCTS services during the years in question). Only significant interactions are reported.
The second set of outcomes examined were counts of the total number of grants awarded to each faculty member during 2017, 2018, and 2019. The independent variable in each of these models were again the use of CCTS services during the preceding 2 years. Gender, race, faculty rank, and appointment in the College of Medicine (vs. other health-related colleges) were also included as covariates in these models, along with prior record of grant awards during the preceding 2 years. In addition, a set of interactions between the use of CCTS services and gender, race and faculty rank were also examined in separate models.
A final series of analyses examined the independent effects of specific types of CCTS service use on the grant submission and award outcome measures. Binary measures were constructed to indicate receipt of services from each of six CCTS cores, including biostatistics, bioinformatics, clinical research support, community engagement, educational support, and participation in the pilot grant process. Using this set of indicators, the main effects models described above were reestimated with these measures of individual core services substituted for the total count of CCTS services provided during the previous 2-year period.
Results
The models examined included between 1252 and 1346 UIC faculty, some of whom were users and some were not. The breakdown for each time period is seen in Table 1, which provides a summary of the sample and compares CCTS service users versus nonusers for all variables examined. Overall, female and Latinx faculty were more likely to be CCTS users in each year examined. Service users were less likely to be White/other for the time period 2017–2018. CCTS users overall were found to have submitted more grant applications during each time period as well as to have received more grants across all time periods. Users and nonusers also differed in their previous grant productivity, with CCTS users having submitted and received more grants for each time period examined.
Table 1. Study variables by Center for Clinical and Translational Science (CCTS) service use status
* Mann–Whitney U test p < 0.05.
** Mann–Whitney U test p < 0.01.
*** Mann–Whitney U test p < 0.001.
The three negative binomial regression models examining the relationship between any CCTS service use in the prior 2 years and grant submissions in 2017, 2018, and 2019 are presented in Table 2. The 2017 grant submission model (first model in Table 2) indicated that, controlling for gender, race/ethnicity, faculty rank, college affiliation, and prior grant submissions during 2015–2016, CCTS service use was associated with greater numbers of submissions in 2017. With White/other faculty serving as the reference group, Latinx faculty members were more productive than their White/other counterparts, and assistant professors were less productive than full professors. Having submitted more grant applications during 2015–2016 was also strongly associated with more grant submissions in 2017. No other covariates were significant. The 2018 model (second model in Table 2) also found that use of CCTS services between 2016 and 2017 was associated with greater numbers of grant submissions in 2018, controlling for grant submissions in 2016–2017, gender, race/ethnicity, faculty rank and college affiliation. Assistant professors were also less productive, compared to full professors, in the 2018 model, and prior grant submission activity during 2016–2017 was again associated with subsequent grant submissions in 2018. The final grant submission model, 2019 (third model in Table 2), again found those who used more CCTS services in 2017–2018 submitted more grants in 2019. As with the 2017 and 2018 models, assistant professors submitted fewer grants in 2019 when contrasted with full professors, and grant submissions during the previous 2-year period were predictive of grant submissions in 2019.
Table 2. Negative binomial regression models of effects of Center for Clinical and Translational Science (CCTS) service use on grant submissions, 2017–2019
IRR, incidence rate ratio.
*p < 0.05.
** p < 0.01.
*** p < 0.001.
The relationship between CCTS service use and number of grants awarded was next examined (see Table 3). Similar to the grant submission models in Table 2, grant awards were examined while adjusting for counts of prior CCTS service use in the previous 2 years, prior grant funding obtained in the preceding 2 years, gender, race, faculty rank, and College of Medicine appointment status. The 2017 grants award model (first model in Table 3) found a significant association between counts of services used in 2015–2016 and number of grants awarded during 2017. Latinx faculty were found to have received more grant awards in 2017 than did White/other faculty, and assistant professors received fewer grant awards that year, compared to full professors. Grant awards during the two previous years (2015–2016) were also associated with successful grant awards in 2017. The 2018 grants awarded model (second model in Table 3) also revealed a significant positive relationship between number of grants awarded and number of CCTS services received between 2016 and 2017 as well as prior number of grants awarded between 2016 and 2017. The 2019 model (third model in Table 3) followed a similar pattern. In each year, assistant professors were again found to be less likely to receive grant awards.
Table 3. Negative binomial regression models of effects of Center for Clinical and Translational Science (CCTS) service use on grant awards, 2017–2019
IRR, incidence rate ratio.
*p < 0.05.
** p < 0.01.
*** p < 0.001.
Of the sets of interactions examined between receipt of CCTS services and various covariates, three significant interactions were identified, all in the 2018 models. Fig. 1 compares covariate-adjusted mean number of grant submissions in 2018 by faculty race/ethnicity. Prior use of CCTS services in 2016–2017 had a modest or no impact on subsequent grant applications from African American and Asian faculty, relative to the effect of CCTS services on the submissions made by Latinx and by White/other faculty. Adjusted number of grant awards by faculty race/ethnicity are compared in Fig. 2. Here, a slight negative relationship was observed between receipt of services and productivity among Asian faculty, with positive effects of CCTS service use observed among Latinix, White/other, and African American faculty. A third significant interaction, between CCTS service use and faculty rank, is depicted in Fig. 3. Here, the impact of prior CCTS service use was most strongly observed among assistant professors, with those engaged with CCTS more likely to be successful in obtaining a grant award in 2018, relative to those who did not utilize CCTS services during the prior 2 years.
Fig. 1. Adjusted mean number of grant applications (by Race/Ethnicity – 2018).
Fig. 2. Adjusted mean number of grant awards (by Race/Ethnicity – 2018).
Fig. 3. Adjusted mean number of grant awards (by Faculty Rank – 2018).
One final set of analyses investigated the independent effect of various types of CCTS services on the measures of productivity for each time period. Receipt of any biostatistics services were predictive of subsequent grant submissions in 2017 and of both submissions and awards in 2019. Pilot grant program participation was associated with increased grant submissions in 2018 and of both submissions and awards in 2018 and 2019, and educational support services were associated with grant submissions in 2019 and with grant funding in 2019. Bioinformatics, clinical research support, and community engagement services were not associated with outcomes in any of the models examined.
Discussion
There are numerous personal and environmental variables associated with research productivity in medical research [Reference Bland and Ruffin29,Reference Bland and Schmitz30]. In this paper, we examined the effects of a unique measure of research support from a CTSA-funded CCTS on grant productivity among investigators within a large medical research campus. Even after accounting for strong predictors of future grantsmanship activities (i.e., past productivity), we found an association between CCTS support and subsequent grant output. The UIC CCTS provides a wide range of services, such as bioinformatics, statistical support, subject recruitment, and dissemination and implementation, to faculty and staff. These are designed to provide additional research support beyond what is typically available at this institution in the hope of increasing the productivity of investigators. We were able to document that these services were collectively associated with two important productivity measures: preparing applications and successfully obtaining external grant funding.
We also examined the effects of the UIC CCTS program on the productivity of historically under-resourced investigators such as junior faculty, women, and minority groups who may not always have access to adequate research support and mentoring. In our main effects models, assistant professors were consistently found to be less productive than full professors in submitting grants and receiving grant awards. One interaction, though, did suggest that assistant professors disproportionately benefited from CCTS services during one of the years examined (Fig. 3). No gender differences in grant productivity were observed, and CCTS support was not found to have a differential effects on the productivity of male versus female faculty for any year examined.
Some race/ethnic differences in grant productivity were also observed, as Latinx faculty, who were also more likely to be CCTS users, were also more likely to submit grants and obtain awards when contrasted with White/other faculty in 2017. In addition, when examining interactions, CCTS support was associated with increased grant submissions and awards obtained by Latinx and other/White faculty, relative to the benefits of service support realized by African American and particularly Asian faculty, in 2018 (Figs. 1 and 2). These interactions were not observed for the other years examined. It may be that the numbers of faculty representing some race/ethnic groups are too small to reliably detect interactions in these models, especially African Americans who make up only 4% of this sample. UIC sits in a unique and diverse part of Chicago that enjoys high levels of representation of Latinx communities along with other minority groups. Additionally, some cores within CCTS, such as the pilot grant program and community engagement, prioritize projects which relate to health priorities set by the municipal government, so-called “Healthy Chicago” areas, which place an emphasis on addressing social determinants of health and health equity. It is possible that Latinx investigators are more likely to be involved in these priority areas and receiving CCTS services to assist in these projects. Without deeper examination of the investigators and their specific grants it is difficult, however, to do more than speculate at this point in time.
We further disaggregated CCTS services into different service types, including biostatistics, pilot grant participation, educational support, bioinformatics, clinical research support, and community engagement services. We found that receiving any biostatistics support was associated with increased grant submissions and/or awards across 2 of the 3 years examined. This may be accounted for by users engaging CCTS services prior to a grant submission where services such as community engagement or clinical research support, which saw no association with grant productivity, may be more likely to be utilized once a grant is received. However, there still remains heterogeneity within services which may obscure some of these associations.
A key limitation of this research is that the analyses being reported are unable to link specific services provided by CCTS directly to any specific grant. Rather, we report longitudinal associations between service receipt and subsequent grant productivity. It is also common for grant applications to go through many transformations and revisions before they are submitted and/or funded, making it difficult to link outcomes with the receipt of any one support service or to know the precise timing of when grants were prepared, relative to when CCTS services were received. Additionally, faculty not actively planning to submit a grant may be less likely to access CCTS services. While we attempted to control for prior productivity, this may not be sufficient for early career faculty who do not have as long of a history of prior grant submission and awards, especially as a principal investigator. Additionally, we have attempted to control for type of service accessed by faculty, but there will remain a large amount of variation in the type of services provided by each of these cores. We also acknowledge that service users and nonusers may differ in other important ways that cannot be easily captured in these analyses.
We were also limited in the number of grant-related outcome measures we were able to examine. Other measures that might be pursued include grant review scores, source of support (federal and other), and success rate for initial applications. Of course, grants are not the only indicators of research productivity – bibliometrics, patents, collaborations, along with potential qualitative measures like social networking, mentoring, and community outreach [Reference Gaughan, Melkers and Welch31] should also be considered. Relatedly, CCTS services are highly heterogeneous and range from options such as “Brown Bag” hour-long workshops to intensive programs which last multiple years such as the KL2 scholars and pilot award funding. While we have attempted to control for this by examining several different service types, further assessment of the intensity of support and productivity would provide further clarity regarding the relationship between CCTS services and grant productivity. Our dataset only included faculty listed as the principal investigator on each grant, which likely leads to those listed as co-investigators to be misclassified in our dataset. We also restricted our analysis to faculty only and were unable to link staff who were receiving services on behalf of faculty. In the future, we plan to continue exploring the relationship between CCTS service provision and these various measures of research productivity and why nonusers do not take the advantage of available services.
Finally, it is also important to consider some of the strengths of this study, which include a large sample size and a relatively complete dataset of all variables and outcomes examined. In particular, the grant application outcomes data are complete as UIC researchers are required to submit an internal approval form for all university applications, and these data were available for our analyses. We also benefited from a long study period (2015–2019) and were able to temporally link independent and dependent variables. We were also able to account for a number of potential covariates. Although we strived to establish longitudinal links between service utilization and subsequent grant productivity, it is nonetheless important to recognize the difficulty of establishing the temporal nature of the activities being examined. That is, it remains difficult to say definitively whether those who efficiently access available support resources become more productive grant writers, or whether more competitive grant writers are more likely to seek out and utilize available support services. This remains an important question that warrants further research.
Acknowledgements
This research was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Award Number UL1TR002003, to the University of Illinois at Chicago Center for Clinical and Translational Science.
Disclosures
The authors state they have no conflicts of interest to report.
Open access
