Skip to main content
Search
Search
Advanced Search
Skip main navigation
Free access
Research Article
10 September 2021

Impact of clinical research coordinators on research output in orthopedic surgery

Authors: Garrett K Berger https://orcid.org/0000-0001-8389-0691 [email protected], John C Neilson, David M King, and Adam N WooldridgeAuthor Info & Affiliations
Volume 10, Number 15
https://doi.org/10.2217/cer-2021-0020
PDF

Abstract

Aim: To describe the efficacy of clinical research coordinators (CRCs) in terms of research output among faculty, residents and medical students. Materials & methods: We analyzed and reviewed departmental research output from the 3 years prior to the hire of 3 CRCs compared with the 3 years following. Results: From 2014 to 2016, the department output was a total of 27 peer-reviewed publications, 16 oral presentations and 33 poster presentations. From 2017–2019 there was a 52% increase in publications (n = 41), 131% increase in oral presentations (n = 37) and a 61% increase in poster presentations (n = 53). Conclusion: The implementation of CRCs has markedly increased tangible research output and trainee involvement. This role may serve as a valuable model for other academic departments.
Research contributions to one’s particular field of practice is a cornerstone of academic medicine. Aligned with this principle is the requirement for scholarly ventures by residents, set by the Accreditation Council of Graduate Medical Education (ACGME) [1]. Practicing surgeons, residents and medical students alike face difficulties in meeting and exceeding these requirements, most notably in the form of restricted time availability [2]. Further, within the realm of orthopedic surgery, many training programs lack formal pipelines for research opportunities and output.
With these restrictions in mind, a number of training programs, surgical and otherwise, have made efforts to aid research output in the form of administrative assistance [2–5]. Individual programs have made interventions that address specific deficits in their system. These range from hiring medical writers to assist with manuscript development, postdoctoral researchers to help with research question formulation and statisticians to aid in data analysis [2,3,4].
In 2016, a new Department Chair took over our Department of Orthopedic Surgery. One pillar of the Chair’s plan for the department revolved around research development. At a faculty retreat, the Institutional Review Board (IRB) process was identified as one of the largest barriers to research output at our institution. As such, in late 2016, the first of three clinical research coordinators (CRCs) was hired to aid in this development, with a goal of reducing the administrative burden and increasing time availability of faculty and residents. CRCs would be responsible for IRB applications, administrative project management and medical student coordination among other administrative duties. This manuscript describes the efficacy of this implementation with hopes of providing a useful framework for other training programs to model, regardless of medical or surgical specialty.

Materials & methods

In order to assess the degree of impact CRCs position had on research output at our institution, we analyzed and reviewed departmental research output from the 3 years prior to the hire (2014–2016) compared with the 3 years following (2017–2019). Data were gathered from several sources and cross referenced with one another. Sources included: ACGME resident research reporting, faculty database records, medical student summer research records and faculty PubMed searches. Records were excluded if the primary institution was an outside facility. For example, certain faculty carry publications which were published while employed by our institution, but for which the research is affiliated with their fellowship. In other cases, faculty were co-authors on studies that were primarily based within another department at our institution. Such studies were excluded from the analysis as they did not utilize the resources of our departmental CRCs. Publications were excluded if they were not peer reviewed.
Upon compiling a database, records were categorized based on research year, research type (publication, poster, oral presentation), author type (medical student, resident, clinical faculty, research faculty) and study type (prospective, retrospective, descriptive, clinical science, basic science). Quality data were assessed using Impact Factor (IF) for publications and level of presentation (local, regional and national) for presentations. IF was assessed based on the year the manuscript was published rather than the current IF, using the Web of Science Journal Citation Report for the respective year [6]. Data analysis compared these variables with respect to whether they were conducted prior to or after the implementation of the first CRC. Statistical analysis was performed using SPSS statistical software (version 25). This retrospective study was deemed exempt by our Institutional Review Board as no patient data were used.

Clinical research coordinators

Of the three CRCs, two have master’s level degrees in science and one with a bachelor’s degree in science. Further, two have obtained Certified Clinical Research Professional (CCRP) certification, an exam-based certification sponsored by the Society of Clinical Research Associates (SOCRA) as an objective way to unify the field. The first of these three CRCs was hired at time point zero (2016), the second 1 year later (2017) and the third in 2019. We expect to see the impact of these additional hires as time continues. None of the CRCs have authorship in any of the publications or presentations discussed.

Results

During the pre-implementation period from 2014 to 2016, the department output summated to a total of 27 peer-reviewed publications, of which 22% (n = 6) involved medical students and 15% (n = 4) involved residents (Table 1). In the post-implementation period of 2017–2019, the department experienced a 52% increase in publications (n = 41) of which 34% (n = 14) involved medical students and 17% (n = 7) involved residents. Medical student and resident output increased by 133% (14 vs 6) and 75% (7 vs 4), respectively. These values have incrementally increased with each ensuing year following CRC implementation. When normalized to volume of publications, pre-implementation project characteristics consisted of a predominance of clinical science studies (74%, n = 20), most of which were retrospective in nature (56%, n = 15). When compared with the post-implementation category, this pattern held true though to a greater extent. Of the 41 publications in this category, 85% (n = 33) were based in the clinical sciences with 69% (n = 27) being retrospective. These increases were seen at the expense of basic science studies (down to 15 from 26%; n = 6 vs 7) and prospective studies (down to 15 from 41%; n = 6 vs 11). Making up the remainder were descriptive reviews up to 15% (n = 6) in the post-implementation period from 4% (n = 1) in the pre-implementation period. A descriptive breakdown of these variables, subcategorized by individual year, can be seen in Table 1.
Table 1. Project descriptors of published manuscripts.
  Project characteristicsTrainee involvement
  n (% total)n (% total)
  Peer reviewed publicationsClinical scienceBasic scienceRetrospective studiesProspective studiesDescriptive reviewsMedical studentResident or fellow
Pre-implementation201453 (60)2 (40)2 (40)2 (40)1 (20)1 (20)0 (0)
201586 (75)2 (25)4 (50)4 (50)0 (0)1 (13)2 (25)
20161411 (79)3 (21)9 (64)5 (36)0 (0)4 (29)2 (14)
Total2720 (74)7 (26)15 (56)11 (41)1 (4)6 (22)4 (15)
Post-implementation20171010 (100)0 (0)8 (80)0 (0)2 (20)2 (20)1 (10)
20181511 (73)4 (27)9 (60)4 (27)2 (13)6 (40)2 (13)
20191612 (86)2 (14)10 (71)2 (14)2 (14)6 (38)4 (25)
Total4133 (85)6 (15)27 (69)6 (15)6 (15)14 (34)7 (17)
Peer reviewed publications: sum of publications, further subcategorized in all columns to the right (clinical vs basic science, retrospective vs prospective vs descriptive. The publications are then subcategorized into medical student and/or resident/fellow involvement.
In the pre-implementation period, there were 16 oral presentations and 33 poster presentations. This as compared with the post-implementation period during which there was a 131% increase in oral presentations (n = 37) and a 61% increase in poster presentations (n = 53). These changes are also visually depicted in Figure 1. When looking at these numbers by year, post implementation, increases are seen on a more incremental level. Publication output was 10, 15 and 16 in 2017, 2018 and 2019, respectively. Likewise, poster presentations increased from 12 to 16 to 25 over these same 3 years, while oral presentations tallied at 13, 17 and 7.
Figure 1. Descriptive analysis of annual research output.
Research output is stratified by type (publication, poster presentation and oral presentation) and depicted based on year published or presented. Clinical research coordinator hiring is depicted by the vertical dashed line.
Objective measures of quality were more difficult to assess. Of the manuscripts published in the pre-implementation and post-implementation periods, a total of 52 publications had IF data available (21 and 31, respectively). The mean IF in the post-implementation period was greater than that of the pre-implementation period, though no statistical significance was found (2.978 ± 1.60 vs 2.342 ± 1.04; p = 0.11). When comparing presentation data, no significant difference was found in the proportion of national, regional or local conferences. Table 2 reports these data in more detail.
Table 2. Quality characteristics of publications and presentation.
 Publication characteristicsPresentation characteristics
 Peer reviewed manuscriptsImpact factorn (% total)Chi-squared
  n†Mean (SD)t-testTotal abstractsNationalRegionalLocal 
Pre-implementation27212.342 (1.04) 4920 (41)4 (8)25 (51) 
Post-implementation41312.978 (1.60)p = 0.119642 (44)5 (5)49 (51)p = 0.77
†
Manuscripts for which IF data were available by year on Web of Science Journal Citation Reports.
Quality characteristics are assessed pre- and post-CRC implementation. Presentations include both oral and poster varieties. Only the highest level of conference is included in the presentation category (i.e., if a poster was presented both locally and regionally, only the regional presentation is tallied).
CRC: Clinical research coordinator; IF: Impact factor.

Discussion

Following the implementation of the CRC program within our Department of Orthopedic Surgery, there have been tangible increases in both research volume as well as trainee involvement. The trend of research output has continued to increase with each additional year following CRC implementation. During that time period, two additional CRCs were hired.
When evaluating these data, it is important to consider the time required to bring a project like this to scale. When the first CRC was hired in late 2016, it took several months to build an organization system and establish a baseline. This is why we chose to set 2017 as the first post-implementation year. Further, research projects take time. While the 2017 data do not show a significant increase in research output, one must consider how many of the 2018 and 2019 publications were actually started in 2017 secondary to CRC involvement. This trend is highlighted with visual depiction in Figure 1. Looking ahead, we expect a continued increase in research production as the pipeline matures. Furthermore, the CRCs have been pivotal in the creation of a clinical database as well as the establishment of prospective studies, both of which will unfold in the coming years resulting in presentations and publications. Critical to the success of the CRC program, a formal structured process begins with residents in the PGY1 year through which CRCs track and assist their progress toward their research requirements.
In order to assess any potential reallocation of effort from other missions to research, we retrospectively reviewed the annual relative value unit ([RVU]; for those less familiar with the concept of RVUs, please see this link from the American Academy of Professional Coders: www.aapc.com/practice-management/rvus.aspx#WhatAreRVUs) output of stable faculty during our study period. Per our data warehouse (dating back to 2015), there were no significant changes in same store sales RVU. In contrast to existing literature in which dedicated research time led to a need for additional clinical hiring, our findings demonstrate a unique increase in research production without such a sacrifice [7]. Further, no changes to the ratio of dedicated clinical time versus research time were made. Stemming from the topic of cost, is the salaries of the CRCs. Our CRC manager earns a base salary of approximately US$80,000 while our additional CRCs earn closer to approximately US$60,000. These salaries are supplemented with fringe benefits at a rate of 22%. These numbers are important for readers to consider when evaluating the feasibility of this model within their respective programs. While one CRC alone is very likely to have an impact, they will be limited by their own time with respect to study coordination.
Our suggestion is to begin with one CRC hire and incrementally increase personnel as demand increases. Further, these data may be utilized by Department Chairs in order to solicit resources from their affiliated colleges. Alternatively, new Chair hires may include these requests in their ‘Chair package’ such that these funds are not coming from clinical allocations, as was the case in our institution.
Our results highlight the increase in clinical studies (up 65% in the post-implementation period; n = 20 vs n = 33); however, basic science studies remained relatively stable (down 14% in the post-implementation period; n = 7 vs n = 6). These findings were not surprising to our team. The vast majority of our basic science faculty perform research as a part of their full-time employment. As a consequence, clinical faculty without protected research time stand to benefit most from the administrative assistance provided by the CRCs. With this added free time, clinical faculty have the opportunity to be more involved in research design, including prospective studies. We expect this trend to continue as the CRC pipeline continues to pay dividends.
Published research quality was loosely assessed by changes in IF for publications and local, regional and national conferences for posters/oral presentations. As highlighted in Table 2, no statistically significant changes were observed in either of these categories. This is not surprising, as we would not necessarily expect the addition of CRCs to impact the quality of research. CRCs are not tasked with vetting research ideas nor ensuring their scientific merit. However, as the implementation of prospective databases continues to mature, we do expect higher quality prospective studies to reach publication. More recently, our Department has created the role of Director of Resident Research, who will work directly with residents in order to improve the quality of their projects. To date, no such projects have yet reached maturity and thus, are not reflected in the data. Notably, no changes were made to the policies surrounding departmental support of publication fees, research education or research mentorship during the time period of this study.
Importantly, and central to the purpose of this study, trainee involvement in research projects was evaluated. A marked increase in medical student involvement was seen post implementation, increasing by 133% overall and accounting for 34% of publications as opposed to 22% in the pre-implementation category. This effect is largely due to the CRC’s involvement with the medical student summer research program at our affiliated medical school. Through this program, the CRC meets with students and assesses their interest in certain areas of orthopedic surgery. Then, based on a running database of potential and ongoing projects, students are matched to a project for which they will work full time during the summer between their first and second year of medical school. Junior medical students, having more time for research in relative terms, are able to commit to hours-intensive tasks such as chart review data abstraction (for retrospective studies) and subject timing and coordination (for prospective studies). Secondary to these added benefits, the overall number of posters and publications saw an increase. Notably, the major barrier to success in this summer program was obtaining IRB approval by the time that students were slated to begin their summer research. In many cases, students were not able to work on their research for a portion of their summer, while awaiting IRB approval. Following the implementation of the CRC program, IRBs were submitted and approved in a timelier manner, thus maximizing student productivity in their limited summer months.
With respect to resident involvement, a 75% increase was seen in published manuscript authorship (n = 7 vs n = 4). While the volume of these figures is not overly abundant, it is important to note that many of the oral/poster presentations were also authored and presented by residents. In the pre-implementation period, residents presented one of the combined 49 poster and oral presentations (2%). This as compared with 17 of 90 in the post-implementation period (19%). With respect to medical students, on the regional national scale, poster and oral presentations increased from 12% (n = 6/49) to 24% (n = 22/90). With these figures in mind, combined with those of published manuscripts, it is evident that trainee involvement in research within our Department of Orthopedic Surgery has, and continues to, make great advancements. Notably, no changes to resident curriculum or schedules were made during this study period.
Ultimately, an increase in research production alone is not of particular importance. Rather, the impact that this increase in research output has on medical literature and careers of those involved should be considered. Orthopedic surgery is an increasingly competitive specialty for medical students to apply into. Research plays an important role in demonstrating an applicant’s ability to start a project and see it through to the finish. Recent literature has shown that high research output (10+ research items) is associated with a 10% increase in interview yield among non-Alpha Omega Alpha orthopedic applicants [8]. Further, studies regarding selection criteria for orthopedic fellowship programs cite subspecialty specific research as an increasingly important parameter [9]. Outside of orthopedic surgery residencies and fellowships, research plays a similarly important role with matched applicants carrying a significantly greater number of research publications (mean 5.18 vs 3.66) [10]. While residency and fellowship applications are certainly more than research numbers alone, a streamlined process for research production can only provide applicants the opportunity for a higher quality research experience with assistance overcoming traditionally daunting administrative barriers.

Limitations

As with any study that is retrospective in nature, original recording errors make data prone to misinterpretation. While it can be argued that the recording of research projects ongoing at our intuition may have increased while the true number of projects did not, our research team finds this to be of marginal impact. We have institutionally kept diligent records of resident research activity for years prior to the CRC implementation for ACGME reporting reasons. With respect to medical student projects, the medical student summer research program coordinator provided independent records for the pre-implementation period which allowed for a complete dataset. Of note, poster presentations were not recorded as a full citation in our database. Instead, only the presenting author was recorded in the majority of cases. As such, data analysis in total to include co-author numbers was not possible for these project types. In the case of publications; however, complete citations were available. Finally, several factors that may have contributed to increased research productivity, outside of CRC involvement, must be considered. The new Department Chair did place a greater emphasis on research than the previous Chair had. With that said, there was no compensation for increased research output during this study period, monetary or otherwise. One additional clinical faculty member was hired with protected research time, but this was late in the study period and this faculty member had not yet produced any publications. There was no other expansion in infrastructure or human resources that we can directly attribute these changes to. While other factors that are difficult to measure undoubtedly played a role in the increase in research productivity in the department, we feel that CRCs were by and large the driving force behind the findings reported in this study.

Conclusion

The implementation of CRCs within our Department of Orthopedic Surgery has markedly increased the tangible research output. Trainee (resident, medical student) involvement in published manuscripts has also increased with each year following implementation. The implementation of this role may serve as a valuable model for other academic departments looking to assist their faculty and trainees in successful research production.
Summary points
•
Practicing surgeons, residents and medical students alike face difficulties in meeting and exceeding requirements for research contributions and scholarly ventures, most notably in the form of restricted time availability.
•
In 2016, a new Department Chair took over the authors Department of Orthopedic Surgery. With an aim of aiding research output, in late 2016, the first of three clinical research coordinators (CRCs) was hired to aid in this development, with a goal of reducing the administrative burden and increasing time availability of faculty and residents.
•
This paper analyzed and reviewed departmental research output from the 3 years prior to the hire of three CRCs compared with the 3 years following, and found an increase in the output of peer-reviewed publications, oral presentations and poster presentations.
•
The implementation of CRCs has markedly increased tangible research output and trainee involvement. This role may serve as a valuable model for other academic departments.

Acknowledgments

The authors wish to thank the CRCs: M Ellestad; K Gonzalez; J Zemaj.
This work has previously been presented as follows: Podium: GK Berger, JC Neilson, DM King, AN Wooldridge. Impact of CRCs on research output in orthopedic surgery. MCW Innovations in Healthcare Education Research Annual Conference, September 2020; and Poster: GK Berger, JC Neilson, DM King, AN Wooldridge. Impact of CRCs on research output in orthopedic surgery. Presented at American Orthopedic Association Annual Leadership Meeting. Albuquerque, NM April 2021.

Financial & competing interests disclosure

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this manuscript.

References

Papers of special note have been highlighted as: • of interest
1.
Education ACGM. Accreditation Council for Graduate Medical Education (ACGME) program requirements (Residency). Accreditation Council for Graduate Medical Education website (2020). www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRResidency2020.pdf
Google Scholar
2.
Penrose LL, Yeomans ER, Praderio C, Prien SD. An incremental approach to improving scholarly activity. J. Grad. Med. Ed. 4(4), 496–499 (2012).
PubMed
Google Scholar
3.
Eder M, Pierce JR Jr. Innovations in faculty development: study of a research assistance unit designed to assist clinician-educators with research. South. Med. J. 104(9), 647–650 (2011).
PubMed
Google Scholar
• Specific to research assistants, though in a more clinical sense.
4.
Manring MM, Panzo JA, Mayerson JL. A framework for improving resident research participation and scholarly output. J. Surg. Ed. 71(1), 8–13 (2014).
PubMed
Google Scholar
5.
Robbins L, Bostrom M, Marx R, Roberts T, Sculco TP. Restructuring the orthopedic resident research curriculum to increase scholarly activity. J. Grad. Med. Ed. 5(4), 646–651 (2013).
PubMed
Google Scholar
6.
Journal Citation Reports Science Edition. Clarivate Analytics (2020). https://jcr.clarivate.com/jcr/home?Init=Yes&SrcApp=IC2LS
Google Scholar
7.
Hanlon MC, Ludbrook J, Jovanovic K, Greer P, Martin JM. Fostering a culture of research within a clinical radiation oncology department. J. Med. Imaging Radiat. Oncol. 62(1), 102–108 (2018).
PubMed
Google Scholar
• Specifically concerning developments beyond clinical research coordinators. How to foster a culture is a difficult task.
8.
Chen AF, Secrist ES, Scannell BP, Patt JC. Matching in orthopaedic surgery. J. Am. Acad. Orthop. Surg. 28(4), 135–144 (2020).
PubMed
Google Scholar
9.
Grabowski G, Walker JW. Orthopaedic fellowship selection criteria: a survey of fellowship directors. J. Bone Joint Surg. Am. 95(20), e154 (2013).
PubMed
Google Scholar
10.
Matthews CN, Estrada DC, George-Weinstein M, Claeson KM, Roberts MB. Evaluating the Influence of research on match success for osteopathic and allopathic applicants to residency programs. J. Am. Osteopath. Assoc. 119(9), 588–596 (2019).
PubMed
Google Scholar
|