Characteristics, completion and publication of PROSPERO records in regional anesthesia for acute perioperative pain
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim: We analyzed characteristics and completion status of Systematic Reviews (SRs) about regional anesthesia for acute perioperative pain relief registered in PROSPERO. Materials & methods: PROSPERO was searched on 5 January 2022. Characteristics of PROSPERO records, completion status and publication information were extracted. PubMed and Google Scholar were searched by 31 May 2022, for additional published SRs. The inconsistency of PROSPERO records' status was analyzed by comparing the most recent PROSPERO status with subsequent publication of completed SR. Time-to-publication was calculated. Results: Almost half of 174 included PROSPERO records (49%) were registered in 2021. A median of 3 (IQR: 2–4.75) authors, and a search of 4 information sources (IQR: 3–5) was planned. At first registration, 51 (29%) records had already started or completed formal screening. PROSPERO records included mostly adults (48%) undergoing trunk blocks (56%), with pain intensity (74%) as a main outcome. 147 (84%) PROSPERO records had a status “ongoing”. Yet, 71 (41%) PROSPERO records were completed SRs, published after a median of 291 days. Inconsistency in status was found in 34% of PROSPERO records. Conclusion: Despite an increase in PROSPERO registrations about regional anesthesia, most were not published, and a third had inaccurate status.
Plain language summary
What is this article about?
Systematic Reviews are considered the cornerstone for providing evidence if a treatment is effective and safe for patients. Registration of a systematic review protocol in a publicly available database, such as PROSPERO, is essential for providing transparency, avoiding duplicate, and wasteful research. Pain after surgery is still not managed well, causing patient suffering, impaired recovery, opioid dependence and development of long-term pain. Regional anesthesia provides effective relief of pain after surgery. In this study, we investigated protocols of Systematic Reviews about regional anesthesia and surgical pain relief that were registered in PROSPERO to see how authors planned to conduct their research, if planned reviews were completed and published in journals, and whether the completion status of the PROSPERO records was accurate.
What were the results?
We found an increasing number of registered protocols about regional anesthesia in PROSPERO during the last 10 years. The protocols analyzed mostly adults (48%) that received regional anesthesia of the trunk (56%). The efficacy of blocks was frequently measured with pain intensity (74%). Most protocols were not published as completed Systematic Reviews in a journal and a third had inaccurate status.
What do the results of the study mean?
Although increasingly more protocols for regional anesthesia are registered in PROSPERO, most are not published in a journal. A better system for updating PROSPERO registrations should be developed so another research group can tell if the topic is abandoned, and in such cases, take over and investigate a relevant research question.
Systematic Reviews (SRs) are evidence syntheses that summarize the relevant studies, which address a question of interest, in a systematic, reproducible fashion, and may help busy practitioners stay up to date with the current medical knowledge [1]. SRs of randomized controlled trials (RCTs) are considered the highest level of evidence for clinical decision-making regarding interventions [1,2], and are used to support the development of clinical practice guidelines [3–6].
To ensure consistent and transparent conduct of an SR, the authors should develop a protocol and register it before the review starts [3].
A priori registration also aims to minimize reporting bias and enables authors of future SRs to identify existing protocols of ongoing SRs, avoiding duplication of the work [7]. There are several possibilities for a priori SR registration, such as publication in a scholarly journal, or publication in one of the following registries: PROSPERO, the Registry of Systematic Reviews/Meta-Analyses in Research, INPLASY, Open Science Framework Registries, and protocols.io [8]. There are also SR registers that are not open to all authors, such as the Campbell Collaboration, which produces SRs of social interventions, and the Cochrane, which registers only SRs of healthcare interventions performed within the Cochrane [8].
PROSPERO is an international prospective register of SRs developed in 2011 by the Centre for Reviews and Dissemination at the University of York in the UK. It allows free registration and public access to SR protocols in all health areas [7]. The content of submitted SR registrations is checked before being registered. Registration should, ideally, take place before the researchers have started formal screening against inclusion criteria, but registrations in PROSPERO are accepted if they have not moved beyond the completion of data extraction [7]. PROSPERO evaluates registries for the completeness and eligibility criteria, but it does not appraise the quality of the proposed SR [9].
A study by Andrade et al. showed that only 7% of SRs registered in PROSPERO between 2011 and 2015 have had their status updated to published, indicating they were either ongoing, or their status has not been updated (either completed or abandoned) [9]. Additionally, a study by Runjic et al. [10] showed that around half of SRs about pain registered in PROSPERO were still unpublished by August 2018, at least 1.3 years after registration. Their results were based on PROSPERO status, a survey of corresponding authors, as well as database searches. A high proportion of unpublished reviews could result in a waste of resources and potentially useful information.
To our knowledge, the characteristics of SRs of RCTs registered in PROSPERO in the field of regional anesthesia for acute perioperative pain relief have not been previously investigated. It is also unknown what proportion of SRs of RCTs in regional anesthesia for acute perioperative pain relief registered in PROSPERO are completed and published. We, therefore, conducted this study to address these two intertwined questions.
The first aim of this study was to evaluate the characteristics of SRs of RCTs in the field of regional anesthesia for acute perioperative pain relief registered in PROSPERO. The second aim of this study was to evaluate the completion status (i.e., whether an SR was completed) and publication status (i.e., whether a completed SR was published in a scholarly journal) of SRs of RCTs in the field of regional anesthesia for acute perioperative pain relief registered in PROSPERO.
Methods
Study design
This was a primary methodological study.
Eligibility criteria
We included in the study SRs of RCTs evaluating regional anesthesia for acute perioperative pain relief registered in PROSPERO. SRs were defined as reviews that addressed a clearly defined research question, used systematic methods to search, screen, select, and appraise the literature; and extracted and analyzed/synthesized data from the included studies [11].
Only registrations of SRs of RCTs of interventions were included; other study designs were not eligible. Participants treated for acute perioperative pain were included without age, gender, or race restriction. The intervention of regional anesthesia included epidural, spinal, combined spinal/epidural anesthesia, caudal epidural, and peripheral nerve blocks. Dental blocks and intravenous regional anesthesia technique were not included. All comparators were eligible: another regional anesthesia technique, infiltration anesthesia, and non-exposed groups with or without placebo/sham block were included. Comparisons of different nerve block adjuvants were excluded since the main focus of such studies is the duration of the block and not analgesia. Only SR registrations that evaluated primary outcome measures of perioperative pain/pain relief, including analgesic efficacy as defined by the authors were eligible.
PROSPERO search & screening
The PROSPERO was searched using the words “nerve block”, “regional anesthesia” and “regional anaesthesia” and filters for systematic review and meta-analysis on January 5, 2022. The search strategy used was: “nerve block OR regional anesthesia OR regional anaesthesia AND (Systematic Review OR Meta-Analysis):RT” and MeSH (Medical Subject Headings) terms “Anesthesia, Conduction” (MeSH DESCRIPTOR Anesthesia, Conduction EXPLODE ALL TREES). Results of the 2 searches were exported to Microsoft Excel® (Microsoft Corp., Redmond, WA, USA) sheet, and duplicates were removed.
Subsequently, the records were screened to determine their eligibility. Before the formal screening process, a calibration exercise was performed between two authors (MC, SD) on 5% of the records. Subsequently, one author screened all remaining search results for eligibility (SD), and the other author (MC) screened 20% of random search results. Discrepancies were resolved by discussion, when necessary.
Systematic Review completion status & publication status
The completion and publication status of PROSPERO registrations was extracted from registered SRs. The completion status was extracted from the PROSPERO field “Stage of Review”, which has the following possible categories: “review ongoing”, “review completed, not published”, “review completed and published”, “review completed, published, and being updated” and “review discontinued”.
For SR registrations labeled as published, information about the publication from the PROSPERO field “Details of final report/publication(s) or preprints if available” was extracted.
If the SR was labeled as published, but there were incomplete details about its publication, the literature was searched to identify the SR publication in a scholarly journal.
Publication identification
We searched PubMed and Google Scholar® to find articles published in scholarly journals reporting SRs whose SR registrations were labeled in PROSPERO as “ongoing” or “completed, not published”, and “completed, published”, and for which publication details were not provided in PROSPERO, by 31 May 2022. Two authors (MC, MČ) searched for journal publications corresponding to the included PROSPERO records. Another author (SD) repeated the same search for possible publications that were not found. The investigators (MC, MČ, SD) calibrated their search approaches before searching for published SRs [12].
The searches were conducted by combining keywords from the registered SR title and last names of the first, second, and last author. If unsuccessful, the search was also attempted by a unique PROSPERO CRD number. PROSPERO registration and publication were a match if the unique PROSPERO CRD number was equal. If the CRD number was unavailable in the publication, we considered that the PROSPERO registration and publication were a match if at least one of the authors, title, and objective were the same. In case there were multiple published SRs corresponding to a single PROSPERO registration, we planned to exclude them; however, we did not have any such cases.
Publication status and analysis of time to publication were conducted for all PROSPERO records. Time to publication was defined as the time from registration of the first version in PROSPERO to the first publication date (electronic, if available) of the completed SR.
Subanalyis of publication status was performed for the records registered before 28 February 2021, with “ongoing” statuses at the time of analysis, to allow for a time lag between registration and completion of a review. The time frame was based on the previous work by Borah et al., which showed a median estimated time of 67.3 weeks for the completion and publication of PROSPERO records [13].
Inconsistency in PROSPERO records status
The inconsistency of PROSPERO records' status was analyzed by comparing the most recent PROSPERO status with subsequent publication of a completed SR. Following scenarios were possible for inconsistency to exist: PROSPERO status was “ongoing”, and completed, and published SR was identified, PROSPERO status was “completed, not published”, and completed, and published systematic review was identified.
Data extraction
Two authors extracted the data (MC, MČ) into a Microsoft Excel® sheet. The third author (SD) checked all entries for inconsistencies, updated entries, and resolved discrepancies, when necessary.
We extracted the following data from the included PROSPERO registrations: PROSPERO identification number, registration date, number of registered versions, all authors and affiliated countries, number of authors with affiliations in multiple countries, contact email address, participants examined, interventions, comparators, all outcomes (primary outcomes and number of additional outcomes including safety outcomes), publication status of eligible studies (published, unpublished, or both), the language of eligible studies (English, all languages, other), electronic information sources searched, additional searches (trial registries, gray literature), planned date of searches, planned risk of bias (quality) assessment, planned certainty of evidence assessment, planned data synthesis and heterogeneity assessment, planned assessment of meta-biases (publication bias assessment, selective reporting), date of the most recent record version, anticipated or actual start date, anticipated completion date, funding source, type of funding source (government, internal (department), non-profit organization, private), and completion status.
Data on Journal Citation Reports (JCR) Journal Impact Factor for the year 2020 and publication model (open access, hybrid, traditional model) were extracted from the journals' websites. When Impact Factor was unavailable on the journal websites, JCR database was searched to obtain the missing data.
Statistical analysis
Descriptive statistics were used for data analysis. Continuous data were presented as the median and interquartile range (IQR), and categorical data as frequencies and percentages.
Publication analysis of PROSPERO records was based on dichotomous data (published vs not published). The time to publication was presented in days.
Analysis was conducted in Microsoft Excel® (Microsoft Corp., WA, USA) software.
Results
Search results & screening
PROSPERO search yielded 928 SRs registered about regional anesthesia. After removing 144 duplicate records, a total of 784 unique PROSPERO records were found. Records were screened for eligibility; we excluded 610 records that were not eligible, and 174 unique records were included (Figure 1). Characteristics of PROSPERO records are presented in Table 1.
| Characteristics | Results |
|---|---|
| Registration year, n (%) 2012 2015 2016 2017 2018 2019 2020 2021 | 1 (0.6%) 4 (2.3%) 2 (1.1%) 5 (2.9%) 6 (3.4%) 22 (13%) 49 (28%) 85 (49%) |
| Authors, n (%) 1 2 3 4 5 ≥6 | 15 (8.6%) 47 (27%) 38 (22%) 30 (17%) 22 (13%) 22 (13%) |
| Authors, median (IQR) | 3 (2–4.75) |
| Country, n (%) China India England Switzerland South Korea United States Canada Germany Other countries | 85 (49%) 8 (4.6%) 8 (4.6%) 7 (4%) 7 (4%) 7 (4%) 6 (3.4%) 5 (2.9%) 41 (24%) |
| Planned electronic database search, n (%) Not stated 1 2 3 4 5 ≥6 | 1 (0.6%) 1 (0.6%) 3 (1.7%) 43(25%) 60 (34%) 36 (21%) 30 (17%) |
| Planned electronic database searches, median (IQR) | 4 (3–5) |
| Planned search of additional sources, n (%) Yes Not stated No | 94 (54%) 79 (45%) 1 (0.6%) |
| Additional sources, n (%) References Clinical trial registries Gray literature | n = 94 61 (65%) 36 (38%) 21 (22%) |
| Planned search date stated, n (%) Yes No | 106 (61%) 68 (39%) |
| Eligible trials based on publication status, n (%) Published Published and unpublished Not stated | 94 (54%) 62 (36%) 18 (10%) |
| Eligible trials based on publication language, n (%) No restriction Not stated English only Other (English and another language) | 80 (46%) 39 (22%) 36 (21%) 19 (11%) |
| Population, n (%) Adults (≥18 years) Not stated Females Children All ages | 84 (48%) 50 (29%) 20 (11%) 14 (8%) 6 (3.4%) |
| Planned main outcomes, n (%) 1 2 ≥3 | 94 (54%) 50 (29%) 30 (17%) |
| Planned main outcomes, median (IQR) | 1 (1–2) |
| Measure of effect for primary outcome stated, n (%) Yes No Not applicable | 149 (86%) 23 (13%) 2 (1.1%) |
| Planned additional outcomes, n (%) 0 1 2 ≥3 | 11 (6.3%) 24 (14%) 24 (14%) 115 (66%) |
| Planned additional outcomes, median (IQR) | 3 (2–5) |
| Planned risk of bias assessment, n (%) Yes | 174 (100%) |
| Tool for risk of bias assessment, n (%) Cochrane Risk of bias tool Cochrane Risk of bias tool 2.0 Jadad scale Other Not stated | 140 (80%) 18 (10%) 6 (3.4%) 8 (4.6%) 2 (1.1%) |
| Planned quality of evidence assessment, n (%) Not stated Yes | 147 (84%) 27 (16%) |
| Tool for quality of evidence assessment, n (%) GRADE GRADE and CINeMA | n = 27 26 (96%) 1 (3.7%) |
| Planned heterogeneity assessment, n (%) Yes Not stated | 124 (71%) 50 (29%) |
| Planned publication bias assessment, n (%) Yes Not stated No | 105 (60%) 68 (39%) 1 (0.6%) |
| Planned subgroup analysis, n (%) Yes Not planned Not stated/not applicable | 133 (76%) 38 (22%) 3 (1.7%) |
| Funding, n (%) Yes No | 52 (30%) 122 (70%) |
| Funding source, n (%) Government Internal resources Internal resources and government Nonprofit organisation and government Privatne funding | n = 52 24 (46%) 23 (44%) 3 (5.8%) 1 (1.9%) 1 (1.9%) |
| Conflict of interest, n (%) Not stated None known Conflict reported | 142 (82%) 28 (16%) 4 (2.3%) |
| Registered PROSPERO versions, n (%) 1 2 ≥3 | 88 (51%) 73 (42%) 13 (7%) |
| Registered PROSPERO versions, median (IQR) | 1 (1–2) |
| Screening started or completed during first registration, n (%) Yes No | 51 (29%) 123 (71%) |
| Status of latest version of PROSPERO record, n (%) Ongoing Completed, not published Completed, published | 147 (84%) 16 (9.2%) 11 (6.3%) |
CINeMA: Confidence in Network Meta-Analysis; GRADE: Grading of Recommendations Assessment, Development and Evaluation; IQR: Interquartile range.
General characteristics of PROSPERO records
Records were registered over a ten-year period (2012–2021), with a steep increase in the number of registrations from the year 2019 onwards (Figure 2). Almost half of the records (85/174, 49%) were registered in the last year of the analyzed period (Table 1).
Half of the authors registered only 1 version (88, 51%) of the PROSPERO record. At the time of first registration, 51 (29%) had already started or completed formal screening against the eligibility criteria.
The majority of PROSPERO records (148, 85%) reported an anticipated completion date before January 2022 in the last registered version.
The median number of registered authors was 3 (IQR: 2–4.75). The highest number of registrations were from authors affiliated with an institution from China (85/174, 49%).
Of the 174 included PROSPERO records, only one did not report a planned search of electronic information sources. The authors planned to search a median of 4 information sources (IQR: 3–5). The most commonly planned information sources for searching were EMBASE (Excerpta Medica dataBASE) (160, 92%), CENTRAL (Cochrane Controlled Register of Trials) (152, 87%) and PubMed (135, 78%). Among Chinese information sources, commonly searched were WanFang (22, 13%), China National Knowledge Infrastructure (CNKI) (21, 12%), and VIP (Chinese Science and Technology Periodical Database) (10, 5.7%).
Around half of the records indicated a plan to search additional information sources (94/174, 54%); most frequently the references of included studies (61/94, 65%). Planned search date was reported in 106 of 174 included PROSPERO records.
Regarding the publication status and language of included RCTs, around a half of PROSPERO records planned to include only published studies (54%), and less than a half placed no language restriction (46%).
A plan for the risk of bias (methodological quality) assessment in RCTs was specified in all PROSPERO records. Cochrane Risk of bias tool was a leading tool (140, 80%), while the new version (RoB 2.0) was planned in 18 (10%) records.
Conversely, the certainty of evidence assessment was reported as planned in only 27 PROSPERO records (16%), with 26 planning to use the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach and 1 both GRADE and CINeMA (Confidence in Network Meta-Analysis). Assessment of heterogeneity (124, 71%), publication bias (105, 60%) and subgroup analysis (133, 76%) were frequently planned.
The authors reported that there was no funding for the majority of planned SRs (122, 70%). Of 52 records that reported funding, 24 stated government support, 23 institutional funds, 3 institutional funds and governmental support, 1 private funding, and 1 nonprofit organization and government support.
Conflict of interest was stated as present in only 4 (2.3%) records, 28 (16%) declared no known conflict of interest, while the rest did not report anything related to a conflict of interest.
Participants, interventions, comparators & outcomes (PICO)
While the population of interest in most records were adults (84/174, 48%); 50 out of 174 records did not specify the age of eligible participants.
All records described planned interventions and comparators. The most common interventions (Table 2) were trunk blocks (97, 56%), lower extremity blocks (24, 14%) and upper extremity blocks (14, 8%).
| Interventions by regions | n | % |
|---|---|---|
| Trunk block | 97 | 56% |
| Lower limb block | 24 | 14% |
| Upper limb block | 14 | 8% |
| Trunk block, neuraxial anesthesia | 6 | 3.4% |
| Infiltration anesthesia | 6 | 3.4% |
| Head and neck block | 5 | 2.9% |
| Neuraxial anesthesia | 4 | 2.3% |
| Any regional anesthesia | 4 | 2.3% |
| Trunk block, infiltration anesthesia | 3 | 1.7% |
| Not applicable (local anesthetics compared) | 2 | 1.1% |
| Lower limb and neuraxial anesthesia | 2 | 1.1% |
| Nerve block (not specified) | 2 | 1.1% |
| Infiltration anesthesia, trunk block, neuraxial anesthesia | 1 | 0.6% |
| Lower limb block, infiltration anesthesia | 1 | 0.6% |
| Neuraxial anesthesia, lower limb block, infiltration anesthesia | 1 | 0.6% |
| Neuraxial anesthesia, trunk block, infiltration anesthesia | 1 | 0.6% |
| Lower limb block, trunk block | 1 | 0.6% |
The interventions were mostly compared with a peripheral nerve block (40, 23%), no nerve block (12, 6.9%), infiltration anesthesia (9, 5.2%) and neuraxial anesthesia (9, 5.2%). A detailed list of comparators is presented in Supplementary Table 1.
Planned main outcomes were stated in all PROSPERO records, with a median of 1 (IQR: 1–2) planned outcome. Measure of effect for primary outcome was reported in 149 (86%) records.
The most common main outcomes were pain intensity (129, 74%) and cumulative opioid consumption (74, 43%), followed by analgesic drug consumption (21, 12%). A total of 33 different efficacy outcomes were planned (Supplementary Table 2).
Only 23 (13%) protocols planned to measure a safety outcome as one of the main outcomes, most frequently nerve block-related complications (8, 35%). Those are presented in Supplementary Table 3.
One hundred and sixty-three (94%) records also planned additional outcomes, with a median of 3 (IQR: 2–5) outcomes per PROSPERO record.
Completion status
The most recent version of the PROSPERO record was checked for completion status, which was “ongoing” in most records (147, 84%). The status of twenty-seven records indicated that the SRs were completed: 16 records (9.2%) had the status “completed, not published”, while 11 (6.3%) records were labelled as “completed, published”.
Publication status
We found 71 (41%) published articles that reported SRs corresponding to all PROSPERO protocols registered until 1 June 2022. Only 20 (28%) had specified a unique PROSPERO CRD number in the abstract.
In a subanalysis of PROSPERO records registered until 28 February 28 2021. with “ongoing” statuses (n = 79), publication proportion was somewhat higher (39/79, 49%).
Inconsistency of PROSPERO record status
A comparison of the last version of PROSPERO records' statuses with subsequent SR publications that we retrieved via literature search showed inconsistency in 60 (34%) records (Table 3).
| PROSPERO record status | ||||
|---|---|---|---|---|
| PROSPERO record published (no/yes) | Review ongoing, n (%) | Review completed not published, n (%) | Review completed published, n (%) | Total, n (%) |
| No | 97 (66) | 6 (37) | 0 (0) | 103 (59) |
| Yes | 50 (34)† | 10 (63)† | 11 (100) | 71 (41) |
| Total | 147 (100) | 16 (100) | 11 (100) | 174 (100) |
†
Indicates inaccuracy.
PROSPERO: Prospective Register of Systematic Reviews.
Out of 147 records labeled as “ongoing”, 50 (34%) were published. We also found matching SR publications for 10 (63%) out of 16 records labeled as “completed, not published”, while 6 were not published. All 11 records from the “completed, published” category had accurate status (Table 3).
Time to publication & journals
Seventy-one PROSPERO records were published after a median of 291 (IQR: 138–418.5) days. One SR was published 16 days before it was registered in PROSPERO. The longest time from PROSPERO registration to SR publication was 1626 days. The authors of SRs about regional anesthesia for perioperative pain published their work in 42 different journals, with a median JCR Journal Impact Factor 4.585 (IQR: 2.811–6.776). Five journals did not have JCR Journal Impact Factor for 2020.
The most common journals where the SRs from our sample were published were “European Journal of Anaesthesiology” (7, 9.9%), followed by “Anaesthesia”, “Cochrane Database of Systematic Reviews”, and “Journal of Clinical Anesthesia”, each represented with 5 (7%) published SRs.
Based on a publishing model, half of SRs were published in journals with a hybrid model (37, 52%).
Discussion
PROSPERO Systematic Reviews on regional anesthesia
The current study is the first study that investigated the characteristics of PROSPERO registrations of SRs on regional anesthesia for perioperative pain management. Perioperative pain is still not managed well, leading to severe consequences: patient suffering, impaired recovery, opioid dependence and development of chronic pain [14].
This study showed that, over the 10-year period, SRs of randomized controlled trials in regional anesthesia for perioperative pain management were increasingly registered in PROSPERO. The increasing trend is especially noticeable after the year 2018, with the largest number of PROSPERO records registered in the last year of the analyzed period.
This follows the recent development of regional anesthesia, which is a growing specialty, especially since the introduction of ultrasound to regional anesthesia. Multiple new blocks have been recently developed. There is emerging evidence that supports their efficacy for various surgical procedures, although most evidence comes from small studies or studies that did not compare analyzed blocks with the gold standard. It is, therefore, not surprising that new randomized trials are increasingly conducted, which is reflected in increased PROSPERO registrations of SRs of RCTs [15].
Methodological aspects of PROSPERO records in regional anesthesia
The methodological rigor and quality of SRs is, besides systematic and reproducible methodology, also based on the a priori documentation of planned methodical steps in the conduction of review that is available in the form of a protocol [16]. Almost a third of PROSPERO records in regional anesthesia were registered after the screening had already started or was completed, which is not in accordance with PRISMA for protocols (PRISMA-P) guideline [16]. PRISMA-P states that protocol should be registered a priori. PROSPERO, since 1 October 2019, accepts registrations that have not progressed beyond the screening phase, although previously it also accepted registrations if the reviewers had not completed data extraction [17].
Furthermore, one SR was retrospectively registered in PROSPERO, 16 days after publication in a scholarly journal. Even though prospective registration of SR protocols is not a mandatory requirement of biomedical journals, it is important, just like prospective registration of clinical trial protocols, as SRs of randomized controlled trials have a key role in evidence-based clinical medicine [8]. It is important to emphasize that the date of registration in PROSPERO may not have been the fault of the authors at all times. In 2019, major delays with the registration of new SRs in the PROSPERO were described, with more than 6 months from the submission to registration and publication. The PROSPERO only recently started publishing the submission date together with the registration date [18].
Our study also identified serious weaknesses in the planned SRs. For example, we found 15 PROSPERO records with only 1 planned author, and 1 PROSPERO record that planned to search only one electronic information source. Studies with such methodology should not be considered SRs [19].
Only half of the included PROSPERO records planned additional searches, most frequently the references of included studies. The study conducted in the area of physical therapy in cancer patients showed the additional value of searching references and contacting expert authors from the field [20].
Another study also showed that contact with authors is particularly important for obtaining unpublished results [21]. Contact with authors was not planned in any of the analyzed PROSPERO records.
A third of the authors did not set limitations based on the publication status of eligible RCTs. Furthermore, only a fifth of authors planned to search “gray literature”, even though it is considered a relevant and important source of data in SRs, and its' inclusion has the potential to reduce the risk of publication bias [22].
All registered PROSPERO records in regional anesthesia planned to analyze the risk of bias in included trials, mostly with the Cochrane Risk of Bias tool. The majority planned for the assessment of heterogeneity, publication bias, and subgroup analysis. On the contrary, the certainty of evidence assessment was planned in only 15% of records.
It is acknowledged that PROSPERO registrations could contain insufficient information because authors may use minimal necessary information needed for a review to be registered. Therefore, the information in a PROSPERO registration may not be an indication of the quality of a completed review. Our intention was not to draw any conclusions about final, completed reviews. Our analysis was focused only on PROSPERO registrations.
Interventions & outcomes
The results of this study show that the vast majority of registered SRs focused on peripheral nerve blocks, while neuraxial anesthesia was infrequently investigated. The trends in the use of peripheral nerve blocks and neuraxial anesthesia have been described by Cozowicz et al. [23] in orthopedic surgery. They compared the period of 2006–2007 with 2012–2013, and showed a decrease in neuraxial anesthesia use in total hip and total knee arthroplasty, compared with an increase in peripheral nerve block use [23].
The most common main outcomes in PROSPERO records about regional anesthesia for perioperative pain management were pain intensity (74%), cumulative opioid consumption (42%) and analgesic drug consumption (12%). Our results show that a total of 33 different outcomes were planned to assess the efficacy of regional anesthesia interventions in analyzed PROSPERO records. Similar to our findings, a recent SR on the use of pain outcomes in regional anesthesia trials demonstrated that pain was reported inconsistently [24]. The most common outcome measures used were pain intensity and total opioid consumption. High heterogeneity of outcomes was reported, with 31 included trials using a total of 15 different pain outcomes [24].
Additionally, few PROSPERO records planned to use a safety outcome as one of the main outcomes. These findings represent a lack of standardization of outcomes relevant to all aspects of regional anesthesia and call for a core outcome set (COS) development in regional anesthesia. The development of a core outcome set is needed to accurately measure and compare the findings of different trials in regional anesthesia [25].
Upon publication of a COS, monitoring of the COS uptake will be needed. It has been shown before that authors may not use the existing COS and that they may not be aware of it [26,27]. Since the registration of an SR should precede the publication of a full report in a scholarly journal, monitoring COS uptake can be done by analyzing outcomes planned in the SR registrations. Thus, our study can serve as a benchmark for monitoring the uptake of future COS.
Completion, publication, & inconsistencies in PROSPERO records' statuses
The majority of SRs in regional anesthesia for perioperative pain management registered in PROSPERO were not published by 1 June 2022, even though 85% reported the anticipated completion date before the date of our search (January 2022) in the latest version of PROSPERO records.
Specifically, when looking into all records, we found that 41% of registered SRs about regional anesthesia for perioperative pain were published in scholarly journals after a median of 291 days (42 weeks). Even when restricting the analysis to records with ongoing statuses registered until 28 February 2021, thus allowing around 15 months for completion and publication, the publication proportion was still less than 50%.
Similar to our findings, a study by Runjic et al. showed that only around half of registered PROSPERO records about pain had been published. In their sample, 1.3 years or more after registration had passed until publication [10]. Andersen et al. [28] analyzed 2475 published SRs and meta-analyses that had a protocol registered in PROSPERO, and also found longer time from registration to publication (median: 16 months; IQR: 10–24 months) compared with our sample.
Another relevant finding was the inconsistency in the status of PROSPERO records. When the most recent version of the PROSPERO record was checked for completion status, the status was “ongoing” in most records (84%). Obviously, that status was not up to date, as we found that 41% of all analyzed PROSPERO records were completed and published in scholarly journals by 1 June 2022. The inconsistency was highest (63%) in the subgroup of records with the status “completed, not published”.
Several studies also showed that many statuses of PROSPERO records had not been updated after the publication of SRs [10,28–30]. Authors of registered PROSPERO records should regularly update their status for other authors to know if the review was abandoned or is still ongoing. That means that other authors could take over the abandoned topics which may be relevant to stakeholders [9,28]. In that respect, it would be useful if PROSPERO would occasionally send a reminder (for example, once a year) to the authors of SR registration to prompt them to adjust the status of their SR if its status has changed in the meantime. Alternatively, PROSPERO could implement automated literature searches, which will help with updating the SR status if the corresponding publication is found in the literature. For example, that is done by a preprint server medRxiv; if a potential journal publication is found and considered a match for a published preprint article, the medRxiv will send an email to the corresponding author, requesting confirmation of the link [31].
Limitations
This study only analyzed regional anesthesia SRs registered in PROSPERO and did not aim to represent all SRs in the field of regional anesthesia, or reviews registered in other available registries. Not all SRs are registered in PROSPERO. A study of SRs published in 2018 indicated that 32% were registered in PROSPERO [30]. A study of SRs published in 2020 and 2021 showed that 38% of non-Cochrane reviews had a protocol either registered, published, or both. PROSPERO was mostly used for protocol registration (96%) [32]. Thus, we acknowledge that our study likely did not include registrations of all SRs that are conducted in the targeted field. This could be considered a selection bias in our sample. However, we have analyzed all the eligible protocols that were available in PROSPERO.
We only sought SRs that were published in scholarly journals. Some PROSPERO records could have been completed and published in “gray literature” repositories, including the databases of preprint articles, such as medRxiv. Therefore, the number of completed and published PROSPERO records could be higher. Subsequently, the proportion of PROSPERO records with inconsistent statuses could also be higher than estimated based on the search for published SRs.
Additionally, some publications might have been missed. Since only about a quarter of published SRs had specified a unique PROSPERO CRD number in the abstract, it was not possible to identify SRs that possibly changed planned PICO question or planned authors. The authors should report their abstracts, including protocol registration, according to PRISMA for Abstracts guidelines [33].
A relatively short follow-up time is another limitation of this study. It is possible that the publication rate would be even higher if a longer time than 15 months from registration was allowed. However, the principal analysis was focused on the characteristics of SR registrations, for which publication status is not relevant.
Conclusion
In conclusion, there is room for improvement in planned methodology of SRs in the field of regional anesthesia for acute perioperative pain registered in PROSPERO. Despite a significant increase in PROSPERO registrations during the last ten years, most of them were not published, and a third did not have an accurate status. A better system for updating PROSPERO registrations should be developed.
•
This is the first study that investigated the characteristics of PROSPERO registrations of systematic reviews (SRs) on regional anesthesia for perioperative pain management.
•
PROSPERO database was searched for registered SRs about regional anesthesia and perioperative pain, from inception to 5 January 2022.
•
Characteristics of PROSPERO records, accuracy, completion status, publication status and time-to-publication were analyzed.
•
A total of 174 PROSPERO records were included; 84% had “ongoing” status.
•
The population of interest were mostly adults (48%), the most common interventions were trunk blocks (56%); and pain intensity (74%) was mostly used as a main outcome.
•
A total of 33 different outcomes were planned to assess the efficacy of regional anesthesia interventions in analyzed PROSPERO records, indicating high heterogeneity.
•
Almost a third of PROSPERO records in regional anesthesia were registered after the screening had already started or was completed, which is not in accordance with PRISMA for protocols (PRISMA-P) guideline.
•
Serious weaknesses in the planned SRs were found: 15 PROSPERO records had only 1 planned author and 1 PROSPERO record planned to search only 1 electronic information source. Studies with such methodology should not be considered SRs.
•
We found 71 (41%) published SRs corresponding to the registered PROSPERO records, published a median of 291 days after registration.
•
Inaccurate status was found in 34% of PROSPERO records. A better system for updating PROSPERO registrations should be developed.
Acknowledgments
This study is based on a Diploma thesis of the first author, Martina Carev, which was written and defended in the Croatian language at the University of Split School of Medicine, Split, Croatia (July 2022) [34].
Financial & competing interests disclosure
The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this manuscript.
Supplementary Material
File (supplementary data.docx)
- Download
- 30.66 KB
References
Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Cook DJ, Mulrow CD, Haynes RB. Systematic Reviews: synthesis of best evidence for clinical decisions. Ann. Intern. Med. 126(5), 376–380 (1997).
2.
Finckh A, Tramèr MR. Primer: strengths and weaknesses of Meta-Analysis. Nat. Clin. Pract. Rheumatol. 4(3), 146–152 (2008).
3.
Moher D, Shamseer L, Clarke M et al. Preferred reporting items for Systematic Review and Meta-Analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 4(1), 1 (2015).
4.
Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, Van Der Wees P. Guidelines International Network: toward international standards for clinical practice guidelines. Ann. Intern. Med. 156(7), 525–531 (2012).
5.
Hollon SD, Areán PA, Craske MG et al. Development of clinical practice guidelines. Ann. Rev. Clin. Psychol. 10(1), 213–241 (2014).
6.
Van Der Wees P, Qaseem A, Kaila M et al. Prospective Systematic Review registration: perspective from the Guidelines International Network (G-I-N). Syst. Rev. 1(1), 3 (2012).
7.
Booth A, Clarke M, Dooley G et al. The nuts and bolts of PROSPERO: an international prospective register of Systematic Reviews. Syst. Rev. 1(1), 2 (2012).
•• Important insights into the PROSPERO register.
8.
Pieper D, Rombey T. Where to prospectively register a Systematic Review. Syst. Rev. 11(1), 8 (2022).
•• Important insights into prospective registration of Systematic Review protocols.
9.
Andrade R, Pereira R, Weir A, Ardern CL, Espregueira-Mendes J. Zombie reviews taking over the PROSPERO Systematic Review registry. It's time to fight back! Br. J. Sports Med. 53(15), 919–921 (2019).
• Description of abandoned Systematic Review protocols in PROSPERO.
10.
Runjic E, Rombey T, Pieper D, Puljak L. Half of Systematic Reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J. Clin. Epidemiol. 116, 114–121 (2019).
• Study on the status of PROSPERO records in the field of pain registered in PROSPERO and their subsequent publication.
11.
Centre for Reviews and Dissemination, University of York. Systematic Reviews: CRD's guidance for undertaking reviews in healthcare. York Publishing Services Ltd, UK (2009). www.york.ac.uk/media/crd/Systematic_Reviews.pdf (Accessed: 16 December 2022).
12.
Livas C, Pandis N, Ren Y. Full-text publication of abstracts presented at European Orthodontic Society congresses. Eur. J. Orthod. 36(5), 569–575 (2014).
13.
Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct Systematic Reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2), e012545 (2017).
14.
Pogatzki-Zahn EM, Liedgens H, Hummelshoj L et al. Developing consensus on core outcome domains for assessing effectiveness in perioperative pain management: results of the PROMPT/IMI-PainCare Delphi Meeting. Pain 162(11), 2717–2736 (2021).
15.
Sari S, Kumar J, Turan A. New peripheral nerve blocks and local anesthetics. Curr. Opin. Crit. Care 27(6), 733–742 (2021).
16.
Shamseer L, Moher D, Clarke M et al. Preferred reporting items for Systematic Review and Meta-Analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 350, g7647 (2015).
17.
PROSPERO. International prospective register of Systematic Reviews. www.crd.york.ac.uk/prospero/#aboutregpage (Accessed: 16 December 2022).
18.
Puljak L. Delays in publishing Systematic Review registrations in PROSPERO are hindering transparency and may lead to research waste. BMJ Evid. Based Med. 26(6), e4 (2021).
19.
Puljak L. If there is only one author or only one database was searched, a study should not be called a Systematic Review. J. Clin. Epidemiol. 91, 4–5 (2017).
20.
Stevinson C, Lawlor DA. Searching multiple databases for Systematic Reviews: added value or diminishing returns? Complement. Ther. Med. 12(4), 228–232 (2004).
21.
Young T, Hopewell S. Methods for obtaining unpublished data. Cochrane Database Syst. Rev. 2011(11), Mr000027 (2011).
22.
Paez A. Grey literature: an important resource in Systematic Reviews. J. Evid. Based Med. (2017).
23.
Cozowicz C, Poeran J, Zubizarreta N, Mazumdar M, Memtsoudis SG. Trends in the use of regional anesthesia: neuraxial and peripheral nerve blocks. Reg. Anesth. Pain Med. 41(1), 43–49 (2016).
24.
Pushpanathan E, Setty T, Carvalho B, Sultan P. A Systematic Review of postoperative pain outcome measurements utilised in regional anesthesia randomized controlled trials. Anesthesiol. Res. Pract. 2018, 9050239 (2018).
•• Systematic Review on the heterogeneity of pain outcomes used in regional anesthesia randomized trials.
25.
Johnston DF, Turbitt LR. Defining success in regional anaesthesia. Anaesthesia 76(S1), 40–52 (2021).
26.
Dosenovic S, Nikolic Z, Ivancev B, Jelicic Kadic A, Puljak L. Awareness and acceptability of initiative on methods, measurement, and pain assessment in clinical trials core outcome set for chronic pain among surveyed neuropathic pain authors. J. Comp. Eff. Res. 8(9), 671–683 (2019).
27.
Boric K, Boric M, Dosenovic S et al. Authors' lack of awareness and use of core outcome set on postoperative pain in children is hindering comparative effectiveness research. J. Comp. Eff. Res. 7(5), 463–470 (2018).
28.
Andersen MZ, Fonnes S, Andresen K, Rosenberg J. Most published meta-analyses were made available within two years of protocol registration. Eur. J. Integrative Med. 44, 101342 (2021).
29.
Tsujimoto H, Tsujimoto Y, Kataoka Y. Unpublished Systematic Reviews and financial support: a meta-epidemiological study. BMC Res. Notes 10(1), 703 (2017).
30.
Rombey T, Doni K, Hoffmann F, Pieper D, Allers K. More Systematic Reviews were registered in PROSPERO each year, but few records' status was up-to-date. J. Clin. Epidemiol. 117, 60–67 (2020).
• Study investigating the status of Systematic Reviews registered in PROSPERO.
31.
medRxiv. Frequently Asked Questions (FAQ). (Accessed: 16 December 2022). www.medrxiv.org/about/FAQ
32.
Van Der Braak K, Ghannad M, Orelio C et al. The score after 10 years of registration of Systematic Review protocols. Syst. Rev. 11(1), 191 (2022).
33.
Beller EM, Glasziou PP, Altman DG et al. PRISMA for abstracts: reporting Systematic Reviews in journal and conference abstracts. PLOS Med. 10(4), e1001419 (2013).
34.
Carev M. Characteristics of Systematic Reviews of randomized controlled trials evaluating regional anesthesia for acute perioperative pain relief registered in PROSPERO. Diploma thesis. University of Split School of Medicine. Split, Croatia (July 2022). https://urn.nsk.hr/urn:nbn:hr:171:226684
Information & Authors
Information
Published In
Copyright
© 2023 Becaris Publishing Limited.
History
Received: 15 July 2022
Accepted: 9 January 2023
Published online: 23 January 2023
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Characteristics, completion and publication of PROSPERO records in regional anesthesia for acute perioperative pain. (2023) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2022-0129
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Judit Németh, Daniela Küllenberg de Gaudry, László Czina, Szimonetta Lohner, Reporting completeness and outcomes of the diet- and nutrition-related systematic review planning process: protocol for a meta-epidemiological study, BMJ Open, 10.1136/bmjopen-2025-108561, 16, 3, (e108561), (2026).
- Adérito Seixas, Prospective protocols in evidence synthesis: building transparency and accountability, Athena Health & Research Journal, 10.62741/ahrj.v3iSuppl.137, 3, Suppl, (2026).
- E. Barrett-Catton, E. P. Jones, R.B. Carlson, Effect of Citation Numbers and Team Members on the Possibility of and Time Needed to Complete Screening for Systematic and Scoping Reviews, Medical Reference Services Quarterly, 10.1080/02763869.2025.2471879, 44, 2, (119-130), (2025).
- Mariana A. Burgos, Diego Ivaldi, Gisela Oltra, Camila M. Escobar Liquitay, Luis Garegnani, Meta‐epidemiological study on the publication rate of South American countries' systematic reviews of interventions registered in PROSPERO, Cochrane Evidence Synthesis and Methods, 10.1002/cesm.70002, 2, 9, (2024).
- Andrija Babić, Ognjen Barcot, Tomislav Visković, Frano Šarić, Aleksandar Kirkovski, Ivana Barun, Zvonimir Križanac, Roshan Arjun Ananda, Yuli Viviana Fuentes Barreiro, Narges Malih, Daiana Anne‐Marie Dimcea, Josipa Ordulj, Ishanka Weerasekara, Matteo Spezia, Marija Franka Žuljević, Jelena Šuto, Luca Tancredi, Anđela Pijuk, Susanna Sammali, Veronica Iascone, Thilo von Groote, Tina Poklepović Peričić, Livia Puljak, Frequency of use and adequacy of Cochrane risk of bias tool 2 in non‐Cochrane systematic reviews published in 2020: Meta‐research study , Research Synthesis Methods, 10.1002/jrsm.1695, 15, 3, (430-440), (2024).