Reporting assessment of multicenter clinical trial protocols: A cross-sectional study

Randomized controlled trials (RCTs) are the top evidence for effectiveness research of a new intervention or treatment. Multicenter design is commonly used in RCTs to increase the sample size and improve the external validity, particularly in Phase II or III studies.1 With the globalization of drug development, increasing emphasis is being placed on multicenter randomized controlled trials (MRCTs).2 However, MRCTs reach their full potential only if they are designed, conducted, and reported appropriately. In our previous survey in 2021, we have analyzed 2844 final reports of MRCTs during 1975–2019 and found that several multicenter-related information was absent or incompletely provided in their publications. To investigate the reason for unsatisfactory reporting level of MRCTs, we also identified that only 19% studies provided their trial protocol that can be accessed, and 11% descripted the consistency or disparity of protocols across centers.3 Given the importance of trial protocols, an international group of stakeholders developed the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials) to provide a basic guidance for the reporting of protocol.4, 5 However, it is unclear whether the reporting of published MRCT protocols are completeness according to the SPIRIT. Furthermore, no previous study has assessed whether multicenter-related aspects are fully considered in these protocols. In particular, some issues include the criteria and responsibilities of participating centers, the quality control regarding administration of intervention(s) and measurements of outcome(s) across centers, the plans of data collection, management and monitoring across centers, and the heterogeneity analysis in the statistical analysis plan (SAP) that are critical for promoting transparency of an MRCT. Therefore, this review aimed to evaluate the reporting characteristics of MRCT protocols based on the SPIRIT 2013 checklist and on a specially designed multicenter-specific checklist. In this review, we defined the protocol of an MRCT as a randomized controlled clinical trial where the data of participants will be collected from more than one center. Accordingly, MRCT protocols published in English or Chinese up to 11 November 2022 were included with no restriction to conditions, interventions, and controls. Except for the exclusion of interim analysis, results publications, single-center studies, nonrandomized or noncontrolled studies, reviews, case reports, and nonhuman studies, we also exclude repeat records, protocols not published as full text (such as abstracts, full reports cannot be accessed or withdrawn), and those unavailable in English or Chinese language. A systematic search was conducted in the following databases: (1) English databases: Ovid of All EBM Reviews, Embase, and Ovid MEDLINE(R). Some duplicates or nonfull text were preliminary removed by Ovid. (2) Chinese databases: CNKI, VIP, and Wanfang. The original search time was up to 7 July 2021, and an updated search to 11 November 2022. The restriction of language was English or Chinese. Search strategy is presented in Supplementary Table S1. Two reviewers (ZYL and LZ) independently screened the titles and abstracts of the retrieved records based on the inclusion and exclusion criteria. Full reports of any potentially relevant papers were reviewed for further assessment of eligibility. Disagreements were settled via discussion or by consulting a third reviewer (XZ). Data extraction was conducted by two reviewers (ZYL and NNW) independently using a predesigned form, which contained the following elements: types of study design (e.g., assignment, randomization, blinding, and sample size), features of participating centers (e.g., number and distribution of centers, international or national trials), types of diseases based on ICD-11, categories of intervention(s) and control(s), funding sources, and trial registration. The reporting quality was evaluated according to (1) the SPIRIT 20136 and (2) a special-designed multicenter checklist, which was developed by four researchers (ZXB, CY, XZ, and TXW) based on discussion (Table 1). Scoring rules are provided in Supplementary Tables S2 and S3. Generally, each item/question was scored in terms of two possibilities: “1” for “fully reported” or “not applicable,” “0” for “insufficiently reported” or “unreported,” Six reviewers (ZYL, FL, JSD, WTC, CWC, and NNW) were trained and participated the evaluations. During the process, the quality assessment of each protocol was conducted by one author and verified by another author. Possible disagreements were resolved with the consultation of a third senior review author (XZ or ZXB). We applied frequency and percentage to present categorical variables and median and interquartile range (IQR) to present continuous variables. For individual item of reporting quality, the compliance rate was calculated with the number of items acquired “1” based on the total number of included reports, which was further categorized as three levels: excellent compliance (>90%), good compliance (between 65% and 90%), and poor compliance (<65%).7-9 For the overall items of checklists, the reporting score was recorded as mean and standard deviations (SD). As the SPIRIT was published in 2013, we divided the included protocols as publication year before and after 2013. Also, as the number of publications are increased rapidly in recent 3 years, we further set up two groups in the column of “after 2013” to achieve the sample balance. Thus, three subgroups for comparisons are “2005-2012,” “2013-2019,” and “2020-Nov 2022,” All data were collected and recorded in Microsoft Office Excel (Version 2016). Statistics analysis were performed using SPSS software, version 25.0. A flowchart of article selection is shown in Supplementary Figure S1. Briefly, a total of 1191 protocols were finally included. Of these, five articles (0.4%) published in Chinese and 1186 (99.6%) in English. The included 1191 MRCT protocols were published between January 2005 to 11 November 2022. The most common design was a parallel-group assignment (86.2%) of pharmaceutical treatments (35.6%) and active control (62.2%). 58.7% trials adopted blinding and 30.8% used central randomization (including stratified by centers). The three conditions most commonly studied were Neoplasms (9.7%); diseases of the circulatory system (9.6%); and mental, behavioral, or neurodevelopmental disorders (9.3%). The included protocols presented a median of seven participating centers, a median of 242 sample size, and a median of three participating countries for 187 (15.7%) international MRCTs. The included MRCTs were distributed in 88 countries, of which the most common countries are China (17.2%), the United Kingdom (8.9%), and the United States (7.1%). Most trials (>94%) reported whether has funding and trial registration or not, but 95.5% were published in the journals with IF < 6. Of 1191 included trials, 53.7% submitted research ethics approval from one central committee while 26.4% from local committees of each center. 39.0% protocols reported the collection plan of biological specimens. Details are shown in Supplementary Tables S4–S6. For the quality assessment, detailed results are presented in Supplementary Tables 7 and 8. In summary, the mean (SD) reporting score of the SPIRIT was 42.8 (5.9). Specifically, the quality of reporting was excellent (>90%) in 25 items (1, 2a, 2b, 3, 4, 5a, 5b, 6a, 7, 9, 10, 11a, 12, 13, 14, 16a, 16b, 20a, 24, 26a, 26b, 28, 31b, and 32); good (65-90%) in 18 items (5c, 5d, 6b, 8, 11b, 15, 16c, 17a, 18b, 19, 20b, 21a, 22, 23, 25, 27, 29, and 31a); and poor (< 65%) in 8 items (11c, 11d, 17b, 20c, 21b, 30, 31c, and 33). However, the mean reporting score of the multicenter-specific items was 8.3 (2.3), markedly low. No item was “excellent,” and the “good” reporting in eight items (Q1, Q2, Q4, Q5, Q6, Q11, Q12, and Q14). The remaining six items were reported poorly (Q3, Q7, Q8, Q9, Q10, and Q13), of which three items showed extremely low (<30%), covering the rationale for using a multicenter design, methods used to ensure consistent intervention administrated and outcomes measurements across centers, sample size preallocation in each center, and statistical analysis plan for heterogeneity assessment. Protocol publications in later years were associated with increased reporting quality, especially after 2013. The reporting scores increased significantly from 39.5 (7.1) during 2005–2012 to 42.5 (6.1) during 2013–2019, further increased slightly to 42.8 (5.5) in recent three years, out of 51 in total. However, it is noted that the scores for multicenter specifics were still quite low 8.3 (2.3), out of 14 in total, even a mild decrease during 2020-Nov 2022 was identified as 8.2 (2.3). This study identified that the current reporting of MRCT protocol publications is not optimal. Given the increasing feasibility of performing MRCTs, the extraordinary value of their findings to patients and science, and the expense of these trials, improving reporting in the phase of study protocol should be a priority. Developing standard reporting items specifically relevant to multicenter trials as an extension to the general SPIRIT statement might be an expedient, efficient, and effective means to achieving the improvement needed.10 ZXB conceived and designed the study; XZ drafted the manuscript; LZ, ZYL, and FL searched and screened the literatures; FL, ZYL, WFX, NNW, JSD, WTC, DNS, YFM, and CWC extracted, assessed, and checked the data; LZ, NNW, CPC, and XZ performed and checked the statistical analysis; TXW, APL, and CY provided critical comments for this research; XZ and ZXB revised and finalized the manuscript. All authors approved the final version of the manuscript. This work was supported by the Health and Medical Research Fund (No. 18192671), Hong Kong, China; Chinese Medicine Development Fund (No. 20B2/027A), Hong Kong, China; and China Center for Evidence Based Traditional Chinese Medicine, CCEBTM (2020YJSZX-5). The funders had no role in the design of the study, in the collection, analysis, and interpretation of data, nor in the writing of the manuscript. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.