All published articles of this journal are available on ScienceDirect.
Adherence to CONSORT for Abstracts in Orthodontic Randomized Controlled Trials Published in Leading Journals (2018-2022)
Abstract
Objective
To investigate the adherence to CONSORT guidelines for reporting abstracts of orthodontic RCTs published in leading journals.
Methods
This retrospective observational study examined abstracts of RCTs published in four orthodontic journals (2018-2022): American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), Journal of Orthodontics (JO), European Journal of Orthodontics (EJO), and Angle Orthodontist Journal (AO). The abstracts were identified using keywords and then assessed for completeness based on the CONSORT for Abstracts checklist. Inter-examiner reliability was assessed to ensure consistency in scoring.
Results
The mean CONSORT score for adherence to reporting guidelines was 65.6%, indicating that abstracts often lacked essential information. There was variation between journals, with the American Journal of Orthodontics and Dentofacial Orthopedics achieving the highest average score. The completeness of reporting varied across different CONSORT items. Essential elements like the study participants, interventions, objectives, outcomes, and conclusions were consistently reported. However, crucial methodological details such as randomization procedures, blinding techniques, recruitment status, adverse events, trial registration, and funding sources were frequently missing from the abstracts.
Conclusion
This study highlights the need for improved reporting quality in abstracts of orthodontic RCTs. Key areas requiring more attention include providing details on trial design, participant characteristics, and funding sources. Standardizing word count limitations across journals could potentially give authors more space to ensure comprehensive reporting within abstracts. Furthermore, by implementing these changes, researchers can ensure that abstracts provide readers with the essential information needed to make informed decisions about orthodontic treatments.
1. INTRODUCTION
The merits of evidence-based dentistry arise from the role of research in making decisions about the efficacy and efficiency of treatment [1]. Randomized clinical trials provide the strongest evidence for evaluating medical interventions, but they must be conducted and published accurately [2]. In light of this, their abstracts should succinctly highlight the primary findings and the methods that were used. The abstract is the first and possibly only component of a randomised clinical trial that is read in order to inform a clinical decision [3, 4]. Moreover, the synthesis of a systematic review requires excluding irrelevant research papers from a research question by utilizing their titles and abstracts. Thus, CONSORT created recommendations to assist authors in producing abstracts that are appropriately reported. Since their development in 1996 and subsequent update in 2010, CONSORT guidelines for reporting abstracts of randomised clinical trials have been endorsed and adopted by many medical journals and editorial organizations [5].
The quality of clinical trial reporting abstracts has been extensively studied in the dental literature [6-11]. It appears that the reporting quality of abstracts of randomised clinical trials has improved modestly but steadily. This might be a result of increased awareness among authors, journal editors, and funding organizations.
Several studies in orthodontics have used CONSORT guidelines to assess the reporting quality of orthodontic clinical trial abstracts [11-16]. The findings of those studies demonstrated that adherence to CONSORT guidelines for reporting abstracts had improved, particularly in reports published in journals that had previously endorsed the guidelines. While advancements in adherence to CONSORT guidelines have been observed, a closer examination revealed inconsistencies in the reporting of specific checklist items within the included studies. These inconsistencies manifested in fundamental elements such as trial registration, funding disclosure, participant numbers, and even the titles of the research projects. As a consequence, the primary objective of this investigation was to comprehensively evaluate the current state of clinical trial abstract reporting within the field of orthodontics.
2. MATERIALS AND METHODS
2.1. Design
The study design was a retrospective observational study.
2.1.1. Sample and Setting
A systematic search strategy was implemented to identify relevant abstracts of randomized controlled trials (RCTs) published within four prominent orthodontic journals during the period of January 2018 to December 2022. The target journals included the American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO), the Journal of Orthodontics (JO), the European Journal of Orthodontics (EJO), and the Angle Orthodontist Journal (AO). Additionally, to mitigate the potential exclusion of pertinent studies, the electronic search was supplemented by a rigorous manual search of the aforementioned journals' archival databases. This comprehensive approach involved screening titles and abstracts for keywords synonymous with RCT methodology, such as “randomized controlled trial,” “assigned,” “prospective,” and “comparative.”. If the report contained one or more of these keywords, the full text was obtained for further examination in order to determine whether the study was a randomised clinical trial. The total number of clinical trials published in the included journal during the study period limited the sample size.
The screening of randomised clinical trials was carried out in duplicate by two authors (SA & FA). Disagreements between the authors were resolved through discussion until mutual agreement was reached. One author (SA) evaluated abstract reporting using the CONSORT for abstract checklist by referring directly to the CONSORT guidelines for abstract reporting and associated explanations. Each abstract item was given a score of “Yes” if it was reported, “No” if it was not reported, or “NA” if it did not apply. An item was marked as NA if the study's design made it impossible to include it, such as blinding patients from the intervention in studies comparing untreated groups to groups receiving active orthodontic treatment [16]. Then, for each trial, a total score was computed and converted to a percentage using the equation: total score= (total number of 'Yes'/ [19-total number of 'NA' items]) /100. For each included study, additional data such as the publishing journal, year of publication, number of authors, continent, and country of the first author, affiliation of the first author, and clinical setting of the trial were collected.
Additionally, to evaluate inter-examiner reliability in applying the CONSORT guidelines, a random selection of 10% (19 papers) of the included papers was independently re-evaluated by a second researcher (FA) using the established criteria for reporting abstracts and their corresponding explanations. This process aimed to assess the level of agreement between the researchers' CONSORT score assignments. Furthermore, to determine intra-examiner reliability, the first author (SA) re-assessed the reporting quality of a separate, randomly chosen subset of 10% of the papers (19 papers) three months after the initial data collection period. This approach allowed for the measurement of consistency in the first author's scoring over time.
2.2. Statistical Analysis
Descriptive statistics were calculated to summarize the data for the entire sample, individual journals, and each publication year. Additionally, the level of adherence to each item on the CONSORT checklist was evaluated and expressed as percentages. Additionally, to investigate potential associations between specific factors and the mean CONSORT score, a linear regression analysis with univariate features was conducted using IBM SPSS Statistics for Macintosh, Version 29.0 (IBM Corp., Armonk, NY). Finally, to assess the reliability of scoring across examiners, both inter-rater and intra-rater reliability were evaluated using intraclass correlation coefficient (ICC) tests.
3. RESULTS
An examination of four orthodontic journals published between 2018 and 2022 identified 192 randomized controlled trials (RCTs) out of 2,114 publications (9%). The majority of these publications were published in the European Journal of Orthodontics (EJO) and the Angle Orthodontist (AO), representing 36% and 38% of the sample, respectively. The American Journal of Orthodontics and Dentofacial Orthopedics (AJODO) and the Journal of Orthodontics (JO) contributed 20% and 6%.
3.1. CONSORT Scores
Mean scores for reporting quality using the CONSORT checklist for abstracts revealed an overall score of 65.6 (95% CI: 63.6-67.5), with AJODO publications achieving the highest score (77.1; 95% CI: 74.2-80.1), followed by EJO (73.1%), AO (52.6%), and JO (63.1%) as shown in Table 1. Univariate analysis showed AJODO to be statistically significantly better than the other journals (Table 2). There was a year-wise variation, with 2018 RCTs achieving the highest score (71.3%; 95% CI: 68.1-74.7) and those published in 2022 scoring the lowest (58.7%; 95% CI: 52.1-65.3). The differences between the years, with 2018 as the reference, were statistically significant. European RCTs displayed the highest mean score (67.8%; 95% CI: 65.2-70.6). Interestingly, Australia, despite contributing the fewest RCTs (6), achieved the highest mean score (81.5%; 95% CI: 66.1-97.1).
| Characteristic |
Number of publication |
Percentage | Mean Score | SD | 95% CI |
|---|---|---|---|---|---|
| Journals | |||||
| AJODO | 39 | 20% | 77.1 | 9.1 | 74.2 to 80.1 |
| JO | 11 | 6% | 63.1 | 9.1 | 57.0 to 69.2 |
| EJO | 69 | 36% | 73.1 | 7.9 | 71.2 to 75.1 |
| AO | 73 | 38% | 52.6 | 9.8 | 50.3 to 54.9 |
| Year | |||||
| 2018 | 53 | 28% | 71.3 | 11.4 | 68.1 to 74.4 |
| 2019 | 39 | 20% | 66.6 | 11.2 | 63.0 to 70.3 |
| 2020 | 39 | 20% | 65.5 | 12.9 | 61.3 to 69.7 |
| 2021 | 43 | 22% | 60.5 | 16.9 | 55.3 to 65.7 |
| 2022 | 18 | 9% | 58.7 | 13.2 | 52.1 to 65.3 |
| Authors | |||||
| Less than 4 | 43 | 22% | 64.7 | 15.7 | 59.9 to 69.5 |
| 4 to 6 | 111 | 58% | 65.6 | 13.6 | 63.1 to 68.2 |
| More than 6 | 38 | 20% | 66.4 | 12.5 | 62.4 to 70.6 |
| Work in Academia | |||||
| No | 24 | 13% | 61.8 | 13.9 | 55.9 to 67.7 |
| Yes | 168 | 88% | 66.1 | 13.8 | 64.1 to 68.2 |
| Setting | |||||
| Private | 5 | 3% | 55.7 | 15.1 | 36.9 to 74.6 |
| University | 147 | 77% | 65.1 | 14.1 | 62.8 to 67.4 |
| Mixed | 40 | 21% | 68.4 | 12.5 | 64.4 to 72.4 |
| Continent | |||||
| Asia | 48 | 25% | 64.3 | 13.6 | 60.2 to 68.2 |
| Africa | 8 | 4% | 56.5 | 6.7 | 50.9 to 62.2 |
| North America | 17 | 9% | 61.9 | 13.4 | 54.9 to 68.8 |
| South America | 22 | 11% | 61 | 15.1 | 54.3 to 67.7 |
| Europe | 91 | 47% | 67.8 | 13.1 | 65.2 to 70.6 |
| Australia | 6 | 3% | 81.5 | 14.7 | 66.1 to 97.1 |
| Overall | 192 | 100% | 65.6 | 13.8 | 63.6 to 67.5 |
| Predictor variables | Univariate analysis | ||
|---|---|---|---|
| Variable | Category or unit | B | 95% CI |
| Journals | AJODO | Baseline (reference) | |
| JO | -14.035 | -20.1 to -7.9* | |
| EJO | -4.043 | -7.6 to -0.47* | |
| AO | -24.513 | -28.0 to -20.9* | |
| Continents | Asia | Baseline (reference) | |
| Africa | -7.6 | -17.7 to 2.4 | |
| North America | 2.3 | -9.7 to 5.1 | |
| South America | -3.2 | -10.1 to 3.5 | |
| Europe | 3.6 | -1.1 to 8.3 | |
| Australia | 17.3 | 5.89 to 28.7* | |
| Year | 2018 | Baseline (reference) | |
| 2019 | -4.6 | -10.2 to 0.89 | |
| 2020 | -5.7 | -11.2 to -0.18* | |
| 2021 | -10.7 | -16.1 to -5.3* | |
| 2022 | 7.35 | -19.6 to -5.3* | |
| Number of authors | 4 to 6 authors | Baseline (reference) | |
| Less than 4 | 0.9 | -5.8 to 4.02 | |
| More than 6 | 0.87 | -4.3 to 5.9 | |
3.2. Completeness of CONSORT Items
Authors, interventions, objectives, outcomes, and conclusions were consistently reported in all RCTs. Title, trial design, participant details, sample size allocation, adherence to analysis protocols, and outcome reporting were present in most RCTs (77% to 96%). However, crucial methodological details like randomization, blinding, recruitment status, adverse events, trial registration, and funding sources were under-reported (7% to 53%). Table 3 shows the report on the CONSORT checklist for all the journals.
| Item | All Journals | AJODO | JO | EJO | AO |
|---|---|---|---|---|---|
| Title/ Identification of the study as randomized | 91% | 100% | 100% | 99% | 78% |
| Authors/ Contact details for the corresponding author (emails or physical address) |
100% | 100% | 100% | 100% | 100% |
| Trial design/ Description of the trial design (e.g. parallel, cluster, non-inferiority) |
77% | 87% | 100% | 91% | 53% |
| Participants/ Eligibility criteria for participants and the settings where the data were collected |
88% | 100% | 100% | 100% | 68% |
| Interventions intended for each group | 100% | 100% | 100% | 100% | 100% |
| Objective/ Specific objective or hypothesis | 100% | 100% | 100% | 100% | 100% |
| Outcome/ Clearly defined primary outcome for this report | 100% | 100% | 100% | 100% | 100% |
| Randomization/ How participants were allocated to interventions | 53% | 82% | 36% | 87% | 7% |
| Clinician blinding | 11% | 15% | 0% | 17% | 4% |
| Patient blinding | 10% | 13% | 9% | 13% | 7% |
| Assessment blinding | 44% | 69% | 27% | 61% | 16% |
| Number of participants randomized to each group | 95% | 100% | 91% | 100% | 88% |
| Recruitment/ Trial status and period or duration | 7% | 5% | 9% | 12% | 3% |
| Number of participants analysed in each group | 92% | 100% | 91% | 100% | 79% |
| Outcome/ For the primary outcome, a result for each group and the estimated effect size and its precision |
96% | 100% | 100% | 100% | 90% |
| Harms Important adverse events or side effects | 22% | 49% | 9% | 29% | 3% |
| Conclusions/ General interpretation of the results | 100% | 100% | 100% | 100% | 100% |
| Trial registration number and name of trial register | 45% | 74% | 27% | 78% | 1% |
| Funding Source | 17% | 74% | 0% | 4% | 0% |
4. DISCUSSION
Given the pivotal role of randomized controlled trial (RCT) abstracts as a gateway to high-level evidence, ensuring their quality is paramount. Comprehensive abstracts empower readers to form a well-informed judgment on the RCT's merit. This saves valuable time for busy practitioners, editors, and researchers when making informed decisions. Additionally, it allows readers without full-text access to glean crucial information for better decision-making regarding the specific RCT.
The findings revealed a suboptimal quality of abstract reporting in orthodontic RCTs, with an average score of 65.6%. This indicates a low adherence to the Consolidated Standards of Reporting Trials (CONSORT) guidelines, even lower than the 69.1% reported for RCTs published between 2012-2017 [12]. However, it shows some improvement over the 60.2% adherence observed for RCTs published between 2006-2011 [12]. The findings of this study align with previous research across various dental specialties and other fields, collectively highlighting significant deficiencies in adherence to CONSORT guidelines for reporting randomized controlled trials (RCTs). Studies in public health dentistry, periodontology, and endodontics consistently report incomplete reporting of critical methodological details such as randomization, blinding, and funding sources [10, 11, 17-19]. Similarly, recent research in pediatric neuro-oncology RCTs reported an adherence rate of 67.4%, while a study on posterior restoration RCTs found a comparable figure of 59% [20, 21]. These findings, along with others across diverse disciplines, indicate a widespread issue of poor adherence to reporting guidelines. Despite some variations and occasional improvements, the overall consensus emphasizes the need for stricter adherence to CONSORT guidelines and standardized reporting practices to enhance the transparency and reliability of RCT abstracts [22-27].
Compared to the 2012-2017 sample, improvements were observed in reporting specific items, including titles, randomization methods, number of participants analyzed in each group, harms, and trial registration. However, areas such as trial design, participant eligibility criteria and settings, and source of funding were less frequently reported, according to this study [12].
Several factors could contribute to this underreporting. One possibility is a lack of strict enforcement of CONSORT-A guidelines by editors and reviewers, despite the fact that all included studies were published in journals endorsing CONSORT for several years [16]. Additionally, some CONSORT-A checklist items may not be applicable to orthodontic trials, such as blinding participants or operators from the intervention. However, explicit reporting of this inapplicability is crucial. Similarly, attempts to blind outcome assessors should be reported even if blinding participants/operators is not feasible. Furthermore, journal word count limitations might hinder authors from comprehensively reporting their RCT abstracts. Fleming et al. suggest that complete reporting guidelines are possible within a 250-word limit [14]. The current author instructions for the included journals require abstracts to be reported in fewer than 250 words (AJODO and AO), 300 words (JO), and 400 words (EJO).
This trend of underreporting critical elements in RCT abstracts, as outlined by the CONSORT statement, is concerning. It restricts access to vital information regarding the applicability and validity of the research for consumers of medical research. Moreover, as shown by other studies [28, 29], poor abstract reporting likely reflects on the accuracy and completeness of the accompanying full text. This underscores the importance of not relying solely on abstracts to draw definitive conclusions.
A potential limitation of this study is the inherent subjectivity involved in scoring reported items. However, the inter and intra-reliability tests conducted helped mitigate this limitation. Additionally, while only RCTs published in four orthodontic journals were included, these journals are highly regarded in the field and known for their rigorous editorial processes. Consequently, it is likely that RCTs published in these journals would exhibit better methodological and reporting standards compared to those published elsewhere.
Future research could explore interventions to improve adherence to CONSORT guidelines among authors, editors, and reviewers. Educational initiatives and workshops could raise awareness of the importance of comprehensive abstract reporting. Additionally, standardizing word count limitations across journals could provide authors with more space for complete reporting. In addition, by addressing these concerns, we can ensure that RCT abstracts in orthodontics provide readers with the necessary information to make informed decisions.
CONCLUSION
This study found that abstracts of RCTs published in four leading journals between 2018 and 2022 fell short of the standards suggested by the CONSORT statement. There is a need to improve reporting of items such as trial design, participant eligibility criteria and settings, and source of funding.
AUTHORS' CONTRIBUTIONS
FA contributed to conceptualization, study design, and analysis. SA contributed to data collection. Both authors contributed to drafting the manuscript.
LIST OF ABBREVIATIONS
| CONSORT | = Consolidated Standards of Reporting Trials |
| RCTs | = Randomized clinical trials |
| SD | = Standard deviation |
| CI | = Confidence interval |
