All published articles of this journal are available on ScienceDirect.
Evaluation of Compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Guidelines for Conducting and Reporting Systematic Reviews in Three Major Periodontology Journals
iD
Ali Alqerban1
Abdullah Saad Alqahtani1
Saeed N Asiri1
Abdullah Almutairi2
Authors Info & Affiliations
Abstract
Background
Data from the systematic review, with or without meta-analysis, form the basis of evidence-based medicine. Therefore, these studies should be conducted and reported according to the mandatory Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. This study evaluated compliance with the PRISMA guidelines for conducting and reporting systematic reviews in three major periodontology journals.
Material and Methods
A hand search was conducted in three major periodontal journals to identify Systematic Reviews (SRs) published between January 2018 and July 2022 using the words “Systematic Review” or “meta-analysis” in the title, abstract, or methodology of an article. The PRISMA statement checklist was used to evaluate eligible SRs, covering various sections of the review process. Descriptive statistics, univariate and multivariate analyses, and inter-examiner and intra-examiner reliability assessments were conducted for data analysis.
Results
A total of 87 SRs with meta-analyses were included in the current study. The proportion of published systematic reviews during the investigation period was 5.7% of the total published articles. 16 items were reported adequately in less than 75% of the included papers. Notably, items such as abstracts, data items, sensitivity analysis methods, synthesis results, reporting biases, evidence certainty, registration and protocol, data, code, and other materials availability were reported inadequately in some reviews.
Conclusion
The findings from this study support previous research demonstrating that compliance with the PRISMA guidelines for the conduct and reporting of systematic reviews can vary, potentially attributing to a lack of understanding regarding these guidelines and their clinical significance.
1. BACKGROUND
Systematic Reviews (SRs) and meta-analyses hold a prominent position at the apex of the evidence pyramid for several reasons. These evidence-based reviews, known as SRs, are characterized by their rigorous and compre- hensive approach [1]. SRs prioritize the development of a robust search strategy to minimize bias and ensure transparency. By critically appraising, identifying, and synthesizing all relevant studies on a specific topic, SRs offer a transparent and precise overview of the available evidence [2]. While SRs are widely utilized in the social sciences and clinical research, their application extends to various other disciplines, including education, public policy, environmental sciences, ecology, engineering, advertising, international development, and basic science research [3-5]. By gathering and evaluating experimental evidence, SRs provide a comprehensive understanding of research outcomes, identify gaps in knowledge, improve research methodology, and contribute to a deeper understanding of the specific field [6, 7]. Additionally, SRs often incorporate a meta-analysis component, employing statistical analysis to derive an overall effect size and estimate based on the included studies [8, 9].
By synthesizing existing evidence, SRs help define the questions for which answers are already available, thus recognizing areas where additional research may not be necessary. When conducted properly, SRs and meta-analyses could yield robust outcomes comparable to those obtained from large-scale randomized controlled trials (RCTs) [10-12].
The accurate understanding of the results of SRs relies heavily on the proper and scientific reporting of the research findings. Therefore, it is imperative to approach the reporting process with meticulousness and adherence to established guidelines, such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). PRISMA serves as a valuable framework that provides researchers with a comprehensive checklist for transparently reporting SRs [13, 14]. The PRISMA 2009 statement consisted of 27 checklist items that outlined the essential information to include in a systematic review report. These items covered various aspects, such as the reasons behind conducting the review, the databases utilized to identify relevant studies, the findings of conducted meta-analyses, and the implications derived from the review's results [14]. By following the PRISMA guidelines, researchers could ensure that all crucial elements of their study are addressed to facilitate the readers' precise comprehension of the outcome [15].
SRs with meta-analysis encounter a gap in uniform reporting across different disciplines, including dentistry [16-18]. Researchers examining dental literature have found a consistent lack of adequate reporting in SRs, including in the field of periodontology [16, 19-23]. This deficiency could hinder readers' understanding of study outcomes and diminish the overall impact of these reviews. Therefore, this study evaluated compliance with the PRISMA guidelines for conducting and reporting systematic reviews in three major periodontology journals.
2. MATERIAL AND METHODS
Given that this study solely focused on assessing compliance with the PRISMA guidelines for conducting and reporting systematic reviews in periodontology journals, ethical approval was not required. The nature of the study did not involve any human or non-human subjects, and there was no direct interaction or intervention with individuals or animals.
2.1. Search and Study Selection
A hand search was conducted on three leading periodontal journals with the highest impact factors (2022 Reuters): The Journal of Periodontology (JOP), the Journal of Periodontal Research (JOPR), and the Journal of Clinical Periodontology (JOCP). Inclusion criteria included any article reporting a systematic review, with or without meta-analysis, published within the predefined three periodontal journals between January 2018 and July 2022. Articles were considered eligible for inclusion if the terms “Systematic Review” or “meta-analysis” were explicitly mentioned in the title, abstract, or methodology section. Exclusion criteria were applied to articles that were inaccessible in full-text format, narrative reviews, clinical practice guidelines, and evidence-based commentaries. The titles and abstracts were screened and reviewed independently by two authors (FA and AA) against the inclusion and exclusion criteria. For potentially eligible studies, two authors (FA and AA) independently reviewed full-text articles in duplicate. Disagreements were resolved through discussion and consultation with other co-authors (SA).
2.2. Assessment of Adherence to the PRISMA Statement
Each included systematic review was assessed for the completeness of reporting the items contained in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist 2020 [14]. The raters directly referred to the PRISMA checklist to assess whether the item mentioned in the checklist was reported or not. The PRISMA statement contains a checklist for each section of the systematic review, which includes the title, abstract, introduction, methods, results, discussion, and other information (Tables 1-4). Each PRISMA item is judged with a 'YES' and assigned a '1', a 'No' and assigned a '0', or 'NA' if it did not apply and was not factored into the final score. A percentage score was computed for each included SR based on the sum of the results for the items that applied. Additional information gathered for each study included the journal of publication, number of authors, affiliation of the lead author, and continent of publication.
Two examiners (FA and AA) assessed a random sample of 10% of the articles to assess the inter-examiner reliability of the PRISMA scores. Three months after the initial data collection, the examiner (AA) assessed a second random sample of 10% of the studies to assess intra-examiner reliability.
| Variable | Mean | N | Percentage (%) | SD | 95% CI |
|---|---|---|---|---|---|
| Journal | |||||
| JOP | 71.4 | 5 | 6 | 8.7 | 60.5 to 82.2 |
| JOCP | 69.6 | 57 | 66 | 8.1 | 67.4 to 71.7 |
| JOPR | 70.1 | 25 | 29 | 15.5 | 63.6 to 76.5 |
| Year of publication | |||||
| 2018 | 71.1 | 8 | 9 | 8.0 | 64.4 to 77.8 |
| 2019 | 67.8 | 27 | 31 | 14.7 | 61.9 to 73.5 |
| 2020 | 69.3 | 33 | 38 | 9.4 | 66.0 to 72.7 |
| 2021 | 74.7 | 14 | 16 | 5.2 | 71.7 to 77.7 |
| 2022 | 68.6 | 5 | 6 | 5.4 | 61.8 to 75.3 |
| Authors | |||||
| less than 4 | 69.0 | 8 | 9 | 6.9 | 63.3 to 74.5 |
| 4 to 6 | 69.8 | 58 | 67 | 12.4 | 66.5 to 73.1 |
| More than 6 | 70.2 | 21 | 24 | 5.9 | 67.5 to 72.8 |
| First continent | |||||
| Asia | 73.4 | 11 | 13 | 7.3 | 68.4 to 78.2 |
| Africa | 66.7 | 1 | 1 | - | . |
| North America | 73.8 | 8 | 9 | 5.4 | 69.2 to 78.3 |
| South America | 68.7 | 10 | 11 | 9.0 | 62.3 to 75.2 |
| Australia | 67.9 | 2 | 2 | 11.8 | (-3.8 to 1.73) |
| Europe | 68.9 | 55 | 63 | 12.1 | 65.6 to 72.1 |
| Is the first author is academic? | |||||
| No | 73.8 | 1 | 1 | - | . |
| Yes | 69.8 | 86 | 99 | 10.7 | 67.5 to 72.1 |
| Settings | |||||
| Private | 73.8 | 1 | 1 | - | . |
| University | 70.2 | 77 | 89 | 11.0 | 67.7 to 72.6 |
| Mixed | 66.7 | 9 | 10 | 8.2 | 60.3 to 73.0 |
| Total | 69.8 | 87 | 100 | 10.7 | 67.5 to 72.1 |
| Variables | Unstandardized Coefficients (B) | 95% Confidence Interval | ||
|---|---|---|---|---|
| Continent | Lower | Upper | ||
| Europe | Baseline (reference) | - | ||
| Asia | 0.045 | -0.026 | 0.116 | |
| Africa | -0.022 | -0.239 | 0.195 | |
| North America | 0.049 | -0.032 | 0.131 | |
| South America | -0.001 | -0.075 | 0.073 | |
| Australia | -0.01 | -0.165 | 0.145 | |
| Journals | - | - | - | |
| JOCP | Baseline (reference) | - | ||
| JOP | 0.018 | -0.082 | 0.118 | |
| JOPR | 0.005 | -0.047 | 0.056 | |
| Authors | - | - | - | |
| 4 t 6 authors | Baseline (reference) | - | ||
| Fewer than 4 | -0.008 | -0.089 | 0.073 | |
| More than 6 | 0.004 | -0.051 | 0.058 | |
| Items | All Journals | JOP | JOCP | JRP |
|---|---|---|---|---|
| Identify the report as a systematic review. | 100% | 100% | 100% | 100% |
| Abstract according to Prisma for abstract | 1% | 20.0% | 0.0% | 0.0% |
| Rationale/Context of existing knowledge | 100% | 100% | 100% | 100% |
| Objective/Questions addressed | 100% | 100% | 100% | 100% |
| Eligibility criteria | 99% | 100% | 100% | 96.0% |
| Information sources(databases/registers) | 99% | 100% | 100% | 96.0% |
| Search strategy | 97% | 100% | 96.5% | 96.0% |
| Selection process/screening | 98% | 100% | 98.2% | 96.0% |
| Data collection process | 97% | 100% | 98.2% | 92.0% |
| Define all outcomes | 98% | 100% | 100% | 92.0% |
| Define other variables/missing data | 29% | 20.0% | 29.8% | 28.0% |
| Risk of bias tool | 87% | 100% | 86.0% | 88.0% |
| Effect measures | 82% | 100% | 77.2% | 88.0% |
| Process used to include studies for synthesis | 97% | 100% | 98.2% | 92.0% |
| Methods for preparing data for synthesis | 97% | 100% | 98.2% | 92.0% |
| Presentation of included study’s findings | 98% | 100% | 100% | 92.0% |
| Meta-analysis methods | 77% | 60.0% | 82.5% | 68.0% |
| Methods to assess heterogeneity | 67% | 60.0% | 66.7% | 68.0% |
| Methods of sensitivity analyses | 24% | 20.0% | 24.6% | 24.0% |
| Reporting bias assessment due to missing data | 25% | 60.0% | 14.0% | 44.0% |
| Certainty assessment | 16% | 60.0% | 12.3% | 16.0% |
| Results of the search/number of records | 100% | 100% | 100% | 100% |
| Excluded results/reasons | 91% | 40.0% | 93.0% | 96.0% |
| Study characteristics | 98% | 100% | 100% | 92.0% |
| Risk of bias report | 89% | 100% | 87.7% | 88.0% |
| Results of individual studies | 98% | 100% | 100% | 92.0% |
| Characteristics/risk of bias for each synthesis | 91% | 100% | 91.2% | 88.0% |
| Results of all statistical syntheses | 99% | 100% | 100.0% | 96.0% |
| Results of heterogeneity | 43% | 20.0% | 43.9% | 44.0% |
| Results of sensitivity analyses | 15% | 0.0% | 15.8% | 16.0% |
| Reporting biases in each synthesis due to missing data | 17% | 20.0% | 10.5% | 32.0% |
| Certainty of evidence report for each outcome | 15% | 40.0% | 7.0% | 28.0% |
| General interpretation | 100% | 100% | 100% | 100% |
| Limitation of evidence included in the review | 89% | 80.0% | 87.7% | 92.0% |
| Limitation of the review | 71% | 80.0% | 71.9% | 68.0% |
| Implication of the results | 92% | 80.0% | 89.5% | 100% |
| Registration number/or stated that it was not registered | 61% | 80.0% | 56.1% | 68.0% |
| Access to protocol / or stated it was not prepared | 7% | 0.0% | 1.8% | 20.0% |
| Explain amendments to protocol/registration | 7% | 0.0% | 1.8% | 20.0% |
| Sources of financial or non-financial support | 68% | 80.0% | 80.7% | 36.0% |
| Declare any competing interests | 97% | 80.0% | 100% | 92.0% |
| Which of the data are publicly available and where | 2% | 0.0% | 1.8% | 4.0% |
| Section and Topic | Item # | Checklist Item |
|---|---|---|
| TITLE | ||
| Title | 1 | Identify the report as a systematic review. |
| ABSTRACT | ||
| Abstract | 2 | See the PRISMA 2020 for Abstracts checklist. |
| INTRODUCTION | ||
| Rationale | 3 | Describe the rationale for the review in the context of existing knowledge. |
| Objectives | 4 | Provide an explicit statement of the objective(s) or question(s) the review addresses. |
| METHODS | ||
| Eligibility criteria | 5 | Specify the inclusion and exclusion criteria for the review and how studies were grouped for the syntheses. |
| Information sources | 6 | Specify all databases, registers, websites, organisations, reference lists and other sources searched or consulted to identify studies. Specify the date when each source was last searched or consulted. |
| Search strategy | 7 | Present the full search strategies for all databases, registers and websites, including any filters and limits used. |
| Selection process | 8 | Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process. |
| Data collection process | 9 | Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process. |
| Data items | 10a | List and define all outcomes for which data were sought. Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect. |
| 10b | List and define all other variables for which data were sought (e.g. participant and intervention characteristics, funding sources). Describe any assumptions made about any missing or unclear information. | |
| Study risk of bias assessment | 11 | Specify the methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process. |
| Effect measures | 12 | Specify for each outcome the effect measure(s) (e.g. risk ratio, mean difference) used in the synthesis or presentation of results. |
| Synthesis methods | 13a | Describe the processes used to decide which studies were eligible for each synthesis (e.g. tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)). |
| 13b | Describe any methods required to prepare the data for presentation or synthesis, such as handling of missing summary statistics, or data conversions. | |
| 13c | Describe any methods used to tabulate or visually display results of individual studies and syntheses. |
|
| 13d | Describe any methods used to synthesize results and provide a rationale for the choice(s). If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used. |
|
| 13e | Describe any methods used to explore possible causes of heterogeneity among study results. (e.g. subgroup analysis, meta-regression). |
|
| 13f | Describe any sensitivity analyses conducted to assess the robustness of the synthesized results. | |
| Reporting bias assessment | 14 | Describe any methods used to assess the risk of bias due to missing results in a synthesis. (arising from reporting biases). |
| Certainty assessment | 15 | Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome. |
| RESULTS | ||
| Study selection | 16a | Describe the results of the search and selection process from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram. |
| 16b | Cite studies that might appear to meet the inclusion criteria but which were excluded, and explain why they were excluded. |
|
| Study characteristics | 17 | Cite each included study and present its characteristics. |
| Risk of bias in studies | 18 | Present assessments of risk of bias for each included study. |
| Results of individual studies | 19 | For all outcomes, present for each study: (a) summary of statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g. confidence/credible interval), ideally using structured tables or plots. |
| Results of syntheses | 20a | For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies. |
| 20b | Present results of all statistical syntheses conducted. If meta-analysis was done, present for each the summary estimate and its precision (e.g. confidence/credible interval) and measures of statistical heterogeneity. If comparing groups, describe the direction of the effect. |
|
| 20c | Present results of all investigations of possible causes of heterogeneity among study results. | |
| 20d | Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results. | |
| Reporting biases | 21 | Present assessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed. |
| Certainty of evidence | 22 | Present assessments of certainty (or confidence) in the body of evidence for each outcome assessed. |
| DISCUSSION | ||
| Discussion | 23a | Provide a general interpretation of the results in the context of other evidence. |
| 23b | Discuss any limitations of the evidence included in the review. | |
| Discussion | 23c | Discuss any limitations of the review processes used. |
| 23d | Discuss the implications of the results for practice, policy, and future research. | |
| OTHER INFORMATION | ||
| Registration and protocol | 24a | Provide registration information for the review, including the register name and registration number, or state that the review was not registered. |
| 24b | Indicate where the review protocol can be accessed or state that a protocol was not prepared. | |
| 24c | Describe and explain any amendments to information provided at registration or in the protocol. | |
| Support | 25 | Describe sources of financial or non-financial support for the review and the role of the funders or sponsors in the review. |
| Competing interests | 26 | Declare any competing interests of review authors. |
| Availability of data, code and other materials | 27 | Report which of the following are publicly available and where they can be found: template data collection forms; data extracted from included studies; data used for all analyses; analytic code; any other materials used in the review. |
2.3. Statistical Analysis
The level of adherence to the PRISMA statement was evaluated by calculating the total number of adequately reported items. Descriptive statistics were used to present continuous data, with means and Standard Deviations (SD). Statistical analysis included univariate and multi- variate association analyses to identify characteristics associated with the mean scores of PRISMA adherence. Inter-examiner and intra-examiner reliability were assessed using the Inter-Correlation Coefficient (ICC). All statistical analyses were conducted using IBM Co.'s Statistical Package for the Social Sciences (SPSS) software, version 22 (Armonk, NY, USA).
3. RESULTS
3.1. Study Selection
Initially, 1506 articles were identified from the hand search from three periodontal journals and evaluated by two examiners. The final sample for the current study included a total of 87 Systematic Reviews (SRs), of which 60 were SRs with meta-analysis and 27 were SRs without meta-analysis. During the investigation period, the proportion of published SRs was 5.7% of the published articles. Almost two-thirds of the SRs, a total of 57 (66.5%), were published in the JOCP, 25 (29%) in the JOPR, and only 5 (6%) in the JOP. Most of the published SRs (63%) were performed in Europe and were written by academicians (86 out of 87 SRs). The characteristics of the 87 systematic reviews are shown in Table 1.
3.2. PRISMA Statement Adherence
The mean overall reporting quality score for all SRs was 69.8% (95% CI: 67.5 to 72.1). The five SRs published in the JOP received the highest scores (mean: 71.4, 95%:95% CI: 60.5 to 82.2). The mean score of SRs published in the JOCP was 69.6% (95%: 67.4 to 71.7), while the JOPR scored 70.1% (95%:63.6 to 76.5). The differences between the journals were not significant (p>0.05), as the univariate analysis revealed (Table 2). Another finding in the study is that SRs based in North America achieved the highest reporting score (mean: 73.8), while European-based systematic reviews (55 out of 87 SRs) scored 68.8.
The reporting of PRISMA checklist items varied across the included papers, ranging from 1% to 100%. Out of the 27 items on the PRISMA checklist, 16 items were reported adequately in less than 75% of the included papers. Specifically, items related to abstract reporting according to PRISMA (item 2), data items (item 10b), methods to assess heterogeneity (item 13e), methods of sensitivity analyses (item 13f), reporting bias assessment due to missing data (item 14), methods to assess certainty (item 15), results of bias assessment (item 18), results of heterogeneity assessment (item 20c), results of sensitivity analyses (item 20d), and registration number of the systematic reviews or stating that the systematic review was not registered (item 24a) were among the items that were reported inadequately in a significant proportion of the papers.
The findings of this study reveal that information regarding the registration of systematic reviews to PROSPERO or other registries specific to systematic reviews was available in 39% (34 out of 87) of the included articles. The detailed percentages of checklist item reporting can be found in Table 3. The mean inter-rater and intra-rater reliability levels were high, at 0.83 and 0.89, respectively, as shown by ICC tests.
4. DISCUSSION
A comprehensive body of research has emerged, evaluating the reporting quality of Systematic Reviews (SRs) across diverse medical fields [24-27]. These prior investigations have focused on specific aspects of SR reporting, including the quality of abstracts [27, 28], methodological rigor [25, 29], and the comprehensiveness of reporting in full-text articles [24, 30-32]. However, a gap exists in the literature regarding the reporting quality of SRs, specifically within the field of periodontology. This study aims to address this critical knowledge gap by comprehensively evaluating the reporting quality of SRs published in the field.
Analysis of 87 Systematic Reviews (SRs) published in three prominent periodontology journals (JOCP, JOPR, and JOP) from 2018 to 2022 revealed key insights regarding their prevalence and reporting quality. SRs comprised a relatively small proportion (5.7%) of all published articles, with JOCP contributing the highest number (66%), followed by JOPR (29%) and JOP (6%). Interestingly, 63% of SRs originated from Europe and were authored by academic institutions. While a previous study by Martin et al. 2019 [19] reported 47.2% of their SR sample originating from Europe, the higher proportion in the present study likely reflects the focus on specific journals and a narrower publication timeframe.
Both the current study and the research by Martin et al. 2019 identified shortcomings in adherence to reporting guidelines. However, direct comparisons are limited due to methodological differences. Martin et al. 2019 [19] assessed abstracts using the PRISMA-A 12-item checklist, while this study employed the more comprehensive PRISMA 27-item checklist for full-text evaluation. Regardless of these variations, both studies highlight the need for improved reporting standards in periodontal SRs.
The observed mean overall reporting quality score for all SRs was 69.8%, with the highest score achieved by five SRs published in JOP. Scores for JOPR and JOCP followed, with no significant differences between these journals. Notably, SRs based in North America achieved the highest reporting scores, followed by those from Europe. This range of quality falls short of the expectations for SRs, which occupy the pinnacle of the evidence hierarchy. While the assessed journals boast high-quality factors, they still fell short of implementing all PRISMA checklist guidelines. This finding aligns with Martin et al. (2019), who reported no significant improvement in reporting quality within leading periodontal journals like JOP or JOCP. Pussegoda et al. 2017 [25] and Nawijn et al. 2019 [33] observed a similar trend in their investigation of systematic reviews published in top emergency medicine journals. Their study identified suboptimal reporting practices, particularly in areas related to methodological details and access to the underlying protocols.
The mean compliance score for full-text reports of SRs in this study was higher compared to the reported score in orthodontics (64.1%) [32]. It is noteworthy that the assessed orthodontic SRs were published between 2000 and 2011, whereas this study examined more recently published SRs (2018-2022). Both aforementioned studies identified significant methodological weaknesses during the assessment of the reviewed literature, which could potentially compromise the interpretation and conclusions drawn from the reviews. However, there are promising indications that following the PRISMA guidelines could contribute to better reporting and improved methodo- logical quality in published reviews.
According to the findings of this study, the overall mean reporting quality score for all the SRs included was 69.8%. The highest scores were observed in SRs published in the JOP, followed by the JOPR and the JOCP, with no significant difference between the journals. SRs originating from North America obtained the highest reporting scores, followed by European-based SRs. However, this level of quality is not considered appropriate for the type of evidence that SRs represent, as they occupy the top position in the Evidence pyramid. The journals assessed in this study were of high quality, but they still fell short of fully implementing all the guidelines outlined in the PRISMA checklist. This aligns with the findings of Martin et al., who found that major periodontal journals such as the JOP and the JOCP did not demonstrate any improvement in the comprehensiveness of reporting.
Despite the essential requirement for authors of SRs to adhere to the PRISMA guidelines and accurately report their reviews, it appears that editors and reviewers often do not enforce the proper implementation of the PRISMA checklist and guidelines effectively. The findings of this study indicate that, in accordance with the PRISMA checklist, less than half of the SRs reported one-third or fewer of the checklist items. This lack of compliance with the PRISMA guidelines is also observed in other dental fields when reporting SRs assessing clinical studies, indicating a widespread issue of poor adherence to the PRISMA guidelines [21, 22].
Our analysis identified several key areas where reporting fell short of optimal standards in the included studies. These shortcomings included inadequate reporting in abstracts, data items, methods for sensitivity analyses, synthesis results, reporting biases, evidence certainty, registration, and access to protocols, data, code, and other materials. However, the remaining PRISMA checklist items were generally well-reported, with adherence ranging from 61% to 100%. These findings resonate with previous research [19, 34-36], which has consistently documented the frequent omission of registration information in SRs across diverse medical and dental disciplines.
The findings regarding the frequent absence of registration information in SR abstracts align with observations made by prior researchers across various medical and dental disciplines [19, 37-39]. In order to maintain a rigorous standard for conducted systematic reviews, it is essential to adhere to the mandatory practice of prospectively registering SRs in databases such as PROSPERO. However, this study and previous research [19, 21, 32, 35, 37] revealed that many SRs are accepted and published without proper registration. Even if an SR is not registered, it is crucial to report its status in the published manuscript and whether it has been registered with a registry specific for systematic review [36, 37, 39].
The present study demonstrates a key strength in the assurance of transparent, accurate, and reliable reporting, as a validated checklist was employed following a rigorous calibration process. However, a potential limitation lies in the possibility of selection bias, given the specific inclusion criteria that restricted the sample to systematic reviews published within a defined timeframe (2018-2022) from the three selected journals. Additionally, the relatively small sample size of the study may limit the generalizability of the findings. Future investigations encompassing a wider timespan and including a larger number of systematic reviews published in the field of periodontology could provide a more comprehensive assessment of the reporting quality in this domain. Nonetheless, the findings of the current study remain informative and valuable for editors, researchers, and funding bodies in terms of understanding the quality of reporting in systematic reviews within the field of periodontology.
CONCLUSION
This study provides insights into the compliance of PRISMA guidelines for reporting SRs and Meta-analysis in the field of periodontology. The findings highlight areas for improvement in reporting practices and call for increased adherence to the PRISMA guidelines to enhance the transparency and reliability of SRs in periodontology. Items such as reporting biases have not been adequately reported, which may have adverse effects on the reliability of many systematic reviews and, therefore, should be considered when a systematic review is studied.
AUTHORS’ CONTRIBUTION
FA collected the data, AA analyzed the data and interpreted the results, and AA, AA, and SA wrote the paper. All authors reviewed the results and approved the final version of the manuscript.
LIST OF ABBREVIATIONS
| PRISMA | = Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
| SRs | = Systematic Reviews |
| RCTs | = Randomized controlled trials |
| JOP | = Journal of Periodontology |
| JOPR | = Journal of Periodontal Research |
| JOCP | = Journal of Clinical Periodontology |
ETHICAL STATEMENT
Given that this study solely focused on assessing compliance with the PRISMA guidelines for conducting and reporting systematic reviews in periodontology journals, ethical approval was not required. The nature of the study did not involve any human or non-human subjects, and there was no direct interaction or intervention with individuals or animals.
