Retention Loss of Resin Based Fissure Sealants - a Valid Predictor for Clinical Outcome?

Literature Search

The references and additional online content of three systematic reviews [20, 26, 27] on the topic of fissure sealants were checked for relevant trials. PubMed/Medline, Directory of Open Access Journals (DOAJ); IndMed and Scielo were searched by both authors (SM and VY), independently, using the search term: “fiss* AND seal*” in order to identify additional studies. The search period was limited to the publication period for the electronic database search from 30.09.2011 to 21.10.2012.

Titles and abstracts of the resulting citations were scanned for possible inclusion in line with the following inclusion criteria:

1. Clinical study reporting on the retention and caries occurrence of resin-based fissure sealants in permanent molar teeth (no distinction was made between different types of resin-based sealants);
2. Minimum 24-month follow-up period;

Articles whose titles and abstracts were in alignment with the inclusion criteria were retrieved in full copy and were reviewed by both authors of this article. Articles were further excluded if: premolars were included in the study and their data analysed together with that of molar teeth; no caries was assessed; no computable data was reported; the number of teeth evaluated for retention and caries was not the same; the number of previously sealed teeth that developed carious lesions/cavities was reported together with the number of teeth filled and extracted. Articles that could not be traced in full copy were also excluded. Disagreements were resolved through discussion and consensus.

Data Extraction and Statistical Analysis

Both authors extracted data from the accepted articles independently without being blinded to authors, institutions, journal names and trial results. The extracted data included: the number of evaluated sealed teeth (N); the number of teeth without completely retained fissure sealants (n_R-) - loss of complete material retention; the number of sealed teeth with carious lesion/cavities (n_C-) at the end of each follow-up period. The choice of using loss of ‘complete material retention’ was based on the consideration that “intact sealant”; i.e. complete sealant without material loss, has been adopted as the leading fissure sealant criterion today [20]. From the extracted n_R- and n_C- –values the number of teeth with complete sealant retention (n_R+) and absence of carious lesions/cavities (n_C+) were calculated (n – N). Based on the n_R+; n_R-; n_C- and n_C+ study data, the following predictive outcomes were established:

1. True positive outcome (TP): The number of n_R- - teeth where carious lesion/cavities (n_C-) have developed;
2. False positive outcome (FP): The number of n_R- - teeth where no carious lesion/cavities (n_C+) have developed;
3. False negative outcome (FN): The number of n_R+ - teeth where carious lesion/cavities (n_C-) have developed;
4. True negative outcome (TN): The number of n_R+ - teeth where no carious lesion/cavities (n_C+) have developed.

The predictive outcomes based on the retention rates extracted from fissure sealants studies were considered as the test-data. In order to establish whether retention loss of resin sealants can predict caries manifestation more accurately than any random guesses by play of chance, random values were generated as control-data by using an online random number generator [28]. One random number between 0 and (N) was generated for each study dataset and these were used instead of the n_R- values of the test data. Based on these random numbers (n_rand), predictive outcome values were calculated and formed the control data.

From the predictive test- and control outcomes, Diagnostic Odds Ratios (DOR) were computed per study dataset. The DOR combines sensitivity and specificity characteristics into one single predictive summary measure and is defined as: DOR = (TP x TN) / (FP x FN) [29]. The DOR may range from zero to infinity. DOR values of less than 1.00 indicate that the prediction may be useful, by inverting its outcome. A DOR value of or close to 1.00 provides no predictive evidence and corresponds to the rising diagonal in Summary Receiver Operating Characteristic (SROC) graphs. The higher the DOR value (>1.00) the better the predictive accuracy [30].

In addition to the computation of DOR values, SROC curves with a 95% Confidence interval (CI) were plotted for the test- and control data. SROC curves in relation to the rising diagonal of the SROC graph (DOR = 1.00) give a graphical representation of the accuracy of predictions according to the established true positive (sensitivity, shown on the y- axis) and false positive rates (1-specificity, shown on the x- axis).

Statistical software Meta-DiSc Version 1.4 [31] was used for computing and plotting all DOR values and SROC curves. Zero cells were converted into 0.5 values prior to computation. The median values with a 25-75% percentile range of the DOR values were computed for the test- and control data and the results plotted in a Box-Whisker graph. Meta-analytic pooling of the individual DOR values was only considered if a low in-between-study-heterogeneity was detected (I² < 30%) [32]. Because the samples of DOR values from both test and control data shared the same caries measurements, they were considered as dependent data samples [33]. Therefore, median differences between both samples were tested for statistical significance, using the Wilcoxon-Matched-Pair test and Sign test (Biostat 2009 statistical software). Alpha level for statistical significance was set at 5%. Rejection of the null-hypothesis was conditional on the basis of the median DOR of the test data being statistically significantly larger than that of the control data (p < 0.05).

Sensitivity Analysis

In order to test whether multiple measurements of the same materials at different observation periods may have had any influence on the results, sensitivity analysis was conducted by retaining only datasets of the longest observation period per sealant material. The following 16 datasets were subsequently removed: DS 10, 12, 27, 28, 31, 32, 64, 66, 80, 104, 118, 120, 122, 162-164 and the remaining data samples were re-analysed.

Literature Search and Data Selection

The literature search generated 95 citations, of which 90 were extracted from the additional online content and reference lists of three systematic reviews [20, 26, 27] and 5 citations were identified through an electronic database search. Details of study design and length of observation period per study are shown in APPENDIX FILE 1. From the 95 studies, 185 datasets (DS) with relevance to sealant retention and caries development were extracted. From the 185 datasets, 95 (from 57 studies) were excluded for the following reasons: exclusive investigation of molar teeth not made explicit (10 DS); caries assessment missing (28 DS); full article could not be traced (21 DS); no computable data reported (3 DS); not the same number of teeth evaluated for retention and caries (9 DS); number of previously sealed teeth that developed carious lesions/cavities reported together with number of teeth filled and extracted (24 DS).

Extracted Data and Statistical Analysis

From the provisionally included 185 datasets, a total of 90 datasets were accepted for data analysis (APPENDIX FILE 2). For the purpose of this study no distinction was made between tooth- or side-specific data. High statistical in-between-study heterogeneity was established (I² = 95.4% and 96.4% for test- and control data, respectively). Thus, no meta-analytic pooling of DOR values was considered.

The computed DOR values per dataset for both data groups are shown in APPENDIX FILE 3. Summary Receiver Operating Characteristic (SROC) curves for the test and control data are shown in Figs. (1 and 2). The median DOR values were 1.21 (0.20 – 10.71) and 0.28 (0.07 – 13.10) for the test and control data, respectively (Fig. 3). The results of the Wilcoxon test (z = 0.56; p = 0.58) and Sign test (z = 1.38; p = 0.17), which evaluated the differences between medians for DOR values, were statistically non-significant. The null-hypothesis that the rate of losing complete retention of resin- based fissure sealants, after an observation period of at least 24-months, does not predict caries manifestation on pits and fissures of permanent molar teeth significantly more accurately than any random values could therefore not be rejected.

Sensitivity Analysis

Sensitivity analysis results showed median DOR values of 0.98 (0.20 – 6.35) and 0.20 (0.06 – 13.10) for the test- and control data, respectively. The results of the Wilcoxon test (z = 0.47; p = 0.64) and Sign test (z = 1.30; p = 0.20) were statistically non-significant. Multiple measurements of the same materials at different observation period points appear to have had no decisive influence on the results and the conclusions of this study.

Limitations of Study Method

The aim of this study was to test the null-hypothesis that the rate of losing complete retention of resin based fissure sealants, after an observation period of at least 24-months, does not predict caries manifestation on pits and fissures of permanent molar teeth significantly more accurately than any random values.

In order to meet this aim, the chosen study methodology focused on the data extracted from resin based fissure sealant trials, without taking aspects of internal trial validity under consideration. Such aspects, particularly related to attrition- and detection bias, may have affected the trial results and thus in turn, the data on which this study is based. Neither, however, was internal trial validity with specific focus on systematic errors/biases investigated during the systematic review by Kühnisch et al., 2012 [20], whose references (provided in its additional online content) constituted the main data source of this investigation (APPENDIX FILE 1). Consequently, this study is in line with the same assumptions concerning internal validity of the same included fissure sealant trials as those reviewed by Kühnisch et al., 2012 [20].

In this study, more than half of the available datasets were excluded (APPENDIX FILE 1). However, the exclusion of premolar teeth is in line with investigations into sealant effectiveness reported by other authors [20] and is based on the observation that caries develops less on premolar teeth than on molars [15]. The exclusion of 24 datasets on the basis of their inclusion of the filled and extracted DMFT component appears justified, as the true reasons for the filling and extraction of formerly sealed teeth are unknown. Other reasons for data exclusion: lack of computable data reported, different numbers of teeth assessed for retention and caries and lack of caries assessment are clear indicators for non-relevance with regard to the study aim. The lack of 21 datasets from trials that remained untraceable in full copy through the available library- and Internet sources or directly from the authors remains a concern. However, these would have comprised just 19% of the total available data and whether some of the missing data would not have been excluded for other reasons too, if the trial reports could have had been traced, remains doubtful. Under such assumption, the lack of the missing data may not have significantly impacted the conclusion of this study.

Study Results

Treatment effects on surrogate endpoints should reliably predict the effect of such treatment on the true clinical endpoint [21-25]. Hence, if the placement of resin-based sealants on pits and fissures of permanent molar teeth result in the retention/loss of the material after >24 months, this should reliably predict the absence/presence of caries manifestation beyond the play of chance. Within the context of this study, the number of teeth with/without completely retained sealants (n_R+/n_R-) should be able to predict reasonably the number of teeth that do not/do develop carious decay in pits and fissures (n_C+/n_C-). Such results form the true negative/positive (TN/TP) predictions. In comparison, the number of false negative/positive (FN/FP) results should not exceed between 2.5 – 10% of all predictions [21].

In this study, the test data for sealant retention (n_R-) was obtained from clinical fissure sealant trials and compared to random values (n_rand) in relation to the true clinical endpoint (= caries manifestation / n_C-) and computed as median Diagnostic Odds ratios (DOR). It was expected, on the basis of current consensus concerning fissure retention [15-20], that the predictive DOR values from the test data would be significantly higher (more accurate) than those from the control. However, no statistically significant differences between the TP/TN and FP/FN dependent DOR values were found (Wilcoxon test: z = 0.56; p = 0.58; Sign test: z = 1.38; p = 0.17). The null-hypothesis (H₀) could therefore not be rejected, thus no sufficient evidence in support of the alternate hypothesis (H₁) was found. These results suggest that predictions based on the relationship of the retention rate of resin-based fissure sealants and caries manifestation are no more accurate than random guesses. Furthermore, the SROC curve (Fig. 2), based on such random values (median DOR 0.28), appears to indicate a higher (albeit inverted and still purely random) predictive accuracy than the sealant retention rate (DOR = 1.21; Fig. 1).

The lack of any significant difference in predictive accuracy between the retention rate and random values may not be surprising. Many factors other than the mere loss of complete sealant retention have a potential effect on caries manifestation in pits and fissures of permanent molar teeth. These factors may include: the position of the molar tooth in the mouth, the period between retention loss and follow-up, access to fluoride, oral hygiene and dietary habits, enamel structure, saliva factors or, possibly, factors that remain unknown. Manifestation of caries may also be related to sealed pits and fissures, if the partially lost sealants were misclassified as ‘completely retained’ or if pits and fissures still contain micro-remnants of sealant materials that provide caries protection but were classified as ‘loss of complete retention’.

Against this background, it may appear obvious that the retention rate alone is unable to account for the influence of other known and unknown factors that may influence caries manifestation. Nonetheless, the retention rate of fissure sealants has been accepted as valid surrogate for caries prevention [21-25]. Valid surrogate endpoints are defined as measurements or physical signs for use as substitutes for true clinical endpoints [34], while true clinical endpoints are defined as clinical, patient-relevant events of which the patient is aware, which the patient wants to avoid and which affect her/his quality of life [21]. Caries manifestation in pits and fissures constitutes a true clinical endpoint that was utilized during first generation of fissure sealant trials [9-11]. In light of the inability of the retention rate to be an accurate predictor of caries in pits and fissures, further (randomised controlled) sealant trials should have retained this clinical endpoint as the outcomes measure while using resin-based sealant materials and sealant techniques, established during the previous trials, as gold standards against which new materials/techniques were to be compared.

The use of invalid surrogates that cannot sufficiently predict their clinical endpoints carries the danger of unexpected adverse effects in clinical trials, therapeutic uncertainties due to ambiguous evidence, as well as the risk of rejection of potentially useful therapies because they do not show benefits in line with the surrogate endpoints [24]. Of these consequences, the latter is especially apparent with regard to glass-ionomer cement (GIC)-based fissure sealants [20], despite the fact that systematic reviews have found no difference between the caries-preventive effect of GIC and that of resin-based sealants [26, 27]. For these reasons, it has been repeatedly argued that: (i) biological outcomes should take precedence over mechanical ones; (ii) because sealants are placed in pits and fissures in order to prevent the onset of caries manifestation or to arrest it, the true outcome of such intervention should therefore be expressed in terms of how well such intervention has managed to achieve this objective [35, 36].