Retention Loss of Resin Based Fissure Sealants - a Valid Predictor for Clinical Outcome?

Steffen Mickenautsch*, Veerasamy Yengopal
SYSTEM Initiative/Department of Community Dentistry, Faculty of Health Sciences, University of the Witwatersrand - 7 York Rd., Parktown/Johannesburg 2193, South Africa

Article Metrics

CrossRef Citations:
Total Statistics:

Full-Text HTML Views: 1911
Abstract HTML Views: 1296
PDF Downloads: 344
Total Views/Downloads: 3551
Unique Statistics:

Full-Text HTML Views: 935
Abstract HTML Views: 787
PDF Downloads: 212
Total Views/Downloads: 1934

Creative Commons License
© Mickenautsch and Yengopal; Licensee Bentham Open.

open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

* Address correspondence to this author at the SYSTEM Initiative/Department of Community Dentistry, Faculty of Health Sciences, University of the Witwatersrand, 7 York Rd., Parktown/Johannesburg, 2193, South Africa; Tel: +27 11 717 2594; Fax: +27 11 717 2625; E-mail:



The null-hypothesis that retention loss of resin fissure sealants predicts caries manifestation no more accurately than random values was tested.


Systematic reviews were checked and electronic databases searched for clinical trials. Trials reporting on the retention of resin sealants and caries occurrence in permanent molar teeth, with minimum 24-month follow-up period, were included. Extracted data: number of sealed teeth, number of teeth without completely retained sealants, number of sealed teeth with caries. The number of teeth with complete sealant retention and absence of carious lesions/cavities was calculated; the predictive outcomes: true/false positive; false/true negative were established. Random values were generated as control-data. Diagnostic Odds ratios (DOR) were computed and tested for statistical difference. Summary Receiver Operating Characteristic curves were plotted.


95 trials were found. Median DOR values were 1.21 and 0.28 for test- and control data, respectively. Wilcoxon test (z = 0.56; p = 0.58) and Sign test (z = 1.38; p = 0.17) results were statistically non-significant. The null-hypothesis was not rejected.


Predictions based on the retention loss of resin sealants, regarding caries manifestation, was no more accurate than random guesses. Sealant retention loss appears not to be a valid predictor for clinical outcome.

Keywords: : Diagnostic Odds ratio, fissure sealant, resin, retention, surrogate endpoint, tooth caries.


Most carious lesions and cavities develop in pits and fissures of occlusal surfaces in primary and permanent posterior teeth [1]. The simultaneous interaction of a susceptible host, cariogenic microflora and suitable substrate (i.e. fermentable carbohydrates from food in the oral environment) has been established as the etiological factor [2]. Exclusion of the cariogenic microflora from substrate may prevent carious lesion and cavity development and is achieved by placing a physical barrier in the form of a seal on pits and fissures [3].

Pit and fissure sealants have evolved, from phosphate cements [4], silver nitrate [5] and black copper cement [6], to the current resin- based materials [7]. Buonocore (1955) reported the first clinical resin-based fissure sealant experiences [8]. Subsequent clinical research compared the development of carious lesions/cavities in sealed teeth with that in non-sealed teeth as controls, mainly in split mouth trials, and established retention longevity of the sealant material functions as a beneficial factor in the prevention of caries [9-11]. A subsequent systematic review by the Cochrane collaboration, confirmed these findings [12].

On the basis of the positive results, the further use of a control group of non-sealed teeth in clinical studies became ethically unacceptable [13]. Instead, subsequent studies compared newer with older sealant materials and/or sealant techniques as their controls [14]. The outcome measure of fissure sealant studies also changed, from the rate of caries manifestation to the retention rate of fissure sealants. Based on the observation of ‘retention’ as a beneficial factor in the prevention of caries, the rate of sealant retention was forthwith assumed to be a valid surrogate endpoint for fissure sealant effectiveness. Rock and Anderson (1982) highlighted it as obvious that (resin-based) fissure sealants are effective only because of their presence on the teeth. Regression results of retention rate with caries manifestation supported this logical conjecture [15]. Weintraub and Bart (1987) agreed that the choice of monitoring the retention rate in fissure sealant studies as the expression of effectiveness was based on the fact that sealant effectiveness was a direct function of its retention [16]. Liebenberg (1994) found it axiomatic that if any prophylactic benefit is to be accrued, retention of the sealant must be completely intact [17]. Locker et al. (2003) and Muller-Bolla et al. (2006) stated that since caries does not develop as long as the sealant remains adherent to the tooth, the length of the time such sealant is retained is justified for use as a surrogate measure of its effectiveness in preventing tooth decay [13, 14]. Beauchamp et al. (2008) declared, on behalf of the American Dental Association Council on Scientific Affairs, that since the effectiveness of sealants depends on clinical retention, the retention rate is acceptable as proxy for caries prevention [18]. Sasa and Donly (2010) also accepted that sealant retention equates with caries prevention [19] and Kühnisch et al. (2012) declared that because optimum protection is only guaranteed if the sealant completely covers all pits and fissures, ‘intact sealant’ (as opposed to lost or partially retained sealant) is the leading fissure sealant criterion today [20].

For the purpose of this study and in accordance with the general consensus [15-20], ‘complete retention’ was assumed to be a valid surrogate endpoint for caries prevention. On the basis of the further consensus that sufficiently correct prediction of a true clinical endpoint by its surrogate endpoint is an essential requirement for surrogate validity [21 - 25], it was also assumed that ‘complete retention’, as the valid surrogate endpoint, would successfully predict caries manifestation. In this context, the aim of this study was to test the null-hypothesis (H0) that the rate of losing complete retention of resin- based fissure sealants, after an observation period of at least 24-months, does not predicts caries manifestation on pits and fissures of permanent molar teeth significantly more accurately than any random values (with the alternate hypothesis, H1, that complete retention predicts caries manifestation more accurately).


Literature Search

The references and additional online content of three systematic reviews [20, 26, 27] on the topic of fissure sealants were checked for relevant trials. PubMed/Medline, Directory of Open Access Journals (DOAJ); IndMed and Scielo were searched by both authors (SM and VY), independently, using the search term: “fiss* AND seal*” in order to identify additional studies. The search period was limited to the publication period for the electronic database search from 30.09.2011 to 21.10.2012.

Titles and abstracts of the resulting citations were scanned for possible inclusion in line with the following inclusion criteria:

  1. Clinical study reporting on the retention and caries occurrence of resin-based fissure sealants in permanent molar teeth (no distinction was made between different types of resin-based sealants);
  2. Minimum 24-month follow-up period;

Articles whose titles and abstracts were in alignment with the inclusion criteria were retrieved in full copy and were reviewed by both authors of this article. Articles were further excluded if: premolars were included in the study and their data analysed together with that of molar teeth; no caries was assessed; no computable data was reported; the number of teeth evaluated for retention and caries was not the same; the number of previously sealed teeth that developed carious lesions/cavities was reported together with the number of teeth filled and extracted. Articles that could not be traced in full copy were also excluded. Disagreements were resolved through discussion and consensus.

Data Extraction and Statistical Analysis

Both authors extracted data from the accepted articles independently without being blinded to authors, institutions, journal names and trial results. The extracted data included: the number of evaluated sealed teeth (N); the number of teeth without completely retained fissure sealants (nR-) - loss of complete material retention; the number of sealed teeth with carious lesion/cavities (nC-) at the end of each follow-up period. The choice of using loss of ‘complete material retention’ was based on the consideration that “intact sealant”; i.e. complete sealant without material loss, has been adopted as the leading fissure sealant criterion today [20]. From the extracted nR- and nC- –values the number of teeth with complete sealant retention (nR+) and absence of carious lesions/cavities (nC+) were calculated (n – N). Based on the nR+; nR-; nC- and nC+ study data, the following predictive outcomes were established:

  1. True positive outcome (TP): The number of nR- - teeth where carious lesion/cavities (nC-) have developed;
  2. False positive outcome (FP): The number of nR- - teeth where no carious lesion/cavities (nC+) have developed;
  3. False negative outcome (FN): The number of nR+ - teeth where carious lesion/cavities (nC-) have developed;
  4. True negative outcome (TN): The number of nR+ - teeth where no carious lesion/cavities (nC+) have developed.

The predictive outcomes based on the retention rates extracted from fissure sealants studies were considered as the test-data. In order to establish whether retention loss of resin sealants can predict caries manifestation more accurately than any random guesses by play of chance, random values were generated as control-data by using an online random number generator [28]. One random number between 0 and (N) was generated for each study dataset and these were used instead of the nR- values of the test data. Based on these random numbers (nrand), predictive outcome values were calculated and formed the control data.

From the predictive test- and control outcomes, Diagnostic Odds Ratios (DOR) were computed per study dataset. The DOR combines sensitivity and specificity characteristics into one single predictive summary measure and is defined as: DOR = (TP x TN) / (FP x FN) [29]. The DOR may range from zero to infinity. DOR values of less than 1.00 indicate that the prediction may be useful, by inverting its outcome. A DOR value of or close to 1.00 provides no predictive evidence and corresponds to the rising diagonal in Summary Receiver Operating Characteristic (SROC) graphs. The higher the DOR value (>1.00) the better the predictive accuracy [30].

In addition to the computation of DOR values, SROC curves with a 95% Confidence interval (CI) were plotted for the test- and control data. SROC curves in relation to the rising diagonal of the SROC graph (DOR = 1.00) give a graphical representation of the accuracy of predictions according to the established true positive (sensitivity, shown on the y- axis) and false positive rates (1-specificity, shown on the x- axis).

Statistical software Meta-DiSc Version 1.4 [31] was used for computing and plotting all DOR values and SROC curves. Zero cells were converted into 0.5 values prior to computation. The median values with a 25-75% percentile range of the DOR values were computed for the test- and control data and the results plotted in a Box-Whisker graph. Meta-analytic pooling of the individual DOR values was only considered if a low in-between-study-heterogeneity was detected (I2 < 30%) [32]. Because the samples of DOR values from both test and control data shared the same caries measurements, they were considered as dependent data samples [33]. Therefore, median differences between both samples were tested for statistical significance, using the Wilcoxon-Matched-Pair test and Sign test (Biostat 2009 statistical software). Alpha level for statistical significance was set at 5%. Rejection of the null-hypothesis was conditional on the basis of the median DOR of the test data being statistically significantly larger than that of the control data (p < 0.05).

Sensitivity Analysis

In order to test whether multiple measurements of the same materials at different observation periods may have had any influence on the results, sensitivity analysis was conducted by retaining only datasets of the longest observation period per sealant material. The following 16 datasets were subsequently removed: DS 10, 12, 27, 28, 31, 32, 64, 66, 80, 104, 118, 120, 122, 162-164 and the remaining data samples were re-analysed.


Literature Search and Data Selection

The literature search generated 95 citations, of which 90 were extracted from the additional online content and reference lists of three systematic reviews [20, 26, 27] and 5 citations were identified through an electronic database search. Details of study design and length of observation period per study are shown in APPENDIX FILE 1. From the 95 studies, 185 datasets (DS) with relevance to sealant retention and caries development were extracted. From the 185 datasets, 95 (from 57 studies) were excluded for the following reasons: exclusive investigation of molar teeth not made explicit (10 DS); caries assessment missing (28 DS); full article could not be traced (21 DS); no computable data reported (3 DS); not the same number of teeth evaluated for retention and caries (9 DS); number of previously sealed teeth that developed carious lesions/cavities reported together with number of teeth filled and extracted (24 DS).

Extracted Data and Statistical Analysis

From the provisionally included 185 datasets, a total of 90 datasets were accepted for data analysis (APPENDIX FILE 2). For the purpose of this study no distinction was made between tooth- or side-specific data. High statistical in-between-study heterogeneity was established (I2 = 95.4% and 96.4% for test- and control data, respectively). Thus, no meta-analytic pooling of DOR values was considered.

The computed DOR values per dataset for both data groups are shown in APPENDIX FILE 3. Summary Receiver Operating Characteristic (SROC) curves for the test and control data are shown in Figs. (1 and 2). The median DOR values were 1.21 (0.20 – 10.71) and 0.28 (0.07 – 13.10) for the test and control data, respectively (Fig. 3). The results of the Wilcoxon test (z = 0.56; p = 0.58) and Sign test (z = 1.38; p = 0.17), which evaluated the differences between medians for DOR values, were statistically non-significant. The null-hypothesis that the rate of losing complete retention of resin- based fissure sealants, after an observation period of at least 24-months, does not predict caries manifestation on pits and fissures of permanent molar teeth significantly more accurately than any random values could therefore not be rejected.

Fig. (1).

SROC curve for test data: retention rate.

Fig. (2).

SROC curve for control data: random values.

Fig. (3).

Median diagnostic Odds ratios (DOR).

Sensitivity Analysis

Sensitivity analysis results showed median DOR values of 0.98 (0.20 – 6.35) and 0.20 (0.06 – 13.10) for the test- and control data, respectively. The results of the Wilcoxon test (z = 0.47; p = 0.64) and Sign test (z = 1.30; p = 0.20) were statistically non-significant. Multiple measurements of the same materials at different observation period points appear to have had no decisive influence on the results and the conclusions of this study.


Limitations of Study Method

The aim of this study was to test the null-hypothesis that the rate of losing complete retention of resin based fissure sealants, after an observation period of at least 24-months, does not predict caries manifestation on pits and fissures of permanent molar teeth significantly more accurately than any random values.

In order to meet this aim, the chosen study methodology focused on the data extracted from resin based fissure sealant trials, without taking aspects of internal trial validity under consideration. Such aspects, particularly related to attrition- and detection bias, may have affected the trial results and thus in turn, the data on which this study is based. Neither, however, was internal trial validity with specific focus on systematic errors/biases investigated during the systematic review by Kühnisch et al., 2012 [20], whose references (provided in its additional online content) constituted the main data source of this investigation (APPENDIX FILE 1). Consequently, this study is in line with the same assumptions concerning internal validity of the same included fissure sealant trials as those reviewed by Kühnisch et al., 2012 [20].

In this study, more than half of the available datasets were excluded (APPENDIX FILE 1). However, the exclusion of premolar teeth is in line with investigations into sealant effectiveness reported by other authors [20] and is based on the observation that caries develops less on premolar teeth than on molars [15]. The exclusion of 24 datasets on the basis of their inclusion of the filled and extracted DMFT component appears justified, as the true reasons for the filling and extraction of formerly sealed teeth are unknown. Other reasons for data exclusion: lack of computable data reported, different numbers of teeth assessed for retention and caries and lack of caries assessment are clear indicators for non-relevance with regard to the study aim. The lack of 21 datasets from trials that remained untraceable in full copy through the available library- and Internet sources or directly from the authors remains a concern. However, these would have comprised just 19% of the total available data and whether some of the missing data would not have been excluded for other reasons too, if the trial reports could have had been traced, remains doubtful. Under such assumption, the lack of the missing data may not have significantly impacted the conclusion of this study.

Study Results

Treatment effects on surrogate endpoints should reliably predict the effect of such treatment on the true clinical endpoint [21-25]. Hence, if the placement of resin-based sealants on pits and fissures of permanent molar teeth result in the retention/loss of the material after >24 months, this should reliably predict the absence/presence of caries manifestation beyond the play of chance. Within the context of this study, the number of teeth with/without completely retained sealants (nR+/nR-) should be able to predict reasonably the number of teeth that do not/do develop carious decay in pits and fissures (nC+/nC-). Such results form the true negative/positive (TN/TP) predictions. In comparison, the number of false negative/positive (FN/FP) results should not exceed between 2.5 – 10% of all predictions [21].

In this study, the test data for sealant retention (nR-) was obtained from clinical fissure sealant trials and compared to random values (nrand) in relation to the true clinical endpoint (= caries manifestation / nC-) and computed as median Diagnostic Odds ratios (DOR). It was expected, on the basis of current consensus concerning fissure retention [15-20], that the predictive DOR values from the test data would be significantly higher (more accurate) than those from the control. However, no statistically significant differences between the TP/TN and FP/FN dependent DOR values were found (Wilcoxon test: z = 0.56; p = 0.58; Sign test: z = 1.38; p = 0.17). The null-hypothesis (H0) could therefore not be rejected, thus no sufficient evidence in support of the alternate hypothesis (H1) was found. These results suggest that predictions based on the relationship of the retention rate of resin-based fissure sealants and caries manifestation are no more accurate than random guesses. Furthermore, the SROC curve (Fig. 2), based on such random values (median DOR 0.28), appears to indicate a higher (albeit inverted and still purely random) predictive accuracy than the sealant retention rate (DOR = 1.21; Fig. 1).

The lack of any significant difference in predictive accuracy between the retention rate and random values may not be surprising. Many factors other than the mere loss of complete sealant retention have a potential effect on caries manifestation in pits and fissures of permanent molar teeth. These factors may include: the position of the molar tooth in the mouth, the period between retention loss and follow-up, access to fluoride, oral hygiene and dietary habits, enamel structure, saliva factors or, possibly, factors that remain unknown. Manifestation of caries may also be related to sealed pits and fissures, if the partially lost sealants were misclassified as ‘completely retained’ or if pits and fissures still contain micro-remnants of sealant materials that provide caries protection but were classified as ‘loss of complete retention’.

Against this background, it may appear obvious that the retention rate alone is unable to account for the influence of other known and unknown factors that may influence caries manifestation. Nonetheless, the retention rate of fissure sealants has been accepted as valid surrogate for caries prevention [21-25]. Valid surrogate endpoints are defined as measurements or physical signs for use as substitutes for true clinical endpoints [34], while true clinical endpoints are defined as clinical, patient-relevant events of which the patient is aware, which the patient wants to avoid and which affect her/his quality of life [21]. Caries manifestation in pits and fissures constitutes a true clinical endpoint that was utilized during first generation of fissure sealant trials [9-11]. In light of the inability of the retention rate to be an accurate predictor of caries in pits and fissures, further (randomised controlled) sealant trials should have retained this clinical endpoint as the outcomes measure while using resin-based sealant materials and sealant techniques, established during the previous trials, as gold standards against which new materials/techniques were to be compared.

The use of invalid surrogates that cannot sufficiently predict their clinical endpoints carries the danger of unexpected adverse effects in clinical trials, therapeutic uncertainties due to ambiguous evidence, as well as the risk of rejection of potentially useful therapies because they do not show benefits in line with the surrogate endpoints [24]. Of these consequences, the latter is especially apparent with regard to glass-ionomer cement (GIC)-based fissure sealants [20], despite the fact that systematic reviews have found no difference between the caries-preventive effect of GIC and that of resin-based sealants [26, 27]. For these reasons, it has been repeatedly argued that: (i) biological outcomes should take precedence over mechanical ones; (ii) because sealants are placed in pits and fissures in order to prevent the onset of caries manifestation or to arrest it, the true outcome of such intervention should therefore be expressed in terms of how well such intervention has managed to achieve this objective [35, 36].


The retention rate of resin-based fissure sealants is not a valid predictor for clinical outcomes. Nevertheless, complete retention of sealants in pits and fissures has been established as a beneficial factor in preventing the development of carious lesions/cavities. On this basis the aim of achieving complete retention, regardless of the type of material, remains an important clinical consideration in the placing of fissure sealants. However, inter-category misinterpretation of ‘beneficial factors’ as ‘valid surrogate endpoints’ should be avoided. In order for a surrogate endpoint to be valid, it needs not only to be a beneficial factor but also to be a sufficiently accurate predictor (beyond the play of chance) of its true clinical endpoint. The retention rate of fissure sealants appears not to fulfil this criterion and thus seems unable to replace ‘caries manifestation’ as the appropriate study outcome in clinical fissure sealant trials.


One of the authors (SM) has been actively involved in the promotion; teaching and research of GIC fissure sealants in the past (from 1998 – 2007).


The authors confirm that this article content has no conflicts of interest.


The authors thank Mrs. Joy Hull for editing the grammar and syntax of the manuscript of this report.


Appendices are available on the publisher’s web site along with the published article.

Download File

Download File

Download File


[1] Splieth CH, Ekstrand KR, Alkilzy M, et al. Sealants in dentistry: outcomes of the ORCA Saturday Afternoon Symposium 2007 Caries Res 2010; 44: 3-13.
[2] Keyes P. Present and future measures for caries control J Am Dent Assoc 1969; 79: 1395-404.
[3] Gore DR. The use of dental sealants in adults: a long-neglected preventive measure Int J Dent Hyg 2010; 8: 198-203.
[4] Wallis CP. Dental materials which can be used to reduce caries in fissures J Dent 1973; 1: 211-5.
[5] Miller WD. The preventive treatment of teeth with special reference to nitrate of silver Dent Cosmos 1905; 47: 913-2.
[6] Miller J. A clinical investigation in preventive dentistry Br Dent J 1951; 91(4): 92-8.
[7] Mejàre I, Lingström P, Petersson LG, et al. Caries-preventive effect of fissure sealants: a systematic review Acta Odontol Scand 2003; 61: 321-0.
[8] Buonocore MG. A simple method of Increasing the adhesion of acrylic filling materials to enamel surfaces J Dent Res 1955; 34: 849-53.
[9] Horowitz HS, Heifetz SB, Poulsen S. Retention and effectiveness of a single application of an adhesive sealant in preventing occlusal caries: final report after five years of a study in Kalispell, Montana J Am Dent Assoc 1977; 95: 1133-9.
[10] Going RE, Haugh LD, Grainger DA, Conti AJ. Four-year clinical evaluation of a pit and fissure sealant J Am Dent Assoc 1977; 95: 72-81.
[11] Thylstrup A, Poulsen S. Retention and effectiveness of a chemically polymerized pit and fissure sealant after 2 years Scand J Dent Res 1978; 86: 21-4.
[12] Ahovuo-Saloranta A, Hiiri A, Nordblad A, Mäkelä M, Worthington HV. Pit and fissure sealants for preventing dental decay in the permanent teeth of children and adolescents Cochrane Database Syst Rev 2008; 4: CD001830.
[13] Locker D, Jokovic A, Kay EJ. Prevention. part 8: the use of pit and fissure sealants in preventing caries in the permanent dentition of children Br Dent J 2003; 195: 375-8.
[14] Muller-Bolla M, Lupi-Pégurier L, Tardieu C, Velly AM, Antomarchi C. Retention of resin-based pit and fissure sealants: a systematic review Community Dent Oral Epidemiol 2006; 34: 321-6.
[15] Rock WP, Anderson RJ. A review of published fissure sealant trials using multiple regression analysis J Dent 1982; 10: 39-43.
[16] Weintraub JA, Burt BA. Prevention of dental caries by the use of pit-and-fissure sealants J Public Health Policy 1987; 8: 542-60.
[17] Liebenberg WH. The fissure sealant impasse Quintessence Int 1994; 25: 741-5.
[18] Beauchamp J, Caufield PW, Crall JJ, et al. Evidence-based clinical recommendations for the use of pit-and-fissure sealants: a report of the American Dental Association Council on scientific affairs J Am Dent Assoc 2008; 139: 257-68.
[19] Sasa I, Donly KJ. Sealants: a review of the materials and utilization J Calif Dent Assoc 2010; 38: 730.
[20] Kühnisch J, Mansmann U, Heinrich-Weltzien R, Hickel R. Longevity of materials for pit and fissure sealing--results from a meta-analysis Dent Mater 2012; 28: 298-303.
[21] Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996; 125: 605-13.
[22] Prentice RL. Surrogate endpoints in clinical trials: Definitions and operational criteria Stat Med 1989; 8: 431-0.
[23] Gupta K, Gupta J, Singh S. Surrogate endpoints: how reliable are they? J Clin Res Best Pract 2010; 8: 1-6.
[24] Holloway RG, Dick AW. Clinical trial end points: on the road to nowhere? Neurology 2002; 58: 679-86.
[25] Ellenberg SS. Surrogate end points in clinical trials BMJ 1991; 302: 63-4.
[26] Mickenautsch S, Yengopal V. Caries-preventive effect of glass ionomer and resin-based fissure sealants on permanent teeth: an update of systematic review evidence BMC Res Notes 2011; 4: 22.
[27] Yengopal V, Mickenautsch S. Resin-modified glass-ionomer cements versus resin-based materials as fissure sealants - a meta-analysis of clinical trials Eur Arch Paediatr Dent 2010; 11: 18-25.
[28] Website. Random Number Generator Available from: [Accessed 1st October 2012];
[29] Deeks JJ. Systematic reviews of evaluations of diagnostic and screening tests In: Egger M, Smith GD, Altman DG, Eds. Systematic reviews in health care. UK: BMJ Publishing group 2001; p. 248.
[30] Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: a single indicator of test performance J Clin Epidemiol 2003; 56: 1129-35.
[31] Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. Meta-DiSc: a software for meta-analysis of test accuracy data BMC Med Res Methodol 2006; 6: 31.
[32] Higgins JPT, Green S, Eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011] – Section 9.5.2. The Cochrane Collaboration, 2011 Available from: [Accessed: 3rd November 2012];
[33] Green SB, Salkind NJ. Using SPSS for Window and Macintosh: Analyzing and understanding data. 5th ed. Upper Saddle River, NJ: Pearson Prentice Hall 2008.
[34] Temple RJ. A regulatory authority's opinion about surrogate endpoints In: Nimmo WS, Tucker GT, Eds. Clinical measurement in drug evaluation. New York: J Wiley 1995.
[35] Frencken JE, Holmgren CJ. Atraumatic restorative treatment for dental caries. Nijmegen: STI Book b.v 1999.
[36] Holmgren CJ, Figueredo MC. Two decades of ART: improving on success through further research J Appl Oral Sci 2009; 17(Suppl): 122-33.