Assessing Treatment Outcomes of a Graduate Orthodontic Program
Abstract
Objectives:
To assess treatment outcomes of a graduate orthodontic program during two different periods.
Materials and Methods:
Consecutive orthodontic patients’ files were selected from the archives of the Graduate Orthodontic Program, Aristotle University of Thessaloniki, Greece. Following the application of certain inclusion criteria, the final sample consisted of 109 patients. The sample was allocated into two groups depending on the time of treatment [Group A: 1998-2003, (n=60); Group B: 2004-2009 (n=49)]. The first period started a few years after the inception of the program and the second 10 years later. All pre- and post-treatment dental casts were blindly assessed by one investigator using the Peer Assessment Rating (PAR) and the Index of Complexity, Outcome and Need (ICON). Statistical analysis included Student’s t-test, Mann-Whitney U test, Pearson Chi-Square test and Spearman correlation coefficient. The level of significance was set at p < 0.05.
Results:
In Group A, the mean PAR index changed from 28.6 before treatment to 5.5 after treatment and the mean ICON index changed from 71.9 to 23.5. In Group B, the mean PAR index changed from 23.6 before treatment to 4 after treatment and the mean ICON index changed from 62.8 to 19.8. The mean PAR score reduction was 78.4% for the 1st group and 81.4% for the 2nd group, respectively. Ninety percent of the cases of Group A and 89.8% of the cases of Group B had a post-treatment ICON score < 31. The severity of the initial malocclusion was found to be positively correlated with the treatment occlusal outcome. Out of the 109 cases, 68 were considered as substantially or greatly improved, 29 moderately improved, 8 showed minimal improvement and 4 cases were considered as not improved or worse. The mean PAR percentage improvement and the ICON score at the end of treatment were not correlated to the presence or absence of tooth extractions in the treatment plan. There was no correlation between the treatment outcome and the number of graduate residents involved in the therapy. The treatment outcomes were not correlated to the gender or age of patients.
Conclusion:
Patients treated by graduate orthodontic residents during 1998-2009 demonstrated significant improvement of their occlusion and the quality of the treatment remained constant throughout the years. The outcome of orthodontic treatment was not correlated to the gender and the age of patients, the number of postgraduate students performing the treatment, and the presence of tooth extractions in the treatment plan.
1. INTRODUCTION
Efforts have been frequently made to assess the treatment outcomes in various graduate orthodontic programs [1-6]. Positive outcomes may get favorable reviews, thus enhancing the reputation and prestige of an orthodontic department as well as the higher education institution in general [7]. Often, an assessment of the treatment outcomes in a graduate orthodontic clinic could provide significant information useful in raising the level of the program and the clinical services provided [3].
Such an evaluation may be qualitative or quantitative. However, in the era of evidence-based science in evaluating orthodontic treatment outcomes, priority is given to quantitative methods [8]. Orthodontists are trained in different educational environments worldwide and at different time periods. However, the use of international objective criteria is necessary for a reliable, consistent, accurate and comparable evaluation of malocclusions and orthodontic treatment outcomes [9]. This need led to the establishment of occlusal indices.
Occlusal indices are quantitative assessment tools employing continuous or numbered scales to assess occlusion. Occlusal features assessed using a particular index are assigned a specific numerical value and their degree is evaluated according to the index type. Orthodontic treatment outcomes may be assessed by different indices and in Europe PAR (Peer Assessment Rating) and ICON (Index of Complexity, Outcome and Need) are widely used to evaluate whether the therapy has been successful or not.
PAR has been developed to objectively assess the outcome of orthodontic treatment [10, 11]. The British Orthodontic Standards Working Party was involved in producing this in its current form following the assessment of 320 dental casts by 74 dentists and orthodontists [2]. It is now used in more than 50 countries worldwide to assess orthodontic results in daily clinical practice as well as for research purposes. The index records and evaluates different components related to: 1. Upper and lower anterior segments; 2. Right and left buccal occlusion in all three planes of space; 3. Overjet; 4. Overbite; and 5. Centerline. All measurements are made on pre- and post- treatment dental casts using a special ruler which briefly summarizes the recording features of the index.
Daniels and Richmond considered the opinions of 97 orthodontists from 8 European countries and the USA to develop the ICON index [12]. These orthodontists gave subjective judgements on the need for treatment, treatment complexity and treatment improvement on a sample of 240 pre-treatment and 98 post-treatment study models of patients 11-13 years old. The ICON is unique in incorporating an esthetic score as an integral part of the treatment needs evaluation. It is a multifunctional index which is used to assess the need for orthodontic treatment and treatment complexity, as well as to evaluate treatment outcome. The ICON encompasses 5 individual occlusal features: 1. Aesthetic component. This consists of 10 photographs showing 10 different levels of dental attractiveness, with grade 1 representing the most attractive and grade 10 the least attractive dentitions. The scale was constructed using 1000 dental photographs of 12-year-old children in a survey carried out by Evans and Shaw [13]. The photographs were rated by a panel of six lay judges; 2. Upper arch crowding or spacing; 3. Presence of a crossbite; 4. Degree of incisor open bite or overbite; 5. The fit of the teeth in the buccal segment in terms of the anteroposterior relationship. All measurements of the features 2-5 are made on pre- and post-treatment dental casts. To assess the orthodontic treatment outcome, the formula “Pre-treatment score – 4x post-treatment score” is used.
Orthodontic treatments performed by graduate residents at the Department of Orthodontics, Aristotle University of Thessaloniki, Greece, are supervised by orthodontic specialists who are faculty teaching staff. The orthodontic graduate program started in 1994 and an assessment of the treatment outcomes a few years after its inception and 10 years later could be very informative regarding the quality of clinical training. Therefore, the aim of this study was to assess the outcome of orthodontic treatment provided at the graduate orthodontic clinic during the period of 1998-2009 using the PAR and ICON indices.
2. MATERIALS AND METHODS
Consecutive orthodontic patients’ files, including dental casts, were selected from the archives of the clinic. The final sample was formed following application of certain criteria including files with complete records, excellent quality of dental casts, healthy individuals, with complete and physiologic permanent dentition (excluding 3rd molars), and no history of previous orthodontic therapy. Orthognathic surgery patients were excluded. The final sample consisted of 109 patients (mean age at start of treatment: 14.1y {min. 10.14y and max. 27.84y}) who received comprehensive orthodontic treatment by means of fixed appliances in both dental arches (prescription Ricketts, slot 0,018”) (mean treatment duration: 28.5 months). The sample was divided into two groups depending on the period when the orthodontic treatment took place. Group A included patients whose orthodontic treatment started in 1998 and was completed by 2003 (n=60) and Group B included the period from 2004 to 2009 (n=49). All pre- and post-treatment dental casts were blindly assessed by the first author using the PAR and ICON indices (Figs. 1 and 2).
In order to test the intra-examiner reliability, the examiner re-assessed 10 cases, which were randomly selected from the original sample, one week after the initial examinations. The intra-examiner reliability in the use of PAR and ICON was assessed using the Root Mean Square (RMS) for the method error according to Richmond and co-workers [14-17]. The RMS is given by the formula:
 |
It is calculated as the square root of sum of the squared differences between rater and standard score, divided by twice the sample size. The RMS error is an estimate of the standard deviation representing the measurement errors of a single measurement. It summarizes both random error and bias error, if present [16].
All data were inserted in an Excel file and analyzed using the Statistical Package for the Social Sciences (SPSS, version 20, Chicago, IL, USA). Statistical analysis included Student’s t-test, Mann-Whitney U test, Pearson Chi-Square test and Spearman correlation coefficient. The level of significance was set at p < 0.05.
3. RESULTS
3.1. Method Error
The RMS of the reproducibility exercise was 2.049 for PAR and 5.78 for ICON. Both scores indicated good reliability, since the criteria for RMS error acceptability are less than 5 for PAR and less than 9 for ICON.
The same reliability was also tested using the non-parametric correlation coefficient (Spearman rank Order), which showed a very high correlation (r = 0.714, p = 0.02 < 0.05 and r = 0.889, p = 0.002 < 0.05 for PAR and ICON, respectively).
3.2. PAR Index
The mean PAR percentage improvement, for the whole sample of 109 patients was 79.8% > 70%, which is considered as a good standard of orthodontic treatment. 57.8% of the sample was found to have a reduction in PAR score greater than 22 points, which indicates a great improvement.
When comparing the two groups, it was found that the mean pre-treatment PAR score for Group A was greater than that of Group B (28.6 and 23.6, respectively), indicating more severe malocclusions for the first group. That difference was found to be statistically significant (Student’s t-test, p = 0.014). The PAR percentage improvement for Group A was 78.4% and for Group B 81.4%, but this difference was not statistically significant (Mann-Whitney U test, p = 0.206).
3.3. ICON Index
Ninety-four out of the 109 cases were considered by the ICON score as needing treatment (Table 1). Regarding complexity, 60 cases fell into the difficult and very difficult categories (25 and 35, respectively). Twenty-three cases were considered as moderate while 26 cases were considered as mild. It was remarkable that none of the cases fell into the easy category (Table 2)
| Needs and Acceptability | Threshold | Number of Cases |
|---|---|---|
| Pre-treatment need | >43 treatment need | 94 |
| End treatment acceptability | <31 acceptable | 98 |
| Complexity Grade | Score Range | Frequency |
|---|---|---|
| Easy | <29 | 0 |
| Mild | 29 to 50 | 26 |
| Moderate | 51 to 63 | 23 |
| Difficult | 64 to 77 | 25 |
| Very difficult | >77 | 35 |
There was a statistically significant difference (Student’s t-test, p = 0.019) between the two groups in terms of the mean ICON score before the orthodontic intervention (71.9 and 62.8, respectively). Group A consisted of more severe malocclusions than Group B.
Out of the 109 cases, 68 cases were considered as substantially or greatly improved (35 and 33, respectively). The moderately improved cases numbered 29, while 8 cases had minimal improvement and 4 cases were considered not improved or worse (Table 3). Ninety-eight post-treatment occlusions were considered acceptable (Table 1).
| Improvement Grade | Score Range | Frequency |
|---|---|---|
| Greatly improved | >-1 | 33 |
| Substantially improved | -25 to -1 | 35 |
| Moderately improved | -53 to -26 | 29 |
| Minimally improved | -85 to -54 | 8 |
| Not improved or worse | <-85 | 4 |
3.4. Extractions
Thirty-six out of the 109 cases were treated with extractions, 26 in Group A and 10 in Group B.
The mean PAR percentage improvement and the ICON score at the end of orthodontic treatment were not correlated to the presence or absence of tooth extractions in the treatment plan (Mann-Whitney U test, p = 0.217; Pearson Chi-Square test, p = 0.355).
3.5. Number of Residents
In some cases, throughout treatment, there was a replacement of the treating resident because of graduation. Fifty-three out of the 109 cases were treated by one resident, 55 by two and only one by three. This last case was excluded from the statistical analysis in the data of this section.
There was no correlation of the treatment outcome to the number of residents involved in the therapy (%PAR reduction: Mann-Whitney U test, p = 0.190; ICON<31: Pearson Chi-Square, p = 0.54).
4. DISCUSSION
The present study aimed at assessing the occlusal outcome of orthodontic treatments performed by residents of Aristotle University of Thessaloniki, a few years after the inception of the orthodontic graduate program and 10 years later. Assessment of graduate programs of higher education institutions may directly affect their reputation and academic profile, as well as influence their financial growth [18]. Moreover, assessing the quality of treatment outcomes, might provide useful information that can be used for educational and clinical improvements [3].
Similar studies have been made in the past, in orthodontic graduate programs all over the world, using different types of occlusal indices [1, 3-6, 19-21]. One of the advantages of the current study is the use of two types of occlusal indices, namely PAR and ICON. The ICON index takes esthetics into consideration, which is a very important factor for the assessment of treatment success.
The reliability of the study was fortified by the fact that the first author was calibrated for the use of the two indices by their co-inventor Professor S. Richmond. All casts were blindly assessed and intra-examiner reliability was tested by re-assessing 10 randomly selected cases.
This study only included patients in permanent dentition. This fact reduced the number of the sample but controlled the possibilities for statistical errors, as the waiting time for all teeth to erupt might have influenced the duration and the outcome of orthodontic treatment.
The mean PAR score reduction was 79.8%. Similar studies have reported reductions of 63-78% [15, 19, 22-24]. It is noteworthy that 57.8% of the sample had a PAR score reduction greater than 22 points, something which indicates that these cases were classified as “greatly improved”.
Using the ICON index, only 11 out of the 109 cases (11.1%) were considered as not acceptable. This percentage could be characterized as quite low in comparison to the percentages reported by other studies conducted in community clinics in Sweden and Greece, with 29% and 12% respectively [15, 25].
The results showed that high standards of orthodontic care were provided in the clinic. In general, the degree of improvement of a case depends on the severity of the initial malocclusion. Richmond and co-workers [24] pointed out that the more severe the initial malocclusion, the greater the improvement in the case will be. The findings of this study confirmed this opinion, as 55% of the sample initially belonged to the category “difficult or very difficult”. The percentages found in two other relevant studies, in the USA [20] and in Sweden [15] were 60% and 74%, respectively.
Extractions did not seem to affect the treatment outcome as was found elsewhere [26]. The percentage of PAR reduction was 78% in extraction cases and 83% without extractions, with the difference being not statistically significant.
No difference in the treatment outcome was found when two residents were involved. These findings are in accordance with previous studies [1, 27, 28].
Moreover, the treatment outcome was not correlated to gender. Likewise, no such correlation was found by Birkeland et al., [22], and by Taylor et al., [29], either. On the contrary, in two other studies [19, 30] it was reported that the treatment outcomes of young female patients were better, but they could not explain why this occurred.
The sample of the present study was further analyzed after being divided into two groups, depending on the period when orthodontic treatment took place. Group A consisted of patients who were treated a few years after the graduate program has started and Group B included patients when the program was well established. This was done in order to assess whether the treatment outcomes of the two periods were statistically different. The respective percentage of PAR reduction was 78.4% and 81.4% for the two groups and the difference was not statistically significant. It seems that time and any other changes (e.g., faculty, materials, facilities) during the periods of assessment did not affect the quality of the provided orthodontic treatment, which remained constant throughout the years.
Two studies were published recently regarding occlusal outcome assessment of orthodontic treatments performed at graduate programs in Spain and the United Arab Emirates. Their results were quite similar with the ones of the present investigation although they utilized significantly smaller samples (50 and 30 patients, respectively) [31, 32].
The first study used only the PAR index. The mean percentage of PAR reduction was 80.5%, while in ours it was 79.8%. A PAR score reduction greater than 22 points (greatly improved) was noticed in 44% of their cases, while for our study these cases were up to 57.8%. This investigation also concluded that gender, age at start of the treatment, treatment methods, extraction-non extraction treatments, and the number of residents treating each case did not influence treatment outcomes with regards to PAR values [31].
The second study used both the PAR index and the ICON. Its results indicated that the occlusal outcome related to the orthodontic care provided was characterized by significant improvement. The mean PAR changed from 19.43 before treatment to 4.63 after and, the mean ICON changed from 53.96 before treatment to 19.06 afterward. “Great improvement” was noticed according to PAR in 46.67% of patients and according to ICON in 23.33% of patients [32].
Based on the findings of the most recent reports it may be noted that the great majority of the examined patients treated at various graduate orthodontic programs demonstrated significant improvement in their occlusion after therapy.
CONCLUSION
The results of this investigation showed that high standards of orthodontic care were provided in the graduate clinic and that the quality of the treatment remained constant throughout the years. The outcome of orthodontic treatment was not correlated to the gender and age of patients, the number of graduate residents performing the treatment or the presence of tooth extractions in the treatment plan. On the contrary, the severity of the initial malocclusion was found to be positively correlated with the treatment outcome.
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
This study was analyzed and approved by the Aristotle University of Thessaloniki, School of Dentistry Ethics Committee (no. 310/07.12.2012).
HUMAN AND ANIMAL RIGHTS
No animals were used in this research. All research procedures followed were in accordance with the ethical standards of the committee responsible for human experimentation (institutional and national), and with the Helsinki Declaration of 1975, as revised in 2008.
CONSENT FOR PUBLICATION
Not applicable.
CONFLICTS OF INTEREST
The authors declare no conflict of interest, financial or otherwise.
ACKNOWLEDGEMENTS
Declared none.
