Assessing Treatment Outcomes of a Graduate Orthodontic Program
RESEARCH ARTICLE

Assessing Treatment Outcomes of a Graduate Orthodontic Program

The Open Dentistry Journal 30 Oct 2018 RESEARCH ARTICLE DOI: 10.2174/1874210601812010896

Abstract

Objectives:

To assess treatment outcomes of a graduate orthodontic program during two different periods.

Materials and Methods:

Consecutive orthodontic patients’ files were selected from the archives of the Graduate Orthodontic Program, Aristotle University of Thessaloniki, Greece. Following the application of certain inclusion criteria, the final sample consisted of 109 patients. The sample was allocated into two groups depending on the time of treatment [Group A: 1998-2003, (n=60); Group B: 2004-2009 (n=49)]. The first period started a few years after the inception of the program and the second 10 years later. All pre- and post-treatment dental casts were blindly assessed by one investigator using the Peer Assessment Rating (PAR) and the Index of Complexity, Outcome and Need (ICON). Statistical analysis included Student’s t-test, Mann-Whitney U test, Pearson Chi-Square test and Spearman correlation coefficient. The level of significance was set at p < 0.05.

Results:

In Group A, the mean PAR index changed from 28.6 before treatment to 5.5 after treatment and the mean ICON index changed from 71.9 to 23.5. In Group B, the mean PAR index changed from 23.6 before treatment to 4 after treatment and the mean ICON index changed from 62.8 to 19.8. The mean PAR score reduction was 78.4% for the 1st group and 81.4% for the 2nd group, respectively. Ninety percent of the cases of Group A and 89.8% of the cases of Group B had a post-treatment ICON score < 31. The severity of the initial malocclusion was found to be positively correlated with the treatment occlusal outcome. Out of the 109 cases, 68 were considered as substantially or greatly improved, 29 moderately improved, 8 showed minimal improvement and 4 cases were considered as not improved or worse. The mean PAR percentage improvement and the ICON score at the end of treatment were not correlated to the presence or absence of tooth extractions in the treatment plan. There was no correlation between the treatment outcome and the number of graduate residents involved in the therapy. The treatment outcomes were not correlated to the gender or age of patients.

Conclusion:

Patients treated by graduate orthodontic residents during 1998-2009 demonstrated significant improvement of their occlusion and the quality of the treatment remained constant throughout the years. The outcome of orthodontic treatment was not correlated to the gender and the age of patients, the number of postgraduate students performing the treatment, and the presence of tooth extractions in the treatment plan.

Keywords: ICON, PAR, Orthodontic treatment, Graduate orthodontic education, Root Mean Square (RMS), Occlusal indices.

1. INTRODUCTION

Efforts have been frequently made to assess the treatment outcomes in various graduate orthodontic programs [1-6]. Positive outcomes may get favorable reviews, thus enhancing the reputation and prestige of an orthodontic department as well as the higher education institution in general [7]. Often, an assessment of the treatment outcomes in a graduate orthodontic clinic could provide significant information useful in raising the level of the program and the clinical services provided [3].

Such an evaluation may be qualitative or quantitative. However, in the era of evidence-based science in evaluating orthodontic treatment outcomes, priority is given to quantitative methods [8]. Orthodontists are trained in different educational environments worldwide and at different time periods. However, the use of international objective criteria is necessary for a reliable, consistent, accurate and comparable evaluation of malocclusions and orthodontic treatment outcomes [9]. This need led to the establishment of occlusal indices.

Occlusal indices are quantitative assessment tools employing continuous or numbered scales to assess occlusion. Occlusal features assessed using a particular index are assigned a specific numerical value and their degree is evaluated according to the index type. Orthodontic treatment outcomes may be assessed by different indices and in Europe PAR (Peer Assessment Rating) and ICON (Index of Complexity, Outcome and Need) are widely used to evaluate whether the therapy has been successful or not.

PAR has been developed to objectively assess the outcome of orthodontic treatment [10, 11]. The British Orthodontic Standards Working Party was involved in producing this in its current form following the assessment of 320 dental casts by 74 dentists and orthodontists [2]. It is now used in more than 50 countries worldwide to assess orthodontic results in daily clinical practice as well as for research purposes. The index records and evaluates different components related to: 1. Upper and lower anterior segments; 2. Right and left buccal occlusion in all three planes of space; 3. Overjet; 4. Overbite; and 5. Centerline. All measurements are made on pre- and post- treatment dental casts using a special ruler which briefly summarizes the recording features of the index.

Daniels and Richmond considered the opinions of 97 orthodontists from 8 European countries and the USA to develop the ICON index [12]. These orthodontists gave subjective judgements on the need for treatment, treatment complexity and treatment improvement on a sample of 240 pre-treatment and 98 post-treatment study models of patients 11-13 years old. The ICON is unique in incorporating an esthetic score as an integral part of the treatment needs evaluation. It is a multifunctional index which is used to assess the need for orthodontic treatment and treatment complexity, as well as to evaluate treatment outcome. The ICON encompasses 5 individual occlusal features: 1. Aesthetic component. This consists of 10 photographs showing 10 different levels of dental attractiveness, with grade 1 representing the most attractive and grade 10 the least attractive dentitions. The scale was constructed using 1000 dental photographs of 12-year-old children in a survey carried out by Evans and Shaw [13]. The photographs were rated by a panel of six lay judges; 2. Upper arch crowding or spacing; 3. Presence of a crossbite; 4. Degree of incisor open bite or overbite; 5. The fit of the teeth in the buccal segment in terms of the anteroposterior relationship. All measurements of the features 2-5 are made on pre- and post-treatment dental casts. To assess the orthodontic treatment outcome, the formula “Pre-treatment score – 4x post-treatment score” is used.

Orthodontic treatments performed by graduate residents at the Department of Orthodontics, Aristotle University of Thessaloniki, Greece, are supervised by orthodontic specialists who are faculty teaching staff. The orthodontic graduate program started in 1994 and an assessment of the treatment outcomes a few years after its inception and 10 years later could be very informative regarding the quality of clinical training. Therefore, the aim of this study was to assess the outcome of orthodontic treatment provided at the graduate orthodontic clinic during the period of 1998-2009 using the PAR and ICON indices.

2. MATERIALS AND METHODS

Consecutive orthodontic patients’ files, including dental casts, were selected from the archives of the clinic. The final sample was formed following application of certain criteria including files with complete records, excellent quality of dental casts, healthy individuals, with complete and physiologic permanent dentition (excluding 3rd molars), and no history of previous orthodontic therapy. Orthognathic surgery patients were excluded. The final sample consisted of 109 patients (mean age at start of treatment: 14.1y {min. 10.14y and max. 27.84y}) who received comprehensive orthodontic treatment by means of fixed appliances in both dental arches (prescription Ricketts, slot 0,018”) (mean treatment duration: 28.5 months). The sample was divided into two groups depending on the period when the orthodontic treatment took place. Group A included patients whose orthodontic treatment started in 1998 and was completed by 2003 (n=60) and Group B included the period from 2004 to 2009 (n=49). All pre- and post-treatment dental casts were blindly assessed by the first author using the PAR and ICON indices (Figs. 1 and 2).

Fig. (1). Mean pre-treatment and post- treatment PAR scores for groups A and B, respectively.
Fig. (2). Mean pre-treatment and post- treatment ICON scores for groups A and B, respectively.

In order to test the intra-examiner reliability, the examiner re-assessed 10 cases, which were randomly selected from the original sample, one week after the initial examinations. The intra-examiner reliability in the use of PAR and ICON was assessed using the Root Mean Square (RMS) for the method error according to Richmond and co-workers [14-17]. The RMS is given by the formula:

It is calculated as the square root of sum of the squared differences between rater and standard score, divided by twice the sample size. The RMS error is an estimate of the standard deviation representing the measurement errors of a single measurement. It summarizes both random error and bias error, if present [16].

All data were inserted in an Excel file and analyzed using the Statistical Package for the Social Sciences (SPSS, version 20, Chicago, IL, USA). Statistical analysis included Student’s t-test, Mann-Whitney U test, Pearson Chi-Square test and Spearman correlation coefficient. The level of significance was set at p < 0.05.

3. RESULTS

3.1. Method Error

The RMS of the reproducibility exercise was 2.049 for PAR and 5.78 for ICON. Both scores indicated good reliability, since the criteria for RMS error acceptability are less than 5 for PAR and less than 9 for ICON.

The same reliability was also tested using the non-parametric correlation coefficient (Spearman rank Order), which showed a very high correlation (r = 0.714, p = 0.02 < 0.05 and r = 0.889, p = 0.002 < 0.05 for PAR and ICON, respectively).

3.2. PAR Index

The mean PAR percentage improvement, for the whole sample of 109 patients was 79.8% > 70%, which is considered as a good standard of orthodontic treatment. 57.8% of the sample was found to have a reduction in PAR score greater than 22 points, which indicates a great improvement.

When comparing the two groups, it was found that the mean pre-treatment PAR score for Group A was greater than that of Group B (28.6 and 23.6, respectively), indicating more severe malocclusions for the first group. That difference was found to be statistically significant (Student’s t-test, p = 0.014). The PAR percentage improvement for Group A was 78.4% and for Group B 81.4%, but this difference was not statistically significant (Mann-Whitney U test, p = 0.206).

3.3. ICON Index

Ninety-four out of the 109 cases were considered by the ICON score as needing treatment (Table 1). Regarding complexity, 60 cases fell into the difficult and very difficult categories (25 and 35, respectively). Twenty-three cases were considered as moderate while 26 cases were considered as mild. It was remarkable that none of the cases fell into the easy category (Table 2)

Table 1.
ICON orthodontic pre-treatment needs and end of treatment acceptability levels of the cases.
Needs and Acceptability Threshold Number of Cases
Pre-treatment need >43 treatment need 94
End treatment acceptability <31 acceptable 98
Table 2.
ICON complexity grade (pre-treatment) of the cases.
Complexity Grade Score Range Frequency
Easy <29 0
Mild 29 to 50 26
Moderate 51 to 63 23
Difficult 64 to 77 25
Very difficult >77 35

There was a statistically significant difference (Student’s t-test, p = 0.019) between the two groups in terms of the mean ICON score before the orthodontic intervention (71.9 and 62.8, respectively). Group A consisted of more severe malocclusions than Group B.

Out of the 109 cases, 68 cases were considered as substantially or greatly improved (35 and 33, respectively). The moderately improved cases numbered 29, while 8 cases had minimal improvement and 4 cases were considered not improved or worse (Table 3). Ninety-eight post-treatment occlusions were considered acceptable (Table 1).

Table 3.
ICON improvement grade (pre-treatment score – 4x post-treatment score) of the cases.
Improvement Grade Score Range Frequency
Greatly improved >-1 33
Substantially improved -25 to -1 35
Moderately improved -53 to -26 29
Minimally improved -85 to -54 8
Not improved or worse <-85 4

3.4. Extractions

Thirty-six out of the 109 cases were treated with extractions, 26 in Group A and 10 in Group B.

The mean PAR percentage improvement and the ICON score at the end of orthodontic treatment were not correlated to the presence or absence of tooth extractions in the treatment plan (Mann-Whitney U test, p = 0.217; Pearson Chi-Square test, p = 0.355).

3.5. Number of Residents

In some cases, throughout treatment, there was a replacement of the treating resident because of graduation. Fifty-three out of the 109 cases were treated by one resident, 55 by two and only one by three. This last case was excluded from the statistical analysis in the data of this section.

There was no correlation of the treatment outcome to the number of residents involved in the therapy (%PAR reduction: Mann-Whitney U test, p = 0.190; ICON<31: Pearson Chi-Square, p = 0.54).

3.5.1. Gender and age

The treatment outcomes were not correlated to gender (PAR reduction: Mann-Whitney U test, p = 0,074; ICON < 31: Pearson Chi-Square, p = 0,290) or age at the start of treatment (Spearman’s rho, p = 0.48).

4. DISCUSSION

The present study aimed at assessing the occlusal outcome of orthodontic treatments performed by residents of Aristotle University of Thessaloniki, a few years after the inception of the orthodontic graduate program and 10 years later. Assessment of graduate programs of higher education institutions may directly affect their reputation and academic profile, as well as influence their financial growth [18]. Moreover, assessing the quality of treatment outcomes, might provide useful information that can be used for educational and clinical improvements [3].

Similar studies have been made in the past, in orthodontic graduate programs all over the world, using different types of occlusal indices [1, 3-6, 19-21]. One of the advantages of the current study is the use of two types of occlusal indices, namely PAR and ICON. The ICON index takes esthetics into consideration, which is a very important factor for the assessment of treatment success.

The reliability of the study was fortified by the fact that the first author was calibrated for the use of the two indices by their co-inventor Professor S. Richmond. All casts were blindly assessed and intra-examiner reliability was tested by re-assessing 10 randomly selected cases.

This study only included patients in permanent dentition. This fact reduced the number of the sample but controlled the possibilities for statistical errors, as the waiting time for all teeth to erupt might have influenced the duration and the outcome of orthodontic treatment.

The mean PAR score reduction was 79.8%. Similar studies have reported reductions of 63-78% [15, 19, 22-24]. It is noteworthy that 57.8% of the sample had a PAR score reduction greater than 22 points, something which indicates that these cases were classified as “greatly improved”.

Using the ICON index, only 11 out of the 109 cases (11.1%) were considered as not acceptable. This percentage could be characterized as quite low in comparison to the percentages reported by other studies conducted in community clinics in Sweden and Greece, with 29% and 12% respectively [15, 25].

The results showed that high standards of orthodontic care were provided in the clinic. In general, the degree of improvement of a case depends on the severity of the initial malocclusion. Richmond and co-workers [24] pointed out that the more severe the initial malocclusion, the greater the improvement in the case will be. The findings of this study confirmed this opinion, as 55% of the sample initially belonged to the category “difficult or very difficult”. The percentages found in two other relevant studies, in the USA [20] and in Sweden [15] were 60% and 74%, respectively.

Extractions did not seem to affect the treatment outcome as was found elsewhere [26]. The percentage of PAR reduction was 78% in extraction cases and 83% without extractions, with the difference being not statistically significant.

No difference in the treatment outcome was found when two residents were involved. These findings are in accordance with previous studies [1, 27, 28].

Moreover, the treatment outcome was not correlated to gender. Likewise, no such correlation was found by Birkeland et al., [22], and by Taylor et al., [29], either. On the contrary, in two other studies [19, 30] it was reported that the treatment outcomes of young female patients were better, but they could not explain why this occurred.

The sample of the present study was further analyzed after being divided into two groups, depending on the period when orthodontic treatment took place. Group A consisted of patients who were treated a few years after the graduate program has started and Group B included patients when the program was well established. This was done in order to assess whether the treatment outcomes of the two periods were statistically different. The respective percentage of PAR reduction was 78.4% and 81.4% for the two groups and the difference was not statistically significant. It seems that time and any other changes (e.g., faculty, materials, facilities) during the periods of assessment did not affect the quality of the provided orthodontic treatment, which remained constant throughout the years.

Two studies were published recently regarding occlusal outcome assessment of orthodontic treatments performed at graduate programs in Spain and the United Arab Emirates. Their results were quite similar with the ones of the present investigation although they utilized significantly smaller samples (50 and 30 patients, respectively) [31, 32].

The first study used only the PAR index. The mean percentage of PAR reduction was 80.5%, while in ours it was 79.8%. A PAR score reduction greater than 22 points (greatly improved) was noticed in 44% of their cases, while for our study these cases were up to 57.8%. This investigation also concluded that gender, age at start of the treatment, treatment methods, extraction-non extraction treatments, and the number of residents treating each case did not influence treatment outcomes with regards to PAR values [31].

The second study used both the PAR index and the ICON. Its results indicated that the occlusal outcome related to the orthodontic care provided was characterized by significant improvement. The mean PAR changed from 19.43 before treatment to 4.63 after and, the mean ICON changed from 53.96 before treatment to 19.06 afterward. “Great improvement” was noticed according to PAR in 46.67% of patients and according to ICON in 23.33% of patients [32].

Based on the findings of the most recent reports it may be noted that the great majority of the examined patients treated at various graduate orthodontic programs demonstrated significant improvement in their occlusion after therapy.

CONCLUSION

The results of this investigation showed that high standards of orthodontic care were provided in the graduate clinic and that the quality of the treatment remained constant throughout the years. The outcome of orthodontic treatment was not correlated to the gender and age of patients, the number of graduate residents performing the treatment or the presence of tooth extractions in the treatment plan. On the contrary, the severity of the initial malocclusion was found to be positively correlated with the treatment outcome.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

This study was analyzed and approved by the Aristotle University of Thessaloniki, School of Dentistry Ethics Committee (no. 310/07.12.2012).

HUMAN AND ANIMAL RIGHTS

No animals were used in this research. All research procedures followed were in accordance with the ethical standards of the committee responsible for human experimentation (institutional and national), and with the Helsinki Declaration of 1975, as revised in 2008.

CONSENT FOR PUBLICATION

Not applicable.

CONFLICTS OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGEMENTS

Declared none.

REFERENCES

1
McGuinness NJ, McDonald JP. The influence of operator changes on orthodontic treatment times and results in a postgraduate teaching environment. Eur J Orthod 1998; 20(2): 159-67.
2
Hamdan AM, Rock WP. An appraisal of the Peer Assessment Rating (PAR) Index and a suggested new weighting system. Eur J Orthod 1999; 21(2): 181-92.
3
Yang-Powers LC, Sadowsky C, Rosenstein S, BeGole EA. Treatment outcome in a graduate orthodontic clinic using the American Board of Orthodontics grading system. Am J Orthod Dentofacial Orthop 2002; 122(5): 451-5.
4
Knierim K, Roberts WE, Hartsfield J. Assessing treatment outcomes for a graduate orthodontics program: Follow-up study for the classes of 2001-2003. Am J Orthod Dentofacial Orthop 2006; 130: 648-55.
5
Campbell CL, Roberts WE, Hartsfield JK Jr, Qi R. Treatment outcomes in a graduate orthodontic clinic for cases defined by the American Board of Orthodontics malocclusion categories. Am J Orthod Dentofacial Orthop 2007; 132(6): 822-9.
6
Vu CQ, Roberts WE, Hartsfield JK Jr, Ofner S. Treatment complexity index for assessing the relationship of treatment duration and outcomes in a graduate orthodontics clinic. Am J Orthod Dentofacial Orthop 2008; 133(1): 9.e1-9.e13.
7
Graham DH. Should we abolish ranking universities by their reputation? 2003. Available from: www.vanderbilt.edu/News/register/Jun5_00/ story12.html.Dec
8
Abei Y, Nelson S, Amberman BD, Hans MG. Comparing orthodontic treatment outcome between orthodontists and general dentists with the ABO index. Am J Orthod Dentofacial Orthop 2004; 126(5): 544-8.
9
Pangrazio-Kulbersh V, Kaczynski R, Shunock M. Early treatment outcome assessed by the Peer Assessment Rating index. Am J Orthod Dentofacial Orthop 1999; 115(5): 544-50.
10
Richmond S, Shaw WC, O’Brien KD, et al. The development of the PAR Index (Peer Assessment Rating): Reliability and validity. Eur J Orthod 1992; 14(2): 125-39.
11
Richmond S, Shaw WC, Roberts CT, Andrews M. The PAR Index (Peer Assessment Rating): Methods to determine outcome of orthodontic treatment in terms of improvement and standards. Eur J Orthod 1992; 14(3): 180-7.
12
Daniels C, Richmond S. The development of the index of complexity, outcome and need (ICON). J Orthod 2000; 27(2): 149-62.
13
Evans R, Shaw W. Preliminary evaluation of an illustrated scale for rating dental attractiveness. Eur J Orthod 1987; 9(4): 314-8.
14
Richmond S, Aylott NA, Panahei ME, Rolfe B, Tausche E. A 2-center comparison of orthodontist’s perceptions of orthodontic treatment difficulty. Angle Orthod 2001; 71(5): 404-10.
15
Richmond S, Ikonomou C, Williams B, Ramel S, Rolfe B, Kurol J. Orthodontic treatment standards in a public group practice in Sweden. Swed Dent J 2001; 25(4): 137-44.
16
Roberts CT, Richmond S. The design and analysis of reliability studies for the use of epidemiological and audit indices in orthodontics. Br J Orthod 1997; 24(2): 139-47.
17
Brown R, Richmond S. An update on the analysis of agreement for orthodontic indices. Eur J Orthod 2005; 27(3): 286-91.
18
Eliades T, Athanasiou AE. Advanced orthodontic education: Evolution of assessment criteria and methods to meet future challenges. Angle Orthod 2005; 75(2): 147-54.
19
Firestone AR, Häsler RU, Ingervall B. Treatment results in dental school orthodontic patients in 1983 and 1993. Angle Orthod 1999; 69(1): 19-26.
20
Onyeaso CO, BeGole EA. Orthodontic treatment standard in an accredited graduate orthodontic clinic in North America assessed using the Index of Complexity, Outcome and Need (ICON). Hel Orthod Rev 2006; 9: 23-34.
21
Mislik B, Konstantonis D, Katsadouris A, Eliades T. University clinic and private practice treatment outcomes in Class I extraction and nonextraction patients: A comparative study with the American Board of Orthodontics Objective Grading System. Am J Orthod Dentofacial Orthop 2016; 149(2): 253-8.
22
Birkeland K, Furevik J, Bøe OE, Wisth PJ. Evaluation of treatment and post-treatment changes by the PAR Index. Eur J Orthod 1997; 19(3): 279-88.
23
O’Brien KD, Shaw WC, Roberts CT. The use of occlusal indices in assessing the provision of orthodontic treatment by the hospital orthodontic service of England and Wales. Br J Orthod 1993; 20(1): 25-35.
24
Richmond S, Shaw WC, Stephens CD, Webb WG, Roberts CT, Andrews M. Orthodontics in the general dental service of England and Wales: A critical assessment of standards. Br Dent J 1993; 174(9): 315-29.
25
Richmond S, Ikonomou C, Williams B, et al. Orthodontic treatment standards in Greece. Hel Orthod Rev 2001; 4: 9-20.
26
Holman JK, Hans MG, Nelson S, Powers MP. An assessment of extraction versus nonextraction orthodontic treatment using the peer assessment rating (PAR) index. Angle Orthod 1998; 68(6): 527-34.
27
Jeremiah HG, Cousley RR, Newton T, et al. Treatment time and occlusal outcome of orthognathic therapy in the East of England region. J Orthod 2012; 39: 206-11.
28
Turbill EA, Richmond S, Wright JL. The time-factor in orthodontics: What influences the duration of treatments in National Health Service practices? Community Dent Oral Epidemiol 2001; 29(1): 62-72.
29
Taylor PJ, Kerr WJ, McColl JH. Factors associated with the standard and duration of orthodontic treatment. Br J Orthod 1996; 23(4): 335-41.
30
John W, Kerr S, Buchanan IB, McNair FI, McColl JH. Factors influencing the outcome and duration of removable appliance treatment. Eur J Orthod 1994; 16(3): 181-6.
31
González-Gil-de-Bernabé P, Bellot-Arcís C, Montiel-Company JM, Gandía-Franco JL. Evaluation of treatment outcomes in a 3 years post-graduate orthodontic program using the peer assessment rating (par). J Clin Exp Dent 2014; 6(4): e364-8.
32
Elshafee FI, Naser-ud-Din S, Khamis AH, Athanasiou AE. Occlusal outcome assessment of orthodontic treatments performed at an educational institute in Dubai. J Leban Dent Assoc 2017; 51: 60-70.