Study design
This diagnostic accuracy and agreement study followed a modification of the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) where each software package was considered as a rater [25]. IRB approval was obtained from the Faculty of Dentistry, Alexandria University (IRB: 00010556-IORG: 0008839) and informed consents sought from the subjects whose scans were used as a study material. Access to the original scans was limited to the principal investigator. All potentially identifiable patient information was removed from the scans. The minimal sample size was calculated based on previous studies that aimed to evaluate the reliability of newly developed software calculating 3D tooth movement [12, 26]. Based on the results, a sample size of 20 scans was deemed enough to conduct this agreement study [27], with minimum accepted reliability ρ0 = 0.6 and maximum expected reliability ρ1 = 0.9, k = 3, where k corresponds to the number of tested software packages. The statistical significance alpha was set at 0.01 to account for multiple comparisons and a statistical power, 1-β = 0.9. The minimum calculated sample size was 18, increased to 20 to account for defective scans.
Sample collection
The sample of this study consisted of full arch pretreatment maxillary and mandibular intraoral digital scans of actual adult patients undergoing CAT. All scans were randomly selected from the records of a single orthodontic office in Mumbai, India with more than 15 years of experience with CAT. A random number list of 20 was generated using Microsoft Excel from the total number of scans available in the office archive. The scanner used was a TRIOS 3-D intraoral scanner (3Shape, Copenhagen, Denmark). The scan data was then exported in STL format file extension and the files were imported into the three studied software and analyzed in the Department of Orthodontics, Alexandria University. The study group comprised scans of 20 patients with a Little’s irregularity index that ranged from 4-6 mm. All teeth in both arches were evaluated for 3D angular tooth movements except for third molars. The inclusion criteria for the scans were (1) Adult subjects treated with CAT who received treatment in both arches, (2) Scans had to be complete and of acceptable quality with a full complement of teeth except for the third permanent molars. Scans were excluded if (1) Treatment involved extraction of permanent teeth, (2) Teeth had surface anomalies or if (3) Scans had soft-tissue lesions covering the palate or the mucogingival junction (MGJ) of the mandibular arch.
All the scans that met the eligibility criteria were given an identification number. All digital scans were de-identified by an independent investigator, and imported into the 3 different tooth measuring software programs for the principal investigator to evaluate Fig. 1.
Procedure
Digital setup
Full arch maxillary and mandibular pretreatment scans (T1) were imported to OrthoAnalyzer software (3Shape Ortho System, Copenhagen, Denmark). Virtual digital setups were done by using virtual segmentation techniques. All tooth movements were visualized and quantified in all directions. Tip, torque and rotation measurements of this Digital Setup (DS) were tabulated for all teeth and used as reference for measuring accuracy of the 3 different software. The DS were exported as STL model files and termed (T2).
T1 and T2 models were imported as STL files to the tooth measuring software programs, for registration and 3D angular measurements. The three studied software packages were:
-
1.
Semiautomatic best fit registration software (S-BF): Geomagic (Geomagic U.S., Research Triangle Park, NC) using landmark based method followed by regional global surface registration [17].
-
2.
Interactive surface-based registration software (I-SB): OrthoAnalyzer (3Shape Ortho System, Copenhagen, Denmark) using surface 3-point method of registration [18].
-
3.
Automatic best fit registration software (A-BF): eModel 9.0 “Compare” - (Geodigm Corporation, Chanhassen, MN) using automatic surface to surface registration [19].
The following steps were conducted before measurements were made:
1. Registration 2. Coordinate system generation 3. Measurement of tooth movement
-
1.
Registration of the initial model and the digital setup using the three software packages Fig. 2
Semiautomatic best fit registration software: Landmark based registration was performed on stable rugae and mucogingival junction (MGJ) points, followed by global and fine regional best fit surface registration based on all points of the two models.
Interactive surface-based registration software: Registration was done using surface 3-point method which involved selection of the same landmarks on each of the corresponding models followed by painting an area of known stability to be used for surface-based registration.
Automatic best fit registration: Model trimming and segmentation of individual teeth of T2 was done. This was followed by global initial alignment based on three-points based on the mesial-buccal cusps of the first molars and the mesial-incisal point of the right central incisor. This initial registration was then refined by 30 iterations of a closest-point algorithm to achieve best fit of the occlusal surfaces. Finally, a best fit surface registration algorithm automatically superimposed individual teeth from the segmented T2 models on the corresponding teeth in the unsegmented T1 models.
-
2.
Coordinate system generation
After registration, a three-dimensional (3D) coordinate system along the 3 principal axes were generated for tooth movement measurements. According to the software used, either model (S-BF and I-SB softwares) or tooth (A-BF) global reference frames were generated. Model global reference frames are defined as a coordinate system of three mutually perpendicular, intersecting axes (x = anteroposterior, y = occluso-gingival, and z = mediolateral). The “x-axis” is defined as the intersection of sagittal and occlusal planes, the “y-axis” as the intersection of the sagittal and coronal planes and the “z-axis” as the intersection of the coronal and occlusal planes [28]. The 3 D planes of space are the occlusal plane (XZ), midsagittal plane (XY), and the coronal plane (YZ).
For S-BF, one global model reference frame with the three mutually perpendicular intersecting axes (X, Y, Z) and orthogonal planes was constructed to measure all tooth movements (Composite Model Coordinates). On the other hand, for I-SB, each tooth required the generation of its own spatial model reference frame to individually measure tooth movements (Repeated Model Coordinates). However, for A-BF, a local tooth reference frame that the software automatically generates, defining the principal local coordinate tooth axes was generated (Automated Tooth Coordinates).
-
3.
3D tooth movement measurements
After all digital models (T1 & T2) were oriented in the same coordinate system via registration, it was possible to evaluate how the tooth positions changed. Registration of the T2 model onto the T1 model resulted in a 3 × 3 rotation matrix that described tooth movement. The change in the angular movement of each tooth between (T1) and (T2) was measured in degrees. The definitions used were as described by Daskalogiannakis et al. [28].
-
A.
Tip: defined as rotation of a tooth around the labiolingual (x-axis) (when referring to an incisor), or around the buccolingual (z-axis) (when referring to a posterior tooth), thereby causing a change in its angulation.
-
B.
Torque: defined as rotation of a tooth around its mesiodistal axis (z-axis) (when referring to an incisor), or around the (x-axis) (when referring to a posterior tooth), thereby causing a change in its inclination.
-
C.
Rotation: defined as rotation of a tooth around its long axis; rotation in the x-z plane, around the y-axis.
The measured angular changes from DS were recorded in Excel (Microsoft Excel: 2016 Microsoft Corporation) for comparisons with similar measurements taken from the three studied software.
Intra and inter-examiner reliability
Initially, one researcher (SA) performed all registrations of pretreatment scans with their digital setups, reference landmarks and axes identification, modification of local coordinates, as well as all tooth movement measurements. Another calibrated investigator (NV) repeated the measurements on 5 randomly selected scan sets for inter-operator reliability. Four weeks later the first researcher (SA) repeated measurements on 5 randomly selected scans to test intra-operator reliability. All measures were pooled to give a summary estimate to calculate Intra Class Correlation Coefficients for intra-examiner and inter-examiner reliability.
Statistical analysis of the data
Statistical analysis was carried out using IBM SPSS software package version 20.0. (Armonk, NY: IBM Corp). Data from individual teeth were pooled to provide an overall estimate of the amount of tooth movement in each degree of freedom and summarized as mean and standard deviation. Two-way fixed-rater single-measure Intra Class Correlation Coefficient (ICC) of absolute agreement were calculated between the pooled amount of tooth movement in each degree of freedom measured by each software package and the amount of tooth movement from the digital setup (reference standard). Overall agreement between the three software packages were similarly calculated. Based on the 95% confidence interval of the ICC estimate, values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively [29]. Statistical significance of the obtained results was expressed at p ≤ 0.01 to account for multiple comparisons.