Reliability and validity of intraoral and extraoral scanners
© Jacob et al. 2015
Received: 12 August 2015
Accepted: 19 October 2015
Published: 27 October 2015
This study evaluated the reliability and validity of one extraoral [Ortho Insight 3D™ (Motionview Software, Hixson, TN/USA)] and two intraoral [ITero™ (Align Technologies, San Jose, CA/USA) and Lythos™ (Ormco Corp., Orange, CA/USA)] scanners.
Fifteen dry human mandibles were scanned twice with each of the scanners, and digital models were generated. Five measurements were made on the dry mandibles and on each of the generated models, including intermolar width, intercanine width, posterior arch length, premolar crown diameter, and canine height. Systematic and random errors were evaluated based on replicate analyses. Differences were assessed using paired Student’s t tests.
Replicate analyses showed statistically significant systematic errors for only one measure (intermolar width measured from Ortho Insight 3D scans). Measurements taken from all three scanners were highly reliable, with intraclass correlations ranging from .926 to .999. Method errors were all less than 0.25 mm (averaged ≈0.12 mm). Posterior arch length and canine height were significantly smaller when measured on the Ortho Insight 3D scans than when measured on the dry mandibles and significantly smaller than when measured from the ITero and Lythos models.
While all three scanners produced reliable measures, Ortho Insight 3D systematically underestimated arch length and canine height.
Plaster models that have been traditionally used in orthodontics for evaluating patients’ occlusal status have several limitations. They are subject to physical and chemical damage and they wear when repeatedly measured. Models can also distort over time due to variation of humidity and temperature [1, 2]. Plaster models are also costly, both in terms of the time required for the impressions, model fabrication, and model storage. To solve these problems, digital models were introduced in the late 1990s.
OrthoCAD™ was the first company to introduce digital models. They allowed orthodontists to store casts electronically, eliminate impressions, and minimize many of the limitations associated with plaster models . Since the introduction of the first digital models to the orthodontic community in 1999, the technology has improved and numerous in-office dental scanners have been introduced. By 2014, digital models were being used for diagnostic purposes in 55 % of Pacific orthodontic practices and 21 % of Northeast practices .
Digital models are produced by digitizing the oral structures, either directly or indirectly, with intra- or extraoral scanners, respectively. Three types of scanners are typically used: mechanical scanners with a touch-probe, laser scanners, or white light scanners [5, 6]. Because light scanners work without touching the areas of interest, they are preferred over the touch-probe scanners . There are four types of imaging technologies employed: triangulation, parallel confocal, accordion fringe interferometry, and 3D in motion video . To gather surface data points, the light (laser or white) is projected from the source onto an object and either reflected back to a sensor or to an absorbing source. While a high-resolution image is an important requisite for accurately measuring dimensions, the mathematical models and algorithms used to reconstruct the model and produce the 3D images also play an important role [8, 9].
Before a scanner can be accepted by the orthodontic community, it has to be shown that it provides a valid and reliable representation of the dentoalveolar structures. Digital models produced with extraoral scanners have been shown to be valid when compared to direct measurement on plaster models, with the differences between the approaches considered to be clinically acceptable [10–17]. Intraoral scanners have also been shown to produce valid and reliable digital models [18, 19], but differences exist. For example, Flugge and coworkers  found that scanning with the ITero is less accurate than scanning with the D250.
The purpose of this study was to compare the reliability and validity of one extraoral scanner (Ortho Insight 3D™) and two intraoral (ITero™ and Lythos™) scanners. The validity and reliability of the Lythos has not been established, and intra- and extraoral scanners have not previously been compared.
Sample and measurements
The sample was comprised of 15 dried human adult mandibles from the Texas A&M University Baylor College of Dentistry Department of Biomedical Science. The mandibles had to be in good condition, with all of the teeth (from second molar to second molar) present. For each mandible, three sets of digital models were produced using three different scanning protocols, including one extraoral [Ortho Insight 3D (Motionview Software, Hixson, TN)] and two intraoral [ITero (Align Technologies, San Jose, CA) and Lythos (Ormco Corp., Orange, CA)]. For the Ortho Insight 3D scans, the mandibles were placed in the scanner and secured with double-sided tape. The two intraoral scans were performed using the manufacturers’ suggested protocols. Each mandible was scanned twice, as least 1 week apart, with each of the three scanners and Standard Tessellation Language (STL) files were created.
The skewness and kurtosis statistics indicated normal distributions. Intraobserver systematic errors between the replicate scans were described as mean differences and compared statistically with paired t tests. Intraobserver random error was estimated using intraclass correlation coefficients (ICCs) and method errors (√(Σd2/2n) . In order to evaluate systematic differences between scanners, the replicate measurements of each scanner were averaged. The systematic differences between scanners were described with means. Differences between scanners and differences between scanners and the actual mandibles were assessed using paired t tests. All statistical procedures were performed using IBM SPSS™ software (version 22.0, SPSS, Armonk, NY) using a significance level of 0.05.
Intraobserver systematic errors (mm) between the first and second replicates for each of the three scanning protocols, along with significances (Sig)
Ortho Insight 3D
Intraobserver random errors between replicates estimated with method errors (ME) and interclass correlations (ICC)
Ortho Insight 3D
Systematic differences (mm) between the measures made directly on the dry mandibles and corresponding measures made on the 3D digital reconstructions for each of the three scanning protocols, with positive values indicating digital underestimation
Ortho Insight 3D
Each of the three scanners produced accurate representations, with no consistent pattern of systematic errors. One of the Ortho Insight 3D measures showed statistically significant differences (0.161 mm) between replicate measurements. Importantly, the systematic errors in the present study were close to errors previously reported (ranging from −0.10 to 0.25 mm) for similar measurements [18, 19]. Measurement differences less than 0.20 mm have been suggested to be clinically acceptable . If the individual has been adequately calibrated and maintains the same landmark definitions, systematic intraobserver differences should not be expected to occur.
All three scanners were also highly reliable, with ICCs ranging from 0.926 to 0.999. A previous study evaluating Ortho Insight 3D showed similar ICCs (95–96 %), which were higher than the ICCs associated with digital models generated with emodel system (GeoDigm, Chanhassen, Minn) and cone-beam computerized tomography . Based on virtual models generated from CT scans, replicates showed ICCs ranging between 0.913 and 0.999 . Considering that reliability coefficients above 0.75 have been considered to be excellent , the substantially higher ICCs obtained in the present study indicate excellent reproducibility.
Measurements taken from the mandibles scanned with ITero and Lythos compared closely to the same measurements taken directly from the dry mandibles. Most of measurements were comparable, with average differences ranging between .002 and .066 mm. While the Lythos scans have not been previously evaluated, ITero scans have been previously shown to be highly accurate . The results indicate that both scanners produce valid presentations of the mandible.
Two of the measurements taken from the Ortho Insight 3D scan reconstructions were slightly smaller than the corresponding measures taken on the dry mandibles. Comparing plaster models and emodels, Mullen and coworkers  also found significant differences, with arch lengths measured on plaster models being approximately 1.5 mm larger than arch lengths measure on emodels. Using an extraoral scanner (Optimet 3D scanner), Redlich and coworkers  also reported statistically significant differences in mandibular arch length measurements obtained from plaster and digital models. Another study comparing plaster models and emodels found significant differences in anterior mandibular arch perimeter (plaster model measurements were 0.40 mm larger), but no differences when perimeter included all of the teeth between the first mandibular molars . Schirmer and Wiltsire  attributed the differences between digital and actual models to the difficulty of measuring a 3D object in two dimensions, i.e., on a computer monitor.
The results of this study were probably limited by the use of dry mandibles. Because measurements on dry mandibles can be more easily standardized, they might be expected to be more reliable than the same measurements taken in vivo. This could explain why intraoral scanning with the ITero is less precise than extraoral scanning with ITero . It would also have been possible to reduce random error by marking the landmarks on the mandibles prior to scanning them, which was not done because we wanted more realistic error estimates.
Measurements made from digital models produced by Ortho Insight 3D, ITero, and Lythos were highly reliable;
While there are no systematic differences between measurements taken direction on dry mandibles and corresponding measures taken from ITero and/or Lythos scans, Ortho Insight 3D scans (<.36 mm) slightly underestimated Arch_length and Canine_height.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Sweeney WT, Taylor DF. Dimensional changes in dental stone and plaster. J Dent Res. 1950;29:749–55.View ArticlePubMedGoogle Scholar
- Phillips RW. Skinner’s science of dental materials. 7th ed. Philadelphia PA: WB Saunders Co; 1973. p. p682.Google Scholar
- Peluso MJ, Sd J, Levine SW, Lorei BJ. Digital models: an introduction. Semin Orthod. 2004;10:226–38.View ArticleGoogle Scholar
- Keim RG, Gottlieb EL, Vogels III DS, Vogels PB. 2014 JCO study of orthodontic diagnosis and treatment procedures, part1: results and trends. J Clin Orthod. 2014;48:607–30.PubMedGoogle Scholar
- Persson A, Andersson M, Oden A, Sandborgh-Englund G. A three-dimensional evaluation of a laser scanner and a touchprobe scanner. J Prosthet Dent. 2006;95:194–200.View ArticlePubMedGoogle Scholar
- Schaefer O, Kuepper H, Sigusch BW, Thompson GA, Hefti AF, Guentsch A. Three-dimensional fit of lithium disilicate partial crowns in vitro. J Dent. 2013;41:271–7.View ArticlePubMedGoogle Scholar
- Kravitz ND, Groth C, Jones PE, Graham JW, Redmond WR. Intraoral digital scanners. J Clin Orthod. 2014;48:337–47.PubMedGoogle Scholar
- Villarrubia JS. Algorithms for scanned probe microscope image simulation, surface reconstruction, and tip estimation. J Res Natl Inst Stand Technol. 1997;102:425–54.View ArticleGoogle Scholar
- Axelsson P. Processing of laser scanner data—algorithms and applications. J Photogrammetry Remote Sensing. 1999;54:138–47.View ArticleGoogle Scholar
- Stevens DR, Flores-Mir C, Nebbe B, Raboud DW, Heo G, Major PW. Validity, reliability, and reproducibility of plaster vs digital study models: comparison of peer assessment rating and Bolton analysis and their constituent measurements. Am J Orthod Dentofacial Orthop. 2006;129:794–803.View ArticlePubMedGoogle Scholar
- Leifert MF, Leifert MM, Efstratiadis SS, Cangialosi TJ. Comparison of space analysis evaluations with digital models and plaster dental casts. Am J Orthod Dentofacial Orthop 2009;136: 16.e1-4.Google Scholar
- Fleming PS, Marinho V, Johal A. Orthodontic measurements on digital study models compared with plaster models: a systematic review. Orthod Craniofac Res. 2011;14:1–16.View ArticlePubMedGoogle Scholar
- Sousa MV, Vasconcelos EC, Janson G, Garib D, Pinzan A. Accuracy and reproducibility of 3-dimensional digital model measurements. Am J Orthod Dentofacial Orthop. 2012;142:269–73.View ArticlePubMedGoogle Scholar
- Hayashi K, Uechi J, Murata M, Mizoguchi I. Comparison of maxillary canine retraction with sliding mechanics and a retraction spring: a three-dimensional analysis based on a midpalatal orthodontic implant. Eur J Orthod. 2004;26:585–9.View ArticlePubMedGoogle Scholar
- Kim J, Heo G, Lagravère MO. Accuracy of laser-scanned models compared to plaster models and cone-beam computed tomography. Angle Orthod. 2014;84:443–50.View ArticlePubMedGoogle Scholar
- Kim JH, Kim KB, Kim WC, Kim JH, Kim HY. Accuracy and precision of polyurethane dental arch models fabricated using a three-dimensional subtractive rapid prototyping method with an intraoral scanning technique. Korean J Orthod. 2014;44:69–76.PubMed CentralView ArticlePubMedGoogle Scholar
- Patzelt SB, Lamprinos SS, Att W. The time efficiency of intraoral scanners: an in vitro comparative study. J Am Dent Assoc. 2014;145:542–51.View ArticlePubMedGoogle Scholar
- Naidu D, Scott J, Ong D, Ho CT. Validity, reliability and reproducibility of three methods used to measure tooth widths for Bolton analyses. Aust Orthod J. 2009;25:97–103.PubMedGoogle Scholar
- Grünheid T, McCarthy SD, Larson BE. Clinical use of a direct chairside oral scanner: an assessment of accuracy, time, and patient acceptance. Am J Orthod Dentofacial Orthop. 2014;146:673–82.View ArticlePubMedGoogle Scholar
- Flügge TV, Schlager S, Nelson K, Nahles S, Metzger MC. Precision of intraoral digital dental impressions with iTero and extraoral digitization with the iTero and a model scanner. Am J Orthod Dentofacial Orthop. 2013;144:471–8.View ArticlePubMedGoogle Scholar
- Dahlberg G. Statistical methods for medical and biological students. New York: Interscience Publications; 1940.Google Scholar
- Schirmer UR, Wiltshire WA. Manual and computer-aided space analysis: a comparative study. Am J Orthod Dentofacial Orthop. 1997;112:676–80.View ArticlePubMedGoogle Scholar
- Akyalcin S, Dyer DJ, English JD, Sar C. Comparison of 3-dimensional dental models from different sources: Diagnostic accuracy and surface registration analysis. Am J Orthod Dentofac Orthop. 2013;144:831–7.View ArticleGoogle Scholar
- El-Zanaty HM, El-Beialy AR, Abou El-Ezz AM, Attia KH, El-Bialy AR, Mostafa YA. Three-dimensional dental measurements: an alternative to plaster models. Am J Orthod Dentofacial Orthop. 2010;137:259–65.View ArticlePubMedGoogle Scholar
- Fleiss JL. Statistical methods for rates and proportions. New York: Wiley; 1981.Google Scholar
- Zilberman O, Huggare JA, Parikakis KA. Evaluation of the validity of tooth size and arch width measurements using conventional and three-dimensional virtual orthodontic casts. Angle Orthod. 2003;73:301–6.PubMedGoogle Scholar
- Dalstra M, Melsen B. From alginate impressions to digital virtual models: accuracy and reproducibility. J Orthod. 2009;36:36–41.View ArticlePubMedGoogle Scholar
- Mullen SR, Martin CA, Ngan P, Gladwin M. Accuracy of space analysis with emodels and plaster models. Am J Orthod Dentofacial Orthop. 2007;132:346–52.View ArticlePubMedGoogle Scholar
- Redlich M, Weinstock T, Abed Y, Schneor R, Holdstein Y, Fischer A. A new system for scanning, measuring and analyzing dental casts based on a 3D holographic sensor. Orthod Craniofac Res. 2008;11:90–5.View ArticlePubMedGoogle Scholar