Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network
Progress in Orthodontics volume 22, Article number: 14 (2021)
The rapid development of artificial intelligence technologies for medical imaging has recently enabled automatic identification of anatomical landmarks on radiographs. The purpose of this study was to compare the results of an automatic cephalometric analysis using convolutional neural network with those obtained by a conventional cephalometric approach.
Material and methods
Cephalometric measurements of lateral cephalograms from 35 patients were obtained using an automatic program and a conventional program. Fifteen skeletal cephalometric measurements, nine dental cephalometric measurements, and two soft tissue cephalometric measurements obtained by the two methods were compared using paired t test and Bland-Altman plots.
A comparison between the measurements from the automatic and conventional cephalometric analyses in terms of the paired t test confirmed that the saddle angle, linear measurements of maxillary incisor to NA line, and mandibular incisor to NB line showed statistically significant differences. All measurements were within the limits of agreement based on the Bland-Altman plots. The widths of limits of agreement were wider in dental measurements than those in the skeletal measurements.
Automatic cephalometric analyses based on convolutional neural network may offer clinically acceptable diagnostic performance. Careful consideration and additional manual adjustment are needed for dental measurements regarding tooth structures for higher accuracy and better performance.
Cephalometric analysis is an essential diagnostic tool for the treatment planning and evaluation of orthodontic patients. Accurate identification of the anatomical landmarks on cephalograms is critical for a reliable cephalometric analysis . Lateral cephalometric radiographs have been employed as an essential tool in orthodontics. However, to analyze such radiographs, the important anatomical structures need to be identified by a landmark identification and manual tracing process. However, this analysis requires a skilled orthodontist, and the process is time-consuming.
In computer science, artificial intelligence (AI) refers to the study of systems that perform tasks that require human intelligence using different computerized algorithms [2, 3]. Machine learning is a method of data analysis that allows computer programs to automatically improve through cognitive content. It is a branch of technology that allows systems to learn from data, identify patterns, and make decisions with minimal human intervention . These programs make decisions by examining large amounts of input data and with known outputs, subsequently, drawing conclusions on the input data with unknown outputs based on the initial “training” process.
In recent years, the use of AI in medicine and healthcare for the diagnosis and treatment of patients has been a topic of significant interest . This has resulted in the application of AI and machine learning technologies to dental processes including the classification of temporomandibular joint osteoarthritis and osteoporosis, prediction of the debonding probability of computer-aided design/computer-aided manufacturing (CAD/CAM) crowns, automatic detection and classification of jaw lesions and periodontal bone loss, survival prediction of oral cancer patients, tooth labeling, detection and diagnosis of dental caries, and detection of osteoporosis [6,7,8,9,10,11,12,13]. Furthermore, programs have been developed to automatically digitize the anatomical structures on lateral cephalometric radiographs. With these programs, automatic cephalometric analysis including diagnostic and analytical imaging tasks can be performed by AI and machine learning technologies. However, to the best of our knowledge, few recent studies about AI performance of cephalometric analysis which is useful for clinicians are available. Previous studies about deep learning algorithm reported that AI accurately detected cephalometric landmarks [14, 15]. In order to further explore the application of these technologies to clinical orthodontics, the results of clinical performance of cephalometric analysis are necessary. The purpose of the present study was to evaluate the accuracy of automatic cephalometric analysis by comparing with that of conventional cephalometric measurements.
Material and methods
This retrospective study was approved by the Institutional Review Board of the Chonnam National University Dental Hospital, Gwangju, Korea (CNUDH-EXP-2019-023). The inclusion criteria were (1) a fully erupted permanent dentition, and (2) no broad prosthetic restorations such as metal crowns or bridges, on the molars. The exclusion criteria were (1) multiple missing tooth and broad prosthetic restorations such as metal crowns or bridges, on the molars and (2) history of orthodontic treatment or orthognathic surgery. Conventional lateral cephalograms of 35 orthodontic patients (20 men, 15 women; mean age = 23.8 years) were obtained using OrthoCeph® OC100 (Instrumentarium Imaging Co., Tuusula, Finland). The cephalograms were imported to the V-cephTM (version 8.0, Cybermed Inc., Seoul, Korea) for the conventional cephalometric analysis and to the CephXTM (ORCA Dental AI Inc., Herzliya, Israel) for the AI analysis (Fig. 1). Sixteen anatomical landmarks were chosen (Table 1), and 15 skeletal cephalometric measurements, 9 dental cephalometric measurements, and 2 soft tissue cephalometric measurements were obtained by an experienced single examiner with over 7 years of experience in orthodontic treatment.
The sample size calculation was performed according to the result of previous study of Hwang et al. . The effect size was calculated to 0.49. A statistical power of 80 percent and a type I error of 5 percent was assumed by the G*power program (version 22.214.171.124, Heinrich-Heine-University, Dusseldorf, Germany). The calculation indicated that 35 individuals were required in the study.
All data were revealed to be normally distributed. Paired t test was then performed to determine the differences between the AI and conventional programs. For the purpose of comparing the two measurements obtained from each two methods graphically, the differences between the two methods were plotted using Bland-Altman analysis . Shapiro-Wilk test and paired t test were conducted using SPSS software package (version 23.0; IBM, Armonk, NY) and Bland-Altman plots were made by MedCalc (Ostend, Belgium). Significance level was set of 5%. To assess the errors of each method, the process of acquiring measurements using the conventional program was repeated after 2 weeks, and the measurement errors were calculated using Dahlberg’s method . The range of error was 0.1 to 0.3 mm for the linear measurements and from 0.1 to 0.3° for the angular measurements. For inter-examiner reproducibility, the second examiner performed the process of acquiring measurements using the conventional program, and the measurements were compared with first examiner’s measurements using the intraclass correlation coefficient (ICC). The ICC values were found to be statistically insignificant showing a mean of 0.91 (ICC 0.88-0.94), indicated excellent reliability.
Table 2 summarizes the differences between the measurements obtained by the conventional and AI methods. Statistically significant differences were found in saddle angle, linear measurements of maxillary incisor to NA line, and mandibular incisor to NB line. The soft tissue measurements did not show any significant difference between the two methods.
All measurements were within the limits of agreement based on the Bland-Altman plots. The measurements that showed significance in the paired t test were within the limits of agreement (Figs. 2, 3, and 4). The widths of limits of agreement were wider in dental measurements than those in the skeletal measurements (Table 3).
In orthodontics, cephalometric analysis is commonly performed by computerized method, which includes manual identification of the landmarks on a monitor. The software automatically calculates the distances and angles which are necessary for the cephalometric analysis. Otherwise, direct tracing of the radiograph is transferred to a computer. These computerized cephalometric analyses may cause some errors, such as transferring and measurement errors, even though the manual landmark identification is performed by a clinician [18, 19]. Leonardi et al.  reported that the accuracy of a cephalometric analysis varies between 60 and 80% for a computerized analysis compared with the fully manual process, where the total errors should be no more than 0.59 mm in the x direction and 0.56 mm in the y direction to be considered acceptable. Recent studies showed that despite this, cephalometric analysis performed by computerized systems appear to be considered reliable [21,22,23]. However, the process of manually identifying cephalometric landmarks on cephalograms requires a lot of time and has possibility of errors regardless of the experience of the clinician. Since the first study on automatic identification of cephalometric landmarks by Levy-Mandel et al.  in 1986, several researchers have tried to automate landmark identification using knowledge-based techniques or image matching methods and learning systems. However, only a few clinical studies have been conducted on automatic landmark identification [25,26,27,28].
The program used in this study was Ceph-X. The program is based on the machine learning; automatic landmark localization algorithm is based on convolutional neural network. The program requires the confirmation of landmark position before calculating measurements. Full automation of all steps is challenging due to overlaying structures and inhomogeneous intensity values in the cephalometric radiographs. Thus, calculating measurements process may not be performed by AI. This study is conducted to provide a clear picture about the possibility of replacing the traditional cephalometric process with the digital one. The study focused mainly to evaluate its usability for cephalometric analysis and measurements using automated program.
A previous study reported that this system shows an accuracy of 96.6% when compared with manual cephalometric approaches, with an acceptable variation of less than approximately 0.5 mm and 1° . Our results showed that three measurements, including the saddle angle, linear measurements of maxillary incisor to NA line and mandibular incisor to NB line exhibit statistically significant differences between the conventional and AI methods. The landmark identification of tooth structures can be affected by the surrounding superimposing anatomical structures, and clinicians also make this error. Particularly, identifying the mandibular incisor is difficult because it is generally located below the maxillary incisor due to overjet and overbite. Moreover, the widths of limits of agreement in the Bland-Altman plots were wider in dental measurements than those in the skeletal measurements. AI may have lower accuracy of performance in detecting tooth structures. The soft tissue measurements did not show any significant difference between the conventional and AI methods.
Based on the Bland-Altman plots, the measurements are in sufficiently good agreement. In the plots, the measurements that showed significant differences in the paired t test were within the limit of agreement (Figs. 2, 3, and 4). The wide limits of agreement in the Bland-Altman plots were defined clinically. Although there were statistically significant differences in some measurements and wide limits of agreement in the Bland-Altman plots between the two methods, the cephalometric analysis can be performed faster with the AI technique. In the present study, no manual adjustment after automatic landmark digitization was performed in order to exclusively evaluate the AI performance. With some manual adjustment made to landmark identification, the AI technique for cephalometric analysis may provide good performance. Considering that AI technologies will continue to improve in terms of the accuracy of measurement analysis with additional data and increasing use, the accuracy of cephalometric analysis based on AI techniques applied to clinical orthodontics will only further improve. Previous study by Hwang et al.  using recently proposed deep-learning method has reported that the mean error in landmark detection between AI and human was 1.46 ± 2.97 mm. In the present study, the mean error in all cephalometric measurements between conventional method and AI was 0.6 ± 3.1 mm. Although the errors in landmark identification cannot be compared directly with cephalometric measurements, the error using AI may be acceptable in clinics. In the study, the time needed for automatic tracing was within 5 s. In the conventional method, the mean time for tracing was about 6 min. Correcting lines requires lots of time. Considering this, automatic cephalometric analyses could help clinicians with manual adjustment.
The limitation of this study is that the sample size is smaller than that employed in previous studies on AI and machine learning technologies [30, 31]. In addition, one kind of radiographic machine was used to take cephalometric radiographs in the present study. Since the software used in this study is a commercially available cephalometric analysis program, it is believed that the performance of the software may be same with the images taken by various radiographic machines.
With the limitation of this study, the results indicate that automatic cephalometric analyses based on convolutional neural network may offer clinically acceptable diagnostic performance. Careful consideration and additional manual adjustment are needed for dental measurements regarding tooth structures for higher accuracy and better performance.
Availability of data and materials
The data and materials obtained in this study belong to the authors, and are therefore available only upon request, after approval by the authors.
Computer-aided design/computer-aided manufacturing
Tng TT, Chan TC, Hagg U, Cooke MS. Validity of cephalometric landmarks. An experimental study on human skulls. Eur J Orthod. 1994;16(2):110–20. https://doi.org/10.1093/ejo/16.2.110.
Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, et al. Deep learning: a primer for radiologists. Radiographics. 2017;37(7):2113–31. https://doi.org/10.1148/rg.2017170077.
Russell S, Norvig P. Artificial intelligence: a modern approach. 3rd ed. Upper saddle river: Pearson; 2009.
Mitchell T. Machine learning. 1st ed. 2 Pennsylvania plaza: McGraw-Hill Education; 1997.
Liew C. The future of radiology augmented with artificial intelligence: a strategy for success. Eur J Radiol. 2018;102:152–6. https://doi.org/10.1016/j.ejrad.2018.03.019.
de Dumast P, Mirabel C, Cevidanes L, Ruellas A, Yatabe M, Ioshida M, et al. A web-based system for neural network based classification in temporomandibular joint osteoarthritis. Comput Med Imaging Graph. 2018;67:45–54. https://doi.org/10.1016/j.compmedimag.2018.04.009.
Yamaguchi S, Lee C, Karaer O, Ban S, Mine A, Imazato S. Predicting the debonding of CAD/CAM composite resin crowns with AI. J Dent Res. 2019;98(11):1234–8. https://doi.org/10.1177/0022034519867641.
Ariji Y, Yanashita Y, Kutsuna S, Muramatsu C, Fukuda M, Kise Y, et al. Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique. Oral Surg Oral Med Oral Pathol Oral Radiol. 2019;128(4):424–30. https://doi.org/10.1016/j.oooo.2019.05.014.
Krois J, Ekert T, Meinhold L, Golla T, Kharbot B, Wittemeier A, et al. Deep learning for the radiographic detection of periodontal bone loss. Sci Rep. 2019;9(1):8495. https://doi.org/10.1038/s41598-019-44839-3.
Kim DW, Lee S, Kwon S, Nam W, Cha IH, Kim HJ. Deep learning-based survival prediction of oral cancer patients. Sci Rep. 2019;9(1):6994. https://doi.org/10.1038/s41598-019-43372-7.
Chen H, Zhang K, Lyu P, Li H, Zhang L, Wu J, et al. A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films. Sci Rep. 2019;9(1):3840. https://doi.org/10.1038/s41598-019-40414-y.
Lee JH, Kim DH, Jeong SN, Choi SH. Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm. J Dent. 2018;77:106–11. https://doi.org/10.1016/j.jdent.2018.07.015.
Cruz AS, Lins HC, Medeiros RVA, Filho JMF, da Silva SG. Artificial intelligence on the identification of risk groups for osteoporosis, a general review. Biomed Eng Online. 2018;17(1):12. https://doi.org/10.1186/s12938-018-0436-1.
Park JH, Hwang HW, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: Part 1-Comparisons between the latest deep-learning methods YOLOV3 and SSD. Angle Orthod. 2019;89(6):903–9. https://doi.org/10.2319/022019-127.1.
Hwang HW, Park JH, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: Part 2-Might it be better than human? Angle Orthod. 2020;90(1):69–76. https://doi.org/10.2319/022019-129.1.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10.
Dahlberg G. Statistical methods for medical and biological students. London: George Allen & Unwin Ltd; 1940. p. 122–32.
Richardson A. A comparison of traditional and computerized methods of cephalometric analysis. Eur J Orthod. 1981;3(1):15–20. https://doi.org/10.1093/ejo/3.1.15.
Turner PJ, Weerakone S. An evaluation of the reproducibility of landmark identification using scanned cephalometric images. J Orthod. 2001;28(3):221–9. https://doi.org/10.1093/ortho/28.3.221.
Leonardi R, Giordano D, Maiorana F, Spampinato C. Automatic cephalometric analysis: a systematic review. Angle Orthod. 2008;78(1):145–51. https://doi.org/10.2319/120506-491.1.
Paixão MB, Sobral MC, Vogel CJ, Araujo TM. Comparative study between manual and digital cephalometric tracing using Dolphin Imaging software with lateral radiographs. Dent Press J Orthod. 2010;15(6):123–30. https://doi.org/10.1590/S2176-94512010000600016.
Erkan M, Gurel HG, Nur M, Demirel B. Reliability of four different computerized cephalometric analysis programs. Eur J Orthod. 2012;34(3):318–21. https://doi.org/10.1093/ejo/cjr008.
Tikku T, Khanna R, Maurya R, Srivastava K, Bhushan R. Comparative evaluation of cephalometric measurements of monitor-displayed images by Nemoceph software and its hard copy by manual tracing. J Oral Biol Craniofacial Res. 2014;4(1):35–41. https://doi.org/10.1016/j.jobcr.2013.11.002.
Lévy-Mandel AD, Venetsanopoulos AN, Tsotsos JK. Knowledge-based landmarking of cephalograms. Comput Biomed Res. 1986;19(3):282–309. https://doi.org/10.1016/0010-4809(86)90023-6.
Yan C, Venetsanopoulos A, Fillery E. An expert system for landmarking cephalograms. 6th Internation Workshop on Expert Systems & Their Applications, Agence de l’Informatique; 1987. p. 337–56.
Parthasarathy S, Nugent ST, Gregson PG, Fay DF. Automatic landmarking of cephalograms. Comput Biomed Res. 1989;22(3):248–69. https://doi.org/10.1016/0010-4809(89)90005-0.
Cohen JD, Dunbar K, McClelland JL. On the control of automatic processes: a parallel distributed processing account of the stroop effect. Psychological review. 1990;97(3):332–61. https://doi.org/10.1037/0033-295X.97.3.332.
Davis DN, Forsyth D. Knowledge-based cephalometric analysis: a comparison with clinicians using interactive computer methods. Computers and Biomedical research. 1994;27(3):210–28. https://doi.org/10.1006/cbmr.1994.1018.
Mosleh MA, Baba MS, Malek S, Almaktari RA. Ceph-X: development and evaluation of 2D cephalometric system. BMC Bioinformatics. 2016;17:S499.
Wang CW, Huang CT, Hsieh MC, Li CH, Chang SW, Li WC, et al. Evaluation and comparison of anatomical landmark detection methods for cephalometric x-ray images: a grand challenge. IEEE Trans Med Imaging. 2015;34(9):1890–900. https://doi.org/10.1109/TMI.2015.2412951.
Nishimoto S, Sotsuka Y, Kawai K, Ishise H, Kakibuchi M. Personal computer-based cephalometric landmark detection with deep learning using cephalograms on the internet. J Craniofac Surg. 2019;30(1):91–5. https://doi.org/10.1097/SCS.0000000000004901.
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (Ministry of Science and ICT) (No. 2020R1F1A1070617 and NRF-2017R1D1A1B03032132). All materials used here belonged to the authors and nothing was provided by third-parts or private companies; therefore, the authors have no conflict of interest related to the present work.
Ethics approval and consent to participate
All participants provided informed consents. The present study was approved by Chonnam National University Dental Hospital Institutional Review Board (CNUDH-EXP-2019-023).
Consent for publication
The authors declare that we have no competing interests in relation to the present work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jeon, S., Lee, K.C. Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network. Prog Orthod. 22, 14 (2021). https://doi.org/10.1186/s40510-021-00358-4
- Cephalometric analysis
- Artificial intelligence
- Machine learning
- Convolutional neural network