Skip to main content

Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network

Abstract

Objective

The rapid development of artificial intelligence technologies for medical imaging has recently enabled automatic identification of anatomical landmarks on radiographs. The purpose of this study was to compare the results of an automatic cephalometric analysis using convolutional neural network with those obtained by a conventional cephalometric approach.

Material and methods

Cephalometric measurements of lateral cephalograms from 35 patients were obtained using an automatic program and a conventional program. Fifteen skeletal cephalometric measurements, nine dental cephalometric measurements, and two soft tissue cephalometric measurements obtained by the two methods were compared using paired t test and Bland-Altman plots.

Results

A comparison between the measurements from the automatic and conventional cephalometric analyses in terms of the paired t test confirmed that the saddle angle, linear measurements of maxillary incisor to NA line, and mandibular incisor to NB line showed statistically significant differences. All measurements were within the limits of agreement based on the Bland-Altman plots. The widths of limits of agreement were wider in dental measurements than those in the skeletal measurements.

Conclusions

Automatic cephalometric analyses based on convolutional neural network may offer clinically acceptable diagnostic performance. Careful consideration and additional manual adjustment are needed for dental measurements regarding tooth structures for higher accuracy and better performance.

Introduction

Cephalometric analysis is an essential diagnostic tool for the treatment planning and evaluation of orthodontic patients. Accurate identification of the anatomical landmarks on cephalograms is critical for a reliable cephalometric analysis [1]. Lateral cephalometric radiographs have been employed as an essential tool in orthodontics. However, to analyze such radiographs, the important anatomical structures need to be identified by a landmark identification and manual tracing process. However, this analysis requires a skilled orthodontist, and the process is time-consuming.

In computer science, artificial intelligence (AI) refers to the study of systems that perform tasks that require human intelligence using different computerized algorithms [2, 3]. Machine learning is a method of data analysis that allows computer programs to automatically improve through cognitive content. It is a branch of technology that allows systems to learn from data, identify patterns, and make decisions with minimal human intervention [4]. These programs make decisions by examining large amounts of input data and with known outputs, subsequently, drawing conclusions on the input data with unknown outputs based on the initial “training” process.

In recent years, the use of AI in medicine and healthcare for the diagnosis and treatment of patients has been a topic of significant interest [5]. This has resulted in the application of AI and machine learning technologies to dental processes including the classification of temporomandibular joint osteoarthritis and osteoporosis, prediction of the debonding probability of computer-aided design/computer-aided manufacturing (CAD/CAM) crowns, automatic detection and classification of jaw lesions and periodontal bone loss, survival prediction of oral cancer patients, tooth labeling, detection and diagnosis of dental caries, and detection of osteoporosis [6,7,8,9,10,11,12,13]. Furthermore, programs have been developed to automatically digitize the anatomical structures on lateral cephalometric radiographs. With these programs, automatic cephalometric analysis including diagnostic and analytical imaging tasks can be performed by AI and machine learning technologies. However, to the best of our knowledge, few recent studies about AI performance of cephalometric analysis which is useful for clinicians are available. Previous studies about deep learning algorithm reported that AI accurately detected cephalometric landmarks [14, 15]. In order to further explore the application of these technologies to clinical orthodontics, the results of clinical performance of cephalometric analysis are necessary. The purpose of the present study was to evaluate the accuracy of automatic cephalometric analysis by comparing with that of conventional cephalometric measurements.

Material and methods

This retrospective study was approved by the Institutional Review Board of the Chonnam National University Dental Hospital, Gwangju, Korea (CNUDH-EXP-2019-023). The inclusion criteria were (1) a fully erupted permanent dentition, and (2) no broad prosthetic restorations such as metal crowns or bridges, on the molars. The exclusion criteria were (1) multiple missing tooth and broad prosthetic restorations such as metal crowns or bridges, on the molars and (2) history of orthodontic treatment or orthognathic surgery. Conventional lateral cephalograms of 35 orthodontic patients (20 men, 15 women; mean age = 23.8 years) were obtained using OrthoCeph® OC100 (Instrumentarium Imaging Co., Tuusula, Finland). The cephalograms were imported to the V-cephTM (version 8.0, Cybermed Inc., Seoul, Korea) for the conventional cephalometric analysis and to the CephXTM (ORCA Dental AI Inc., Herzliya, Israel) for the AI analysis (Fig. 1). Sixteen anatomical landmarks were chosen (Table 1), and 15 skeletal cephalometric measurements, 9 dental cephalometric measurements, and 2 soft tissue cephalometric measurements were obtained by an experienced single examiner with over 7 years of experience in orthodontic treatment.

Fig. 1
figure1

Cephalometric analysis using conventional (a) and AI (b) methods

Table 1 Description of cephalometric landmarks used in this study

Statistical analysis

The sample size calculation was performed according to the result of previous study of Hwang et al. [15]. The effect size was calculated to 0.49. A statistical power of 80 percent and a type I error of 5 percent was assumed by the G*power program (version 3.1.9.2, Heinrich-Heine-University, Dusseldorf, Germany). The calculation indicated that 35 individuals were required in the study.

All data were revealed to be normally distributed. Paired t test was then performed to determine the differences between the AI and conventional programs. For the purpose of comparing the two measurements obtained from each two methods graphically, the differences between the two methods were plotted using Bland-Altman analysis [16]. Shapiro-Wilk test and paired t test were conducted using SPSS software package (version 23.0; IBM, Armonk, NY) and Bland-Altman plots were made by MedCalc (Ostend, Belgium). Significance level was set of 5%. To assess the errors of each method, the process of acquiring measurements using the conventional program was repeated after 2 weeks, and the measurement errors were calculated using Dahlberg’s method [17]. The range of error was 0.1 to 0.3 mm for the linear measurements and from 0.1 to 0.3° for the angular measurements. For inter-examiner reproducibility, the second examiner performed the process of acquiring measurements using the conventional program, and the measurements were compared with first examiner’s measurements using the intraclass correlation coefficient (ICC). The ICC values were found to be statistically insignificant showing a mean of 0.91 (ICC 0.88-0.94), indicated excellent reliability.

Results

Table 2 summarizes the differences between the measurements obtained by the conventional and AI methods. Statistically significant differences were found in saddle angle, linear measurements of maxillary incisor to NA line, and mandibular incisor to NB line. The soft tissue measurements did not show any significant difference between the two methods.

Table 2 Comparison of the cephalometric measurements between conventional and artificial intelligence methods

All measurements were within the limits of agreement based on the Bland-Altman plots. The measurements that showed significance in the paired t test were within the limits of agreement (Figs. 2, 3, and 4). The widths of limits of agreement were wider in dental measurements than those in the skeletal measurements (Table 3).

Fig. 2
figure2

Bland-Altman plots for the skeletal measurements in each conventional and AI methods. For each plot, the x-axis represents the mean of the compared measurements, and the y-axis represents the difference between the compared measurements. The blue line represents the bias, and the red-hashed lines represent the upper and lower limits of agreement

Fig. 3
figure3

Bland-Altman plots for the dental measurements in each conventional and AI methods

Fig. 4
figure4

Bland-Altman plots for the soft tissue measurements in each conventional and AI methods

Table 3 Bland-Altman analysis for the cephalometric measurements between conventional and artificial intelligence methods

Discussion

In orthodontics, cephalometric analysis is commonly performed by computerized method, which includes manual identification of the landmarks on a monitor. The software automatically calculates the distances and angles which are necessary for the cephalometric analysis. Otherwise, direct tracing of the radiograph is transferred to a computer. These computerized cephalometric analyses may cause some errors, such as transferring and measurement errors, even though the manual landmark identification is performed by a clinician [18, 19]. Leonardi et al. [20] reported that the accuracy of a cephalometric analysis varies between 60 and 80% for a computerized analysis compared with the fully manual process, where the total errors should be no more than 0.59 mm in the x direction and 0.56 mm in the y direction to be considered acceptable. Recent studies showed that despite this, cephalometric analysis performed by computerized systems appear to be considered reliable [21,22,23]. However, the process of manually identifying cephalometric landmarks on cephalograms requires a lot of time and has possibility of errors regardless of the experience of the clinician. Since the first study on automatic identification of cephalometric landmarks by Levy-Mandel et al. [24] in 1986, several researchers have tried to automate landmark identification using knowledge-based techniques or image matching methods and learning systems. However, only a few clinical studies have been conducted on automatic landmark identification [25,26,27,28].

The program used in this study was Ceph-X. The program is based on the machine learning; automatic landmark localization algorithm is based on convolutional neural network. The program requires the confirmation of landmark position before calculating measurements. Full automation of all steps is challenging due to overlaying structures and inhomogeneous intensity values in the cephalometric radiographs. Thus, calculating measurements process may not be performed by AI. This study is conducted to provide a clear picture about the possibility of replacing the traditional cephalometric process with the digital one. The study focused mainly to evaluate its usability for cephalometric analysis and measurements using automated program.

A previous study reported that this system shows an accuracy of 96.6% when compared with manual cephalometric approaches, with an acceptable variation of less than approximately 0.5 mm and 1° [29]. Our results showed that three measurements, including the saddle angle, linear measurements of maxillary incisor to NA line and mandibular incisor to NB line exhibit statistically significant differences between the conventional and AI methods. The landmark identification of tooth structures can be affected by the surrounding superimposing anatomical structures, and clinicians also make this error. Particularly, identifying the mandibular incisor is difficult because it is generally located below the maxillary incisor due to overjet and overbite. Moreover, the widths of limits of agreement in the Bland-Altman plots were wider in dental measurements than those in the skeletal measurements. AI may have lower accuracy of performance in detecting tooth structures. The soft tissue measurements did not show any significant difference between the conventional and AI methods.

Based on the Bland-Altman plots, the measurements are in sufficiently good agreement. In the plots, the measurements that showed significant differences in the paired t test were within the limit of agreement (Figs. 2, 3, and 4). The wide limits of agreement in the Bland-Altman plots were defined clinically. Although there were statistically significant differences in some measurements and wide limits of agreement in the Bland-Altman plots between the two methods, the cephalometric analysis can be performed faster with the AI technique. In the present study, no manual adjustment after automatic landmark digitization was performed in order to exclusively evaluate the AI performance. With some manual adjustment made to landmark identification, the AI technique for cephalometric analysis may provide good performance. Considering that AI technologies will continue to improve in terms of the accuracy of measurement analysis with additional data and increasing use, the accuracy of cephalometric analysis based on AI techniques applied to clinical orthodontics will only further improve. Previous study by Hwang et al. [15] using recently proposed deep-learning method has reported that the mean error in landmark detection between AI and human was 1.46 ± 2.97 mm. In the present study, the mean error in all cephalometric measurements between conventional method and AI was 0.6 ± 3.1 mm. Although the errors in landmark identification cannot be compared directly with cephalometric measurements, the error using AI may be acceptable in clinics. In the study, the time needed for automatic tracing was within 5 s. In the conventional method, the mean time for tracing was about 6 min. Correcting lines requires lots of time. Considering this, automatic cephalometric analyses could help clinicians with manual adjustment.

The limitation of this study is that the sample size is smaller than that employed in previous studies on AI and machine learning technologies [30, 31]. In addition, one kind of radiographic machine was used to take cephalometric radiographs in the present study. Since the software used in this study is a commercially available cephalometric analysis program, it is believed that the performance of the software may be same with the images taken by various radiographic machines.

Conclusion

With the limitation of this study, the results indicate that automatic cephalometric analyses based on convolutional neural network may offer clinically acceptable diagnostic performance. Careful consideration and additional manual adjustment are needed for dental measurements regarding tooth structures for higher accuracy and better performance.

Availability of data and materials

The data and materials obtained in this study belong to the authors, and are therefore available only upon request, after approval by the authors.

Abbreviations

AI:

Artificial intelligence

CAD/CAM:

Computer-aided design/computer-aided manufacturing

References

  1. 1.

    Tng TT, Chan TC, Hagg U, Cooke MS. Validity of cephalometric landmarks. An experimental study on human skulls. Eur J Orthod. 1994;16(2):110–20. https://doi.org/10.1093/ejo/16.2.110.

    Article  PubMed  Google Scholar 

  2. 2.

    Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, et al. Deep learning: a primer for radiologists. Radiographics. 2017;37(7):2113–31. https://doi.org/10.1148/rg.2017170077.

    Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Russell S, Norvig P. Artificial intelligence: a modern approach. 3rd ed. Upper saddle river: Pearson; 2009.

    Google Scholar 

  4. 4.

    Mitchell T. Machine learning. 1st ed. 2 Pennsylvania plaza: McGraw-Hill Education; 1997.

    Google Scholar 

  5. 5.

    Liew C. The future of radiology augmented with artificial intelligence: a strategy for success. Eur J Radiol. 2018;102:152–6. https://doi.org/10.1016/j.ejrad.2018.03.019.

    Article  PubMed  Google Scholar 

  6. 6.

    de Dumast P, Mirabel C, Cevidanes L, Ruellas A, Yatabe M, Ioshida M, et al. A web-based system for neural network based classification in temporomandibular joint osteoarthritis. Comput Med Imaging Graph. 2018;67:45–54. https://doi.org/10.1016/j.compmedimag.2018.04.009.

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Yamaguchi S, Lee C, Karaer O, Ban S, Mine A, Imazato S. Predicting the debonding of CAD/CAM composite resin crowns with AI. J Dent Res. 2019;98(11):1234–8. https://doi.org/10.1177/0022034519867641.

    Article  PubMed  Google Scholar 

  8. 8.

    Ariji Y, Yanashita Y, Kutsuna S, Muramatsu C, Fukuda M, Kise Y, et al. Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique. Oral Surg Oral Med Oral Pathol Oral Radiol. 2019;128(4):424–30. https://doi.org/10.1016/j.oooo.2019.05.014.

    Article  PubMed  Google Scholar 

  9. 9.

    Krois J, Ekert T, Meinhold L, Golla T, Kharbot B, Wittemeier A, et al. Deep learning for the radiographic detection of periodontal bone loss. Sci Rep. 2019;9(1):8495. https://doi.org/10.1038/s41598-019-44839-3.

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Kim DW, Lee S, Kwon S, Nam W, Cha IH, Kim HJ. Deep learning-based survival prediction of oral cancer patients. Sci Rep. 2019;9(1):6994. https://doi.org/10.1038/s41598-019-43372-7.

    Article  PubMed  PubMed Central  Google Scholar 

  11. 11.

    Chen H, Zhang K, Lyu P, Li H, Zhang L, Wu J, et al. A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films. Sci Rep. 2019;9(1):3840. https://doi.org/10.1038/s41598-019-40414-y.

    Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Lee JH, Kim DH, Jeong SN, Choi SH. Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm. J Dent. 2018;77:106–11. https://doi.org/10.1016/j.jdent.2018.07.015.

    Article  PubMed  Google Scholar 

  13. 13.

    Cruz AS, Lins HC, Medeiros RVA, Filho JMF, da Silva SG. Artificial intelligence on the identification of risk groups for osteoporosis, a general review. Biomed Eng Online. 2018;17(1):12. https://doi.org/10.1186/s12938-018-0436-1.

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Park JH, Hwang HW, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: Part 1-Comparisons between the latest deep-learning methods YOLOV3 and SSD. Angle Orthod. 2019;89(6):903–9. https://doi.org/10.2319/022019-127.1.

    Article  PubMed  Google Scholar 

  15. 15.

    Hwang HW, Park JH, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: Part 2-Might it be better than human? Angle Orthod. 2020;90(1):69–76. https://doi.org/10.2319/022019-129.1.

    Article  PubMed  Google Scholar 

  16. 16.

    Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10.

    Article  Google Scholar 

  17. 17.

    Dahlberg G. Statistical methods for medical and biological students. London: George Allen & Unwin Ltd; 1940. p. 122–32.

    Google Scholar 

  18. 18.

    Richardson A. A comparison of traditional and computerized methods of cephalometric analysis. Eur J Orthod. 1981;3(1):15–20. https://doi.org/10.1093/ejo/3.1.15.

    Article  PubMed  Google Scholar 

  19. 19.

    Turner PJ, Weerakone S. An evaluation of the reproducibility of landmark identification using scanned cephalometric images. J Orthod. 2001;28(3):221–9. https://doi.org/10.1093/ortho/28.3.221.

    Article  PubMed  Google Scholar 

  20. 20.

    Leonardi R, Giordano D, Maiorana F, Spampinato C. Automatic cephalometric analysis: a systematic review. Angle Orthod. 2008;78(1):145–51. https://doi.org/10.2319/120506-491.1.

    Article  PubMed  Google Scholar 

  21. 21.

    Paixão MB, Sobral MC, Vogel CJ, Araujo TM. Comparative study between manual and digital cephalometric tracing using Dolphin Imaging software with lateral radiographs. Dent Press J Orthod. 2010;15(6):123–30. https://doi.org/10.1590/S2176-94512010000600016.

    Article  Google Scholar 

  22. 22.

    Erkan M, Gurel HG, Nur M, Demirel B. Reliability of four different computerized cephalometric analysis programs. Eur J Orthod. 2012;34(3):318–21. https://doi.org/10.1093/ejo/cjr008.

    Article  PubMed  Google Scholar 

  23. 23.

    Tikku T, Khanna R, Maurya R, Srivastava K, Bhushan R. Comparative evaluation of cephalometric measurements of monitor-displayed images by Nemoceph software and its hard copy by manual tracing. J Oral Biol Craniofacial Res. 2014;4(1):35–41. https://doi.org/10.1016/j.jobcr.2013.11.002.

    Article  Google Scholar 

  24. 24.

    Lévy-Mandel AD, Venetsanopoulos AN, Tsotsos JK. Knowledge-based landmarking of cephalograms. Comput Biomed Res. 1986;19(3):282–309. https://doi.org/10.1016/0010-4809(86)90023-6.

    Article  PubMed  Google Scholar 

  25. 25.

    Yan C, Venetsanopoulos A, Fillery E. An expert system for landmarking cephalograms. 6th Internation Workshop on Expert Systems & Their Applications, Agence de l’Informatique; 1987. p. 337–56.

    Google Scholar 

  26. 26.

    Parthasarathy S, Nugent ST, Gregson PG, Fay DF. Automatic landmarking of cephalograms. Comput Biomed Res. 1989;22(3):248–69. https://doi.org/10.1016/0010-4809(89)90005-0.

    Article  PubMed  Google Scholar 

  27. 27.

    Cohen JD, Dunbar K, McClelland JL. On the control of automatic processes: a parallel distributed processing account of the stroop effect. Psychological review. 1990;97(3):332–61. https://doi.org/10.1037/0033-295X.97.3.332.

    Article  PubMed  Google Scholar 

  28. 28.

    Davis DN, Forsyth D. Knowledge-based cephalometric analysis: a comparison with clinicians using interactive computer methods. Computers and Biomedical research. 1994;27(3):210–28. https://doi.org/10.1006/cbmr.1994.1018.

    Article  PubMed  Google Scholar 

  29. 29.

    Mosleh MA, Baba MS, Malek S, Almaktari RA. Ceph-X: development and evaluation of 2D cephalometric system. BMC Bioinformatics. 2016;17:S499.

    Article  Google Scholar 

  30. 30.

    Wang CW, Huang CT, Hsieh MC, Li CH, Chang SW, Li WC, et al. Evaluation and comparison of anatomical landmark detection methods for cephalometric x-ray images: a grand challenge. IEEE Trans Med Imaging. 2015;34(9):1890–900. https://doi.org/10.1109/TMI.2015.2412951.

    Article  PubMed  Google Scholar 

  31. 31.

    Nishimoto S, Sotsuka Y, Kawai K, Ishise H, Kakibuchi M. Personal computer-based cephalometric landmark detection with deep learning using cephalograms on the internet. J Craniofac Surg. 2019;30(1):91–5. https://doi.org/10.1097/SCS.0000000000004901.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (Ministry of Science and ICT) (No. 2020R1F1A1070617 and NRF-2017R1D1A1B03032132). All materials used here belonged to the authors and nothing was provided by third-parts or private companies; therefore, the authors have no conflict of interest related to the present work.

Author information

Affiliations

Authors

Contributions

SJ acquired all data and KCL analyzed and interpreted the data. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Kyungmin Clara Lee.

Ethics declarations

Ethics approval and consent to participate

All participants provided informed consents. The present study was approved by Chonnam National University Dental Hospital Institutional Review Board (CNUDH-EXP-2019-023).

Consent for publication

Not applicable.

Competing interests

The authors declare that we have no competing interests in relation to the present work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jeon, S., Lee, K.C. Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network. Prog Orthod. 22, 14 (2021). https://doi.org/10.1186/s40510-021-00358-4

Download citation

Keywords

  • Cephalometric analysis
  • Artificial intelligence
  • Machine learning
  • Convolutional neural network