ORIGINAL ARTICLE

Hippokratia 2014, 18(3):251-257

Papamanthos MK1,2, Varitimidis SE1, Dailiana ZH1, Kogia EI3, Malizos KN1
1Department of Orthopaedic Surgery and Musculoskeletal Trauma, University of Thessaly School of Medicine, Larissa, 2Department of Dentistry, Achillopouleion General Hospital of Volos, Volos, 3Department of Radiology, Achillopouleion General Hospital of Volos, Volos, Greece

Abstract

Background: To assess the diagnostic accuracy of mandibular cortical width (MCW) as an indicator of the presence of osteoporosis.
Materials and methods:  The study included 343 women between 45-75 years of age. After informed consent, all subjects underwent dual energy X-ray absorptiometry (DXA) of the hip and the lumbar spine (L1 to L4) in order to establish a gold standard diagnosis of osteoporosis and an orthopantomogram (OPG).
From the initial subjects, 28 (8.2%) did not meet the inclusion criteria. In the final sample 315 patients were included, of whom 293 were postmenopausal (93.3%) and 22 perimenopausal (6.7%).  Based on the DXA examination the sample was divided into three groups: a) normal controls (n=106), b) osteopenics (n=103) and c) osteoporotics (n=106). The MCW index was calculated by three different observers using the Emago image processing software. Inter - rater agreement was considered important since MCW was being tested as a possible screening tool for osteoporosis.
Results: Comparisons of MCW values between normal controls, osteopenic and osteoporotic women in our sample showed that these values clearly differentiate between the three groups, especially between normal and osteoporotic subjects. Furthermore, the receiver operator characteristic curve (ROC) analysis showed this to hold true in real diagnostic terms, giving a threshold value of 3.24 for differentiating between normal and osteoporotic women.
Conclusions: With ROC values ranging between 0.80 and 0.87 we can reasonably assume that the MCW index is a reliable indication of the presence of osteoporosis in a patient.

Read PDF

Key words:  osteoporosis, panoramic radiographs, mandibular cortical width, dual energy X-ray absorptiometry, radiomorphometric indices

Corresponding author: Sokratis  E. Varitimidis, MD, Associate Professor of Orthopaedics, Department of Orthopaedic Surgery and Musculoskeletal Trauma, University of Thessaly, Biopolis, 41110 Larissa, Greece, tel: +302413501199, +302413502722, fax: +302413501011, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Introduction

Successful treatment of osteoporosis is mainly based on early intervention. Early diagnosis is beneficial for the patient, minimizing the rate of fractures and ensuring a better quality of life, as well as for the public health care system, since the nursing cost is reduced. The development of a low cost screening program for osteoporosis is of major importance. Consequently, in recent decades, there have been several attempts to develop alternative screening tools to detect women at high risk of osteoporosis1.Each year a huge number of panoramic radiographs are performed worldwide for dental treatment needs2. This has provided an opportunity for research using radiomorphometric indices such as mandibular cortical width (MCW), alveolar bone resorption degree (M/M ratio), panoramic mandibular index (PMI), and mandibular cortical index (MCI) in order to identify such women3-7.

These indices usually show good sensitivity8,9 but low specificity and this can lead to high false positive rates and unnecessary referrals for dual energy X-ray absorptiometry (DXA) examinations10,11. Recently, Alman et al12, published their results on the use of fractal dimension (FD) as a good index of low bone mass density (BMD) in both women and men.

The present study focuses on the study of the MCW index in a sample of the Greek women. Our objective was to evaluate the diagnostic accuracy of MCW index when measured by using Emago image processing software, in order to diagnose osteoporosis.

Material and Methods

From March 2009 until April 2011, of the 954 adult female patients which attended the Dental Department of the “Achillopoulion” General Hospital in Volos, Greece, for routine or emergency dental care, 599 needed to undergo panoramic radiography. Of them, 343 volunteer subjects, after informed consent agreed to participate in the study but finally 315 of them were recruited while 28 (8.2%) did not meet the inclusion criteria. The age ranged from 45 to 75 years (mean ± SD: 59.64 ± 8.19).

The exclusion criteria were previous diagnosis of osteoporosis or any other metabolic bone disease, or medication received affecting bone metabolism.

Subjects were classified into two groups based on their menopausal status: 293 (93.3%) were postmenopausal and 22 (6.7%) were perimenopausal.

All subjects underwent DXA examination of the hip and the lumbar spine (L1 to L4) in order to establish a gold standard diagnosis of osteoporosis and provide a panoramic examination (OPG). DXA scans were performed by the same radiographer on the Lunar DPX Protm (GE-Healthcare, Wisconsin, USA).

Bone Mass Density (BMD) values were classified as normal (T-score ≥ -1.0), osteopenia (T- score between -1.0 and -2.5), and osteoporosis (T- score ≤ -2.5), according to World Health Organization (WHO) criteria.

DXA Hip and DXA Spine values are different in the Osteopenia and Osteoporosis groups whereas they are similar in the normal group (Table 1).  These differences were statistically significant (p=0.0000).  Osteoporosis was defined as a BMD T-score ≤ -2.5 at either the lumbar spine or the hip. Based on the DXA results the subjects were diagnosed as a) normal b) osteopenic and c) osteoporotic.

The radiographs were taken the same day of the DXA scan by using for all patients the same panoramic machine (Orthopantomograph OP100©, Instrumentarium, PaloDEx Group Oy, Finland), at 10 mA and 15s and the voltage varied between 60kV and 75kV depending on the patient. The panoramic images were saved in JPEG format and the pixel size was determined. A photostimulable phosphor plate system (ADC Solo©, Agfa, Morstel, Belgium) was used for image capture and read out.

A reference object was used in order to control the magnification of the OPG images. The subjects held a plastic bite block containing a 3.175 mm diameter ball bearing between their left premolar teeth, during the radiographic exposures.

In accordance with the European Commission Guidelines on Quality Standards for Panoramic Radiography, any OPG-images of poor quality were excluded from the study13.

The MCW index was assessed by two oral radiologists and one dental surgeon, acting as observers, blinded to the reference DXA results of the subjects. The OPGs were studied under the same conditions by using standard flat panel computer monitors (TFT). All observers assessed the width of the mandibular lower border cortex (MCW) below the mental foramina bilaterally using the Emago 5.2.5. software (Emago©, Oral Diagnostic Systems - ACTA / Oral Radiology, Amsterdam, Nederland). They were instructed to perform measurements using the method described in detail by Ledgerton et al14, and further used and described in OSTEODENT project11,15 as follows:  “A line was drawn from the midpoint of each foramen to the lower border of the mandible, at right angles to the tangent to the lower border at this point. The width of the cortical bone at the lower border was measured along this line from the inferior mandibular border to the inner edge of the cortex” (Figure 1). This way of MCW measurement is the one accepted worldwide so far, and for the first time introduced by Taguchi et al, in 199316 and reconfirmed by the same in later studies17.



Figure 1: Sample image used for mandibular cortical width (MCW) measurement.

Each observer independently measured MCW on both sides of the OPGs at the mental foramen region11. The bilateral measurements were averaged for each patient. All radiographic cortical width measurements were corrected for distortion errors by using width and height measurements of a ball bearing reference image.

In the present study, the classification of women into three categories as above was done using the mean MCW value from three different observers.

Statistical analysis used

The STATISTICA 8.0 for Windows (StatSoft Inc., Tulsa, OK, USA), was used for the statistical evaluation of the data.  In order to assess correctly the inter rater agreement the concordance correlation coefficient was used.

A comparison was made between the MCW values in the three groups in the study by means of the Multivariate ANOVA paradigm and MedCalc 11 for Windows. Receiver operator characteristic (ROC) curve analysis was used to measure the diagnostic accuracy of MCW for diagnosis of osteoporosis.  The areas under ROC curves (Az) were calculated using the Medcalc® software programme (MedCalc Software, Mariakerke, Belgium) as described in appendix 1.

Results

Based on the DXA results from the 315 subjects, 106 (33.6%) were classified as normal 103 (32.7%) as osteopenic and 106 (33.6%) as osteoporotic. The mean age of the study group was 59.64 and the standard deviation was 8.19. Mean radiographic magnification was 0.854 as calculated from the ball bearing diameter averaged over all observers. These factors were applied to the observers’ MCW measurements to provide corrected measurements for analysis.

The bilateral measurements of MCW were averaged for each patient (Table 2).

There were statistically significant differences (Wilks λ=0.64, F4=35.20, p=0.00) and the result was statistically significant for both left and right measurements (F2=68.56, p=0.00 and F2=75.75, p=0.00 respectively).  Finally, the differences were statistically significant between the three study groups (Least Significant Difference - LSD test) as can be seen in Figure 2.



Figure 2: Mean values and confidence limits for mandibular cortical width (MCW) measurements for the three groups.

It is apparent that the differentiation between normal and osteopenic subjects is not as clear as between normal and osteoporotic subjects and osteopenic and osteoporotic subjects.  MCW values are simply indications when it comes to differentiating between normality and osteopenia.  On the contrary, MCW values are a potent indicator of the existence of osteoporosis.  As can be seen in Table 1, normal subjects had the highest MCW value followed closely by osteopenic subjects with osteoporotic subjects having the lowest of the group with a difference of almost one unit (0.95 lower than normals and 0.61 lower than osteopenics).  The mean MCW distance between normal and osteopenic subjects was half as much (0.34).

The above observations were further investigated by means of an ROC analysis using the DXA categorization as a gold standard (Figures 3, 4, and 5).  The results verified the initial observations in the sense that MCW values satisfactorily differentiate between normal and osteopenic subjects and almost perfectly between normal and osteoporotic subjects (Figures 3,4, and 5) and the corresponding ROC analysis (Tables 3, 4 and 5) also confirms this.



Figure 3: Receiver operator characteristic (ROC) curve analysis, normal vs. osteopenics.



Figure 4: Receiver operator characteristic (ROC) curve analysis, osteopenics vs. osteoporotics.



Figure 5: Receiver operator characteristic (ROC) curve analysis, normal vs. osteoporotics.

The ROC analysis reinforced the findings of the ANOVA and established that the clearest differentiation is between normal and osteoporotic subjects, whereas the differentiation between normal and osteopenic subjects is not as strong and definitive.

Despite an inter-rater disagreement on measurements done by the three independent observers, the measurements of each observer alone did differentiate between normal and osteoporotic women. Based on the findings that osteopenics were closer to normals as compared to osteoporotics we can group them together and see if the 3.24 cut off point is functional in either right or left MCW measurements in all three observers (Tables 6 and 7).

In this respect we have the following findings:

Observer 1:

88% correct normal/osteopenic identification

48-50% correct osteoporotic identification

Observer 2:

87% correct normal/osteopenic identification

44-46% correct osteoporotic identification

Observer 3:

74-77% correct normal/osteopenic identification

74-84% correct osteoporotic identification

Discussion and Conclusions

With ROC values ranging between 0.800 and 0.868 and a completely symmetrical curve (sensitivity 80.19 and specificity 81.13) it can be safely ascertained that the MCW index is a reliable indication of the presence of osteoporosis.

In our study the rate of agreement was not satisfactory (0.42 to 0.63).  We believe the higher levels of agreement reported in other studies may be an artefact of the method employed to evaluate this agreement.

Certain previously published studies that correlated radiographic indices with osteopenia and osteoporosis yielded inconsistent results as to the usefulness of these indices in the diagnosis or the prognosis of osteoporosis.

Mohajery and Brooks18 measured the thickness of the cortex at the angle of the mandible, sinus floor and lamina dura of the tooth socket. They did not find any significant differences in any of the mandibular measurements between the normal and osteoporotic subjects.

In contrast, in a recent study Roberts et al, confirmed that there is a correlation between a reduction in the thickness of the mandibular cortex and age-related bone loss at the hip8.

In the Yasar and Akgunlu19 study, there was a statistically significant differentiation in the mandibular cortical index (MCI) between osteoporotic and healthy individuals.  However, there was no statistically significant difference between the two groups in regard to MCW or tooth loss.

Horner and Devlin20,21, reported the value of the MCW index to be significantly correlated to the jaw BMD whereas in a more recent study by Karayanni et al11, such a significant correlation is reported between MCW and BMD in the jaw and the femoral neck.

The Arifin et al22 study is the first attempt to compare manual and computer assisted measurements of the MCW index using digitized panoramic radiographs.  The authors report that the only essential difference between the manual and the computer assisted MCW measurements was the speed of the computerized method.  They also noted that even with the computerized method dentists had to define the mental foramen themselves.  It is, therefore, plausible that individual differences in experience influence the precision of mental foramen location and that these differences in experience are reflected in the low inter-rater agreement found in their study.

In contrast, the study by Lopez et al23, concludes that the use of a computer system to measure MCW, PMI and M/M is more reliable in terms of repeatability and reproducibility compared with manual measurement of these radiomorphometric indices. However, the number of patients in this study was too small for reliable conclusions to be drawn.

In the Alman et al12study, one of the few studies that included men and women, there were 241 subjects in total, of whom 56 were finally evaluated (the criterion was that the subject should have had a panoramic radiograph within six months of  their last DXA). Two main findings were reported:

The MCW index correlated higher with osteopenia in women than in men.

The threshold for establishing the degree of osteopenia (osteopenia vs. severe osteopenia) should take into account the sex and the age of the patient).

It should be noted that, in this study, there were no osteoporotic women included and, furthermore, the number of subjects was relatively small for safe conclusions to be drawn (56 six subjects, 39 men and 17 women), a point acknowledged by the authors themselves, who suggest that “future research should be conducted in larger populations with the ability to define thresholds by different gender and age groups”.

A study by Damilakis and Vlasiadis24is probably the first to compare three different radiomorphometric indices in identifying women (n=151) with a low BMD.  These were MCW, M/M ratio and PMI and the women were differentiated according to the WHO classification into normal, osteopenics and osteoporotics with DXA measurements from the spinal cord, the cervix and the femur bone.  Their conclusion was that the MCW was more effective than the other two indices but not entirely satisfactory since the ideal threshold value showed low sensitivity and specificity, suggesting that this particular index is less suitable for differentiating subjects into the three categories.

In relation to the ideal threshold for the use of MCW values as a reliable predictive index for low BMD there are two values reported in the literature. The first is 3mm11,25 and the second 4mm26,27.  Klemetti et al27 report the value of 4mm as “optimal but not sufficient by itself for the excellent classification” in classifying women into three groups, i.e. normal, osteopenics and osteoporotics.

Regarding inter - rater agreement, Taguchi et al28published an article on an extensive study carried out over the internet with 60 respondents using a simple 3 group classification rule (Klemetti et al) and reported what they consider satisfactory intra-observer agreement values (Kappa > 0.6) but again there was a totally unsatisfactory high sensitivity vs. low specificity combination which rendered the entire procedure pointless.  We used raw MCW values in order to eliminate these shortcomings.

In the present study, the concordance correlation coefficient was used to assess inter - rater agreement as it was considered to be better suited for such purposes compared to more general ones such as the Pearson correlation coefficient or even the improved index suggested by Devlin and Horner20.  This index is more appropriate because not only does it check for agreement between pairs of values but it also takes into account the deviation from the ideal 45° line. By the use of this index the common mistake that is made when using indices such as the Pearson is avoided.  For example, when using the Pearson correlation coefficient the pairs (2, 4), (4, 8), (6, 12), and (7, 15) give a perfect 1.0 whereas the regression line deviates strongly from the 45° position.  The concordance correlation coefficient29,30on the other hand gives a value of 0.36 for the exact set of pairs, a value that more accurately reflects the agreement between two raters.  McBride suggests the following categorization of the concordance correlation coefficient values: < 0.90 low, 0.90-0.95 moderate, 0.95-0.99 substantial, >0.99 almost perfect31.

The low level of agreement between raters points to an inherent problem in all evaluations based on digital technology, i.e. the final display, which is the PC monitor.  This low agreement level was most probably the result of different types of computer monitors and resolutions being used by each rater. It must be noted that the inter rater agreement would be higher had we used a different method for evaluating it, such as the Pearson.

We consider the choice of this particular index to be crucial to the validity of our study.  If MCW is to be used as a screening tool or even diagnostic tool, inter rater agreement must be of the highest order in order to make results comparable.

In a recent study, Kavitha et al32 have demonstrated how image processing can enhance the readability of panoramic radiographs so as to give high sensitivity and specificity values (90.9 and 83.8 respectively) in identifying women with low BMD. However, there are certain shortcomings with this method and the final classification was short of separating normal women from osteopenics, an area that our study also encountered problems with. It would be of interest if further investigations employing the Kavitha et al, algorithm and our suggestions in respect to monitor calibration and measurement standardization were carried out in the future how image processing can enhance the readability of panoramic radiographs so as to give high sensitivity and specificity values (90.9 and 83.8 respectively) in identifying women with low BMD.

We propose the development of standardized criteria for the evaluation of radiographs to obtain the MCW index that should include the following:

Medical grade monitors

Digital imaging and communications in medicine (DICOM) format as a more reliable alternative to joint photographic experts group (JPEG).

Similar conditions of observation in respect to ambient lighting, etc.

Individual monitor calibration at least once before use.

We believe that these criteria would greatly improve inter - rater agreement and, furthermore, they would certainly improve the validity of the MCW index as a potent alternative to DXA.

With ROC values ranging between 0.800 and 0.868 and a completely symmetrical curve (sensitivity 80.19 and specificity 81.13), it can be safely ascertained that the MCW index is a reliable indication of the presence of osteoporosis.

Expanding the findings of similar studies in a new population, Greek in this particular case, is considered important if the MCW index is to be used for routine screening for osteoporosis in subjects who underwent panoramic radiography for dental purposes.  Also, by including osteopenic individuals in the study we have demonstrated the necessity for the development of finer procedures so as to reliably differentiate osteopenics from normals, thus contributing to the early detection of possible osteoporosis development.  Finally, we have identified the lack of appropriate monitor calibration as a source of bias in the MCW measurements.  This finding will help improve procedures and enhance the diagnostic accuracy of the method.

Conclussion

In conclusion we believe that the MCW index cannot be an exclusive diagnostic tool of osteoporosis to replace DEXA. Just as a large number of women worldwide undergo panoramic radiography for dental purposes, the measurement of MCW index in these subjects can be a useful tool for early diagnosis of an osteoporotic lesion in progress. Then certainly further diagnostic tests will be required to assess the osteoporosis, predominantly a DEXA test.

Conflict of Interest Stat

The authors declare no conflict of interest or financial support.

References

1. White SC. Oral radiographic predictors of osteoporosis. Dentomaxillofacial Radiol. 2002; 31: 84-92.
2. Taguchi A. Triage screening for osteoporosis in dental clinics using panoramic radiographs. Oral Dis. 2010; 16: 316-327.
3. Hastar E, Yilmaz HH, Orhan H. Evaluation of mental index, mandibular cortical index and panoramic mandibular index on dental panoramic radiographs in the elderly. Eur J Dent. 2011; 5: 60-67.
4. Drozdzowska B, Pluskiewicz W, Tarnawska B. Panoramic-based mandibular indices in relation to mandibular bone mineral density and skeletal status assessed by dual energy X-ray absorptiometry and quantitative ultrasound. Dentomaxillofac Radiol. 2002; 31: 361-367.
5. Cakur B, Dagistan S, Sahin A, Harorli A, Yilmaz AB. Reliability of mandibular cortical index and mandibular bone mineral density in the detection of osteoporotic women.  Dentomaxillofac Radiol. 2009; 38: 255-261.
6. Leite AF, Figueiredo PT, Guia CM, Melo NS, de Paula AP. Correlations between seven panoramic radiomorphometric indices and bone mineral density in postmenopausal women. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2010; 109: 449-456.
7. Roberts M, Yuan J, Graham J, Jacobs R, Devlin H. Changes in mandibular cortical width measurements with age in men and women. Osteoporos Int. 2011; 22: 1915-1925.
8. Richy F, Gourlay M, Ross PD, Sen SS, Radican L, De Ceulaer F, et al. Validation and comparative evaluation of the osteoporosis self-assessment tool (OST) in a Caucasian population from Belgium. QJM. 2004; 97: 39-46.
9. Gourlay ML, Miller WC, Richy F, Garrett JM, Hanson LC, Reginster JY. Performance of osteoporosis risk assessment tools in postmenopausal women aged 45-64 years. Osteoporosis Int. 2005; 16: 921-927.
10. Pongchaiyakul C, Nguyen ND, Eisman JA, Nguyen TV. Clinical risk indices, prediction of osteoporosis, and prevention of fractures: diagnostic consequences and costs. Osteoporosis Int. 2005; 16: 1444-1450.
11. Karayianni K, Horner K, Mitsea A, Berkas L, Mastoris M, Jacobs R, et al. Accuracy in osteoporosis diagnosis of a combination of mandibular cortical width measurement on dental panoramic radiographs and a clinical risk index (OSIRIS): the OSTEODENT project. Bone. 2007; 40: 223-229.
12. Alman AC, Johnson LR, Calverley DC, Grunwald GK, Lezotte DC, Hokanson JE. Diagnostic capabilities of fractal dimension and mandibular cortical width to identify men and women with decreased bone mineral density. Osteoporos Int. 2012; 23: 1631-1636.
13. European Commission. Radiation Protection 136. European Guidelines on Radiation Protection in Dental Radiology. Office for Official Publications of the European Communities, Luxembourg, 2004, 59.
14. Ledgerton D, Horner K, Devlin H, Worthington H. Radiomorphometric indices of the mandible in a British population. Dentomaxillofacial Radiol. 1999; 28: 173-181.
15. Horner K, Allen P, Graham J, Jacobs R, Boonen S, Pavitt S, et al. The relationship between the OSTEODENT index and hip fracture risk assessment using FRAX. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2010; 110: 243-249.
16. Taguchi A, Tanimoto K, Suei Y, Wada T. Tooth loss and mandibular osteopenia. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 1995; 79: 127-132.
17. Taguchi A, Suei Y, Sanada M, Higashi Y, Ohtsuka M, Nakamoto T, et al. Detection of vascular disease risk in women by panoramic radiography. J Dent Res. 2003; 82: 838-843.
18. Mohajery M, Brooks SL. Oral radiographs in the detection of early signs of osteoporosis. Oral Surg Oral Med Oral Pathol. 1992; 73: 112-117.
19. Yaşar F, Akgünlü F. The differences in panoramic mandibular indices and fractal dimension between patients with and without spinal osteoporosis. Dentomaxillofac Radiol. 2006; 35: 1-9.
20. Devlin H, Horner K. Mandibular radiomorphometric indices in the diagnosis of reduced skeletal bone mineral density. Osteoporosis Int. 2002; 13: 373-378.
21. Horner K, Devlin H. The relationship between mandibular bone mineral density and panoramic radiographic measurements. J Dent. 1998; 26: 337-343.
22. Arifin AZ, Asano A, Taguchi A, Nakamoto T, Ohtsuka M, Tsuda M, et al. Computer-aided system for measuring the mandibular cortical width on dental panoramic radiographs in identifying postmenopausal women with low bone mineral density. Osteoporos Int. 2006; 17: 753-759.
23. López-López J, Alvarez-López JM, Jané-Salas E, Estrugo-Devesa A, Ayuso-Montero R, Velasco-Ortega E, et al. Computer-aided system for morphometric mandibular index computation. (Using dental panoramic radiographs). Med Oral Patol Oral Cir Bucal. 2012; 17: e624-e632.
24. Damilakis J, Vlasiadis K. Have panoramic indices the power to identify women with low BMD at the axial skeleton? Phys Med. 2011; 27: 39-43.
25. Horner K, Devlin H, Harvey L. Detecting patients with low skeletal bone mass. J Dent. 2002; 30: 171-175.
26. White SC, Taguchi A, Kao D, Wu S, Service SK, Yoon D, Suei Y, et al. Clinical and panoramic predictors of femur bone mineral density. Osteoporos Int. 2005; 16: 339-346.
27. Klemetti E, Kolmakov S, Kröger H. Pantomography in assessment of the osteoporosis risk group. Scand J Dent Res. 1994; 102: 68-72.
28. Taguchi A, Asano A, Ohtsuka M, Nakamoto T, Suei Y, Tsuda M, et al; OSPD International Collaborative Group. Observer performance in diagnosing osteoporosis by dental panoramic radiographs: results from the osteoporosis screening project in dentistry (OSPD). Bone. 2008; 43: 209-213.
29. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45: 255-268.
30. Lin LI. A note on the concordance correlation coefficient. Biometrics. 2000; 56: 324-325.
31. McBride GB. A proposal for strength-of-agreement criteria for Lin’s Concordance Correlation Coefficient. NIWA Client Report: HAM2005-062 on NIWA Project MOH05201, 2005: 6.
32. Kavitha MS, Asano A, Taguchi A, Kurita T, Sanada M. Diagnosis of osteoporosis from dental panoramic radiographs using the support vector machine method in a computer-aided system. BMC Med Imaging. 2012; 12:1.