Skip to main content

A novel perimeter-based index for evaluating the similarity of structure contours

  • King Hang Wicger Wong (a1), Lucullus HT Leung (a1) and Dora LW Kwong (a2)

The authors proposed a novel perimeter-based index (PBI) that was capable of evaluating the accuracy in the appraisal of auto-segmentation software. A quantitative value, that is time saved in editing the auto-segmented contours, was used to compare the effectiveness of two other commonly used indices in this study.


The relationship between the proposed index and the amount of the contouring time that could be saved was studied. The performances of two other commonly used similarity indices, namely Dice similarity coefficient (DSC) and the modified normalised average Hausdorff distance (MNAHD), were also evaluated. Ten nasopharyngeal cases and ten prostate cases that were previously treated with intensity-modulated radiation therapy technique were recruited as the validation cases in this study. Three observers were invited to contour four structures (bladder, rectum, brain stem and parotid gland) on computed tomography images of the validation cases without any aids. The time taken for contouring was recorded as the manual contouring time. By using an atlas-based auto-segmentation software, three sets of contours were generated for each validation case with different library sizes to produce different degrees of similarity level. The values of the three similarity indices of the auto-segmented contours were calculated. The observers were asked to edit the auto-segmented contours and the editing time was recorded.

The correlation between the editing time and the similarity indices was studied. The amount of time saved was calculated by subtracting the editing time from the manual contouring time. The performances of PBI, DSC and MNAHD were evaluated using Pearson correlation coefficient and receiver operating curve (ROC) analysis.


The PBI showed a positive linear relationship with the amount of contouring time saved. Pearson correlation coefficient ranged from 0·73 to 0·86 for the four structures. The PBI had a stronger correlation than the DSC in bladder and parotid gland, while there was no significant difference between the two indices in rectum and brain stem. The MNAHD had an inferior correlation than the proposed index. For the ROC analysis, the cut-off values for the PBI were 0·549, 0·401 and 0·301 for the three levels of contouring time saved, namely 50, 25 and 0%, respectively. The accuracy of PBI was over 77% and the Youden index was >0·6 for all three levels.


The proposed index showed a stronger relationship to the amount of contouring time saved. It was a simple tool that could be used to evaluate the performance of different segmentation algorithms.

Corresponding author
Author for correspondence: King Hang Wicger Wong, Princess Margaret Hospital, Rm 603, 6/F, Block H, Kwai Chung, Hong Kong SAR. Tel: +852-29902778. E-mail:
Hide All
1. Harari, PM, Song, S, Tomé, WA. Emphasizing conformal avoidance versus target definition for IMRT planning in head-and-neck cancer. Int J Radiat Oncol Biol Phys 2010; 77: 950958.
2. Teguh, DN, Levendag, PC, Voet, PW et al. Clinical validation of atlas-based auto-segmentation of multiple target volumes and normal tissue (swallowing/mastication) structures in the head and neck. Int J Radiat Oncol Biol Phys 2011; 81: 950957.
3. Lin, A, Kubicek, G, Piper, JW, Nelson, AS, Dicker, AP, Valicenti, RK. Atlas-based segmentation in prostate IMRT: timesavings in the clinical workflow. Int J Radiat Oncol Biol Phys 2008; 72: S328S329.
4. La Macchia, M, Fellin, F, Amichetti, M et al. Systematic evaluation of three different commercial software solutions for automatic segmentation for adaptive therapy in head-and-neck, prostate and pleural cancer. Radiat Oncol 2012; 7: 160.
5. Young, AV, Wortham, A, Wernick, I, Evans, A., Ennis, RD. Atlas-based segmentation improves consistency and decreases time required for contouring postoperative endometrial cancer nodal volumes. Int J Radiat Oncol Biol Phys 2011; 79: 943947.
6. Yang, D, Zheng, J, Nofal, A, Deasy, J, El Naqa, I M. Techniques and software tool for 3D multimodality medical image segmentation. J Radiat Oncol Inf 2009; 1 (1): 121.
7. Velazquez, ER, Parmar, C, Jermoumi, M et al. Volumetric CT-based segmentation of NSCLC using 3D-slicer. Sci Rep 2013; 3: 3529.
8. Hwee, J, Louie, AV, Gaede, S et al. Technology assessment of automated atlas based segmentation in prostate bed contouring. Radiat Oncol 2011; 6: 110.
9. Huttenlocher, DP, Klanderman, GA, Rucklidge, WJ. Comparing images using the Hausdorff distance. IEEE Trans Pattern Anal Mach Intell 1993; 9: 850863.
10. Al-Mayah, A, Moseley, J et al. Biomechanical-based image registration for head and neck radiation treatment. Phys Med Biol 2010; 55: 64916500.
11. Zou, KH, Warfield, SK, Bharatha, A et al. Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol 2004; 11 (2): 178189.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Radiotherapy in Practice
  • ISSN: 1460-3969
  • EISSN: 1467-1131
  • URL: /core/journals/journal-of-radiotherapy-in-practice
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed