An exploration of uncertainty information for segmentation quality assessment

Hoebel, Katharina (Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA, USA ; Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA, USA) ; Andrearczyk, Vincent (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; Beers, Andrew (Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA, USA) ; Patel, Jay (Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA, USA ; Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA, USA) ; Chang, Ken (Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA, USA ; Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA, USA) ; Depeursinge, Adrien (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; Centre Hospitalier Universitaire Vaudois (CHUV), Lausanne, Switzerland) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; University of Geneva, Switzerland) ; Kalpathy-Cramer. Jayashree (Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA, USA)

Including uncertainty information in the assessment of a segmentation of pathologic structures on medical images, offers the potential to increase trust into deep learning algorithms for the analysis of medical imaging. Here, we examine options to extract uncertainty information from deep learning segmentation models and the influence of the choice of cost functions on these uncertainty measures. To this end we train conventional UNets without dropout, deep UNet ensembles, and Monte-Carlo (MC) dropout UNets to segment lung nodules on low dose CT using either soft Dice or weighted categorical cross-entropy (wcc) as loss functions. We extract voxel-wise uncertainty information from UNet models based on softmax maximum probability and from deep ensembles and MC dropout UNets using mean voxel-wise entropy. Upon visual assessment, areas of high uncertainty are localized in the periphery of segmentations and are in good agreement with incorrectly labelled voxels. Furthermore, we evaluate how well uncertainty measures correlate with segmentation quality (Dice score). Mean uncertainty over the segmented region (Ulabelled) derived from conventional UNet models does not show a strong quantitative relationship with the Dice score (Spearman correlation coefficient of -0.45 for the soft Dice vs -0.64 for the wcc model respectively). By comparison, image-level uncertainty measures derived from soft Dice as well as wcc MC UNet and deep UNet ensemble models correlate well with the Dice score. In conclusion, using uncertainty information offers ways to assess segmentation quality fully automatically without access to ground truth. Models trained using weighted categorical cross-entropy offer more meaningful uncertainty information on a voxel-level.


Keywords:
Conference Type:
published full paper
Faculty:
Economie et Services
School:
HEG-VS
Institute:
Institut Informatique de gestion
Subject(s):
Informatique
Publisher:
Houston, USA, 15-20 February 2020
Date:
2020-02
Houston, USA
15-20 February 2020
Pagination:
10 p.
Published in:
Proceedings of Medical Imaging 2020: Image Processing
Series Statement:
Proceedings of SPIE, vol. 11313
DOI:
ISBN:
9781510633933
Appears in Collection:

Note: The status of this file is: restricted


 Record created 2020-11-17, last modified 2020-11-20

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)