Multimodal latent semantic alignment for automated prostate tissue classification and retrieval

Lara, Juan S. (Universidad Nacional de Colombia, Bogotá, Colombia) ; Contreras O., Victor H. (Universidad Nacional de Colombia, Bogotá, Colombia) ; Otálora, Sebastián (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; González, Fabio A. (Universidad Nacional de Colombia, Bogotá, Colombia)

This paper presents an information fusion method for the automatic classification and retrieval of prostate histopathology whole-slide images (WSIs). The approach employs a weakly-supervised machine learning model that combines a bag-of-features representation, kernel methods, and deep learning. The primary purpose of the method is to incorporate text information during the model training to enrich the representation of the images. It automatically learns an alignment of the visual and textual space since each modality has different statistical properties. This alignment enriches the visual representation with complementary semantic information extracted from the text modality. The method was evaluated in both classification and retrieval tasks over a dataset of 235 prostate WSIs with their pathology report from the TCGA-PRAD dataset. The results show that the multimodal-enhanced model outperform unimodal models in both classification and retrieval. It outperforms state–of–the–art baselines by an improvement in WSI cancer detection of 4.74% achieving 77.01% in accuracy, and an improvement of 19.35% for the task of retrieving similar cases, obtaining 64.50% in mean average precision.


Note: Due to the COVID-19 outbreak, the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention venue in Lima was cancelled. The proceedings of the online conference are however published according to the original schedule.


Keywords:
Conference Type:
published full paper
Faculty:
Economie et Services
School:
HEG-VS
Institute:
Institut Informatique de gestion
Subject(s):
Informatique
Publisher:
Lima, Peru, 4-8 October 2020
Date:
2020-10
Lima, Peru
4-8 October 2020
Pagination:
Pp. 572-581
Published in:
Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention
Series Statement:
Lecture Notes in Computer Science, vol. 12265
DOI:
ISSN:
0302-9743
ISBN:
978-3-030-59721-4
Appears in Collection:

Note: The status of this file is: restricted


 Record created 2021-01-26, last modified 2021-01-29

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)