Deep multimodal case–based retrieval for large histopathology datasets

Jimenez-del-Toro, Oscar (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; University of Geneva (UNIGE) Switzerland) ; Otálora, Sebastian (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; University of Geneva (UNIGE) Switzerland) ; Atzori, Manfredo (University of Geneva (UNIGE) Switzerland) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; University of Geneva (UNIGE) Switzerland)

The current gold standard for interpreting patient tissue samples is the visual inspection of whole–slide histopathology images (WSIs) by pathologists. They generate a pathology report describing the main findings relevant for diagnosis and treatment planning. Search-ing for similar cases through repositories for differential diagnosis is often not done due to a lack of efficient strategies for medical case–based re-trieval. A patch–based multimodal retrieval strategy that retrieves sim-ilar pathology cases from a large data set fusing both visual and text information is explained in this paper. By fine–tuning a deep convolu-tional neural network an automatic representation is obtained for the vi-sual content of weakly annotated WSIs (using only a global cancer score and no manual annotations). The pathology text report is embedded into a category vector of the pathology terms also in a non–supervised approach. A publicly available data set of 267 prostate adenocarcinoma cases with their WSIs and corresponding pathology reports was used to train and evaluate each modality of the retrieval method. A MAP (Mean Average Precision) of 0.54 was obtained with the multimodal method in a previously unseen test set. The proposed retrieval system can help in differential diagnosis of tissue samples and during the training of pathol-ogists, exploiting the large amount of pathology data already existing digital hospital repositories.


Faculté:
Economie et services
Ecole:
HEG VS HES-SO Valais-Wallis - Haute Ecole de Gestion & Tourisme
Institut:
Institut Informatique de gestion
Classification:
Informatique
Adresse bibliogr.:
Cham, Springer
Date:
Cham
Springer
2017
Pagination:
pp. 149-157
Titre du document hôte:
Patch-based techniques in medical imaging : third International Workshop, Patch-MI 2017, held in conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Proceedings
DOI:
ISBN:
978-3-319-67433-9
Ressource(s) externe(s):
Le document apparaît dans:

Note: The status of this file is: restricted


 Notice créée le 2017-11-11, modifiée le 2017-11-14

Fichiers:
Télécharger le document
PDF

Évaluer ce document:

Rate this document:
1
2
3
 
(Pas encore évalué)