Deep multimodal classification of image types in biomedical journal figures

Adrearczyk, Vincent (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; University of Geneva, Switzerland)

This paper presents a robust method for the classification of medical image types in figures of the biomedical literature using the fusion of visual and textual information. A deep convolutional network is trained to discriminate among 31 classes including compound figures, diagnostic image types and generic illustrations, while another shallow convolutional network is used for the analysis of the captions paired with the images. Various fusion methods are analyzed as well as data augmentation approaches. The proposed system is validated on the ImageCLEF 2013 classification task, largely improving the currently best performance from 83.5% to 93.7% accuracy.


Faculté:
Economie et Services
Ecole:
HEG-VS
Institut:
Institut Informatique de gestion
Classification:
Informatique
Adresse bibliogr.:
Cham, Springer
Date:
2018-09
Cham
Springer
Pagination:
Pp. 3-14
Publié dans:
Experimental IR meets multilinguality, multimodality, and interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings
Auteur du livre :
Bellot, P. ; (ed.)
Trabelsi, C. ; (ed.)
Mothe, J ; (ed.)
DOI:
ISBN:
978-3-319-98931-0
Ressource(s) externe(s):
Le document apparaît dans:

Note: The status of this file is: restricted


 Notice créée le 2018-11-16, modifiée le 2019-11-28

Fichiers:
Télécharger le document
PDF

Évaluer ce document:

Rate this document:
1
2
3
 
(Pas encore évalué)