Medical image retrieval: : a multi-modal approach

Cao, Yu ( Department of Computer Science, The University of Massachusetts Lowell, Lowell, MA, USA.) ; Steffey, Shawn ( Department of Computer Science, The University of Massachusetts Lowell, Lowell, MA, USA.) ; He, Jianbiao ( School of Information Science and Engineering, Central South University) ; Xiao, Degui ( College of Computer Science and Electronic Engineering, Hunan University, Changsha, PR China. ) ; Tao, Cui ( School of Biomedical Informatics, The University of Texas, Health Science Center at Houston, Houston, TX, USA) ; Chen, Ping ( Department of Computer Science, University of Massachusetts Boston, Boston, MA, USA) ; Müller, Henning ( University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis))

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

Type d'article:
Economie et Services
HEG VS HES-SO Valais-Wallis - Haute Ecole de Gestion & Tourisme
Institut Informatique de gestion
Titre du document hôte:
Cancer informatics
Numérotation (vol. no.):
2014, suppl. 3, pp. 125-136
Le document apparaît dans:

 Notice créée le 2015-11-21, modifiée le 2018-02-15

Télécharger le document

Évaluer ce document:

Rate this document:
(Pas encore évalué)