Medical image retrieval : a multi-modal approach

Cao, Yu ( Department of Computer Science, The University of Massachusetts Lowell, Lowell, MA, USA.) ; Steffey, Shawn ( Department of Computer Science, The University of Massachusetts Lowell, Lowell, MA, USA.) ; He, Jianbiao ( School of Information Science and Engineering, Central South University) ; Xiao, Degui ( College of Computer Science and Electronic Engineering, Hunan University, Changsha, PR China. ) ; Tao, Cui ( School of Biomedical Informatics, The University of Texas, Health Science Center at Houston, Houston, TX, USA) ; Chen, Ping ( Department of Computer Science, University of Massachusetts Boston, Boston, MA, USA) ; Müller, Henning ( University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis))

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

Article Type:
Economie et Services
Institut Informatique de gestion
Published in:
Cancer informatics
Numeration (vol. no.):
2014, suppl. 3, pp. 125-136
Appears in Collection:

 Record created 2015-11-21, last modified 2020-10-27

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)