Using crowdsourcing for multi-label biomedical compound figure annotation

García Seco de Herrera, Alba (Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, USA) ; Schaer, Roger (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; Antani, Sameer (Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, USA) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis))

Abstract. Information analysis or retrieval for images in the biomedical literature needs to deal with a large amount of compound figures (figures containing several subfigures), as they constitute probably more than half of all images in repositories such as PubMed Central, which was the data set used for the task. The ImageCLEFmed benchmark proposed among other tasks in 2015 and 2016 a multi–label classification task, which aims at evaluating the automatic classification of figures into 30 image types. This task was based on compound figures and thus the figures were distributed to participants as compound figures but also in a sep-arated form. Therefore, the generation of a gold standard was required, so that algorithms of participants can be evaluated and compared. This work presents the process carried out to generate the multi–labels of ∼ 2650 compound figures using a crowdsourcing approach. Automatic algorithms to separate compound figures into subfigures were used and the results were then validated or corrected via crowdsourcing. The im-age types (MR, CT, X–ray, ...) were also annotated by crowdsourcing including detailed quality control. Quality control is necessary to insure quality of the annotated data as much as possible. ∼ 625 hours were invested with a cost of ∼ 870$.


Mots-clés:
Type de conférence:
full paper
Faculté:
Economie et Services
Ecole:
HEG VS HES-SO Valais-Wallis - Haute Ecole de Gestion & Tourisme
Institut:
Institut Informatique de gestion
Classification:
Economie/gestion
Adresse bibliogr.:
Athens, Greece, 17-21 October 2016
Date:
Athens, Greece
17-21 October 2016
2016
Pagination:
10 p.
Publié dans
Proceedings of the 19th International Conference on medical image computing & Computer Assisted Intervention (MICCAI 2016)
Le document apparaît dans:

Note: The status of this file is: restricted


 Notice créée le 2017-10-26, modifiée le 2018-08-31

Fichiers:
Télécharger le document
PDF

Évaluer ce document:

Rate this document:
1
2
3
 
(Pas encore évalué)