How to exploit weaknesses in biomedical challenge design and organization

Reinke, Annika (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Eisenmann, Matthias (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Onogur, Sinan (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Stankovic, Marko (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Scholz, Patrick (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Full, Peter M. (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Bogunovic, Hrvoje (University Vienna, Austria) ; Landman, Bennett A. (Vanderbilt University, Nashville, USA) ; Maier, Oskar (University of Lübeck, Germany) ; Menze, Bjoern (University of Munich, Germany) ; Sharp, Gregory C. (Massachusetts General Hospital, Boston, USA) ; Sirinukunwattana, Korsuk (University of Oxford, UK) ; Speidel, Stefanie (National Center for Tumor Diseases Dresden, Germany) ; Van der Sommen, Fons (Eindhoven University of Technology, The Netherlands) ; Zheng, Guoyan (University of Bern, Switzerland) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; Kozubek, Michal (Masaryk University, Brno, Czech Republic) ; Arbel, Tal (McGill University, Montreal, Canada) ; Bradley, Andrew P. (Queensland University of Technology, Brisbane, Australia) ; Jannin, Pierre (University of Rennes, France) ; Kopp-Schneider, Annette (German Cancer Research Center (DKFZ) Heidelberg, Germany) ; Maier-Hein, Lena (German Cancer Research Center (DKFZ) Heidelberg, Germany)

Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.


Faculty:
Economie et Services
School:
HEG-VS
Institute:
Institut Informatique de gestion
Subject(s):
Informatique
Publisher:
Cham, Springer
Date:
2018-09
Cham
Springer
Pagination:
Pp. 388-395
Published in:
Medical image computing and computer assisted intervention – MICCAI 2018 : 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV
Author of the book:
Frangi, Alejandro F. ; (ed.) ; University of Leeds, United Kingdom
DOI:
ISBN:
978-3-030-00936-6
External resources:
Appears in Collection:

Note: The status of this file is: restricted


 Record created 2018-11-14, last modified 2019-03-01

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)