Consistency of scale equivariance in internal representations of CNNs

Andrearczyk, Vincent (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)) ; Graziani, Mara (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; Hopitaux Universitaires de Genève (HUG), Geneva, Switzerland) ; Müller, Henning (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; Hopitaux Universitaires de Genève (HUG), Geneva, Switzerland) ; Depeursinge, Adrien (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; Centre Hospitalier Universitaire Vaudois (CHUV), Lausanne, Switzerland)

Despite the approximate invariance to scale learned in deep Convolutional Neural Networks (CNNs) trained on natural images, intermediate layers have been shown to contain information of scale while the invariance is only obtained in the final layers. In this paper, we experimentally analyze how this scale information is encoded in the hidden layers. Linear regression of scale is used to (i) evaluate whether scale information can be encoded, at a given layer, by individual response maps or a combination of many of them is necessary; (ii) evaluate whether the encoding of scale is shared among classes. If we can find a direction representative of scale variations in the hidden space, is this consistent across the data manifold? Or is it rather encoded locally within class-specific neighborhoods? We observe that scale information is encoded as a combination of a few response maps (around 3%) and that the encoding is relatively consistent across classes, with some amount of class-specific encoding.


Conference Type:
published full paper
Faculty:
Economie et Services
School:
HEG-VS
Institute:
Institut Informatique de gestion
Subject(s):
Informatique
Publisher:
Virtual conference, 31 August - 3 September 2020
Date:
2020-08
Virtual conference
31 August - 3 September 2020
Pagination:
Pp. 53-60
Published in:
Proceedings of the Irish Machine Vision and Image Processing Conference (IMVIP) 2020
ISBN:
978-0-9934207-5-7
External resources:
Appears in Collection:



 Record created 2020-11-17, last modified 2021-02-05

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)