Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems

de Prado, Miguel (ETH Zürich, Zürich, Switzerland) ; Pazos Escudero, Nuria (School of Engineering – HE-Arc Ingénierie, HES-SO // University of Applied Sciences Western Switzerland) ; Benini, Luca (ETH Zürich, Zürich, Switzerland)

Deep Learning is increasingly being adopted by industry for computer vision applications running on embedded devices. While Convolutional Neural Networks' accuracy has achieved a mature and remarkable state, inference latency and throughput are a major concern especially when targeting low-cost and low-power embedded platforms. CNNs' inference latency may become a bottleneck for Deep Learning adoption by industry, as it is a crucial specification for many real-time processes. Furthermore, deployment of CNNs across heterogeneous platforms presents major compatibility issues due to vendor-specific technology and acceleration libraries.In this work, we present QS-DNN, a fully automatic search based on Reinforcement Learning which, combined with an inference engine optimizer, efficiently explores through the design space and empirically finds the optimal combinations of libraries and primitives to speed up the inference of CNNs on heterogeneous embedded devices. We show that, an optimized combination can achieve 45x speedup in inference latency on CPU compared to a dependency-free baseline and 2x on average on GPGPU compared to the best vendor library. Further, we demonstrate that, the quality of results and time "to-solution" is much better than with Random Search and achieves up to 15x better results for a short-time search.


Keywords:
Conference Type:
published full paper
Faculty:
Ingénierie et Architecture
School:
HE-Arc Ingénierie
Institute:
Aucun institut
Publisher:
Florence, Italy, 25-29 March 2019
Date:
2019-03
Florence, Italy
25-29 March 2019
Pagination:
pp. 1409-1414
Published in:
Proceedings of 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), 25-29 March 2019, Florence, Italy
DOI:
ISBN:
978-3-9819263-2-3
Appears in Collection:

Note: The status of this file is: restricted


 Record created 2021-04-21, last modified 2021-04-23

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)