# 2021 CominLabs PhD <div align="center"><table><tr><td><img src="https://i.imgur.com/FjbvarH.png" alt="Context" height=50px/></td><td><img src="https://i.imgur.com/YNJmkva.jpg" alt="Context" height=75px></td> </tr></table></div> <div align="center"><table><tr><td><img src="https://project.inria.fr/cominlabs/files/2020/12/banniere_labex_cominlabs.jpg" alt="Context" height=75px/></td></tr></table></div> ## * Towards Portable Performance of Embedded Vision Deep Learning * Keywords: `design automation`, `embedded systems`, `formal methods`, `image processing`, `machine learning`, `dataflow processing`, `low energy processing`, `sustainable AI`, `repurposable systems` ###### tags: `VAADER` `PhD Thesis` `CominLabs` ## Context and Objective <p align="justify"> Smart vision sensors embedding deep learning processing are appearing in a wide range of innovative applications, including autonomous cars, medical devices such as CT, MRI, mammography and X-ray machines, and industrial vision systems within Industry 4.0. As an example, 65% of the surveillance cameras shipped in 2025 will include AI hardware acceleration (https://bit.ly/3h1cjLw). However, deep learning algorithms are compute hungry and their processing at the edge raises several challenges. In particular, the fast evolution of AI algorithms and technologies makes full ASIC specialization unsustainable, systems becoming obsolete after only a few years of operation. Thus, a trade-off needs to be found between reconfigurability and energy efficiency [Fuchs19]. This thesis focuses on the design of reconfigurable and repurposable energy efficient embedded vision systems, with the aim of offering system designers more sustainable technologies for their embedded vision systems. By repurposable, we refer to systems which computing can change drastically during product lifetime. In this context, the thesis focuses on the design automation of smart vision sensors embedding heterogeneous reconfigurable or reprogrammable processing substrates (CPUs, GPUs, FPGA). Dataflow programming, as offered by the HoCL programming language (https://github.com/jserot/hocl)[Serot18], will be leveraged to expose application concurrency and methods based on dataflow programming will be studied to increase the energy efficiency of these smart sensors by automating the most impactful algorithm-to-architecture adaptations while keeping independence between Model of Computation (MoC) and Model of Architecture (MoA). In particular, MoAs will be proposed and evaluated to early predict the performance impact of design decisions: application representation, tasks and data mapping, scheduling, data formats, etc. The thesis is conducted in the context of the CominLabs International Chair of Pr. Shuvra S. Bhattacharyya on "Design Methodologies and Tools for Adaptive Machine Learning at the Network Edge" [1]. Stays at University of Maryland, College Park, USA, are planned during the PhD. </p> ## Workplan This thesis will be organized in three periods: * At the beginning of the thesis, the work will consist of specifying a set of vision applications and hardware architectures to be experimented within the context of smart vision sensors, as well as the minimum energy and temporal performances required. This work will be based on an existing collaboration with the company EDIXIA which will allow to determine realistic, experimentable and ambitious use cases. A bibliography will be carried out on the exploration of the design space, the architecture models and the tools for automatic generation of code based on tensor representations and data flow models. * In a second step, according to the spotted challenges, different analytical models of architecture (MoAs) will be proposed, experimented and confronted in order to reach different speed-fidelity compromises. Indeed, the objective is not so much to obtain an accurate model as to obtain a model with fidelity, i.e., one that allows design decisions to be compared pairwise [Pelcat17]. For this purpose, the libraries of data flow code of the IETR VAADER team (such as github.com/preesm/preesm-apps) will be exploited, as well as deep learning applications developed in the team. The targeted architectures will include multi-core, GPUs and embedded FPGAs * In a third step, the created MoAs will be experimented in a generative design process to evaluate its capacity to improve improve design automation and the quality/performance trade-off of the system. ## Skills * C/C++ and Java programming. Experience on deep learning and/or functional programming is a plus. * Embedded system programming . * Good english command. * Notions of optimization and statistics. ## Characteristics * Duration: 3 years * Start: October 2021 * Salary : 1582€ Net, 1968€ Gross ## Team and Location * Research team: VAADER team, IETR laboratory. The Vaader research team at IETR works on image and video processing and its efficient porting to embedded processing systems. The targeted areas are dataflow models of computation, machine vision, representation and compression of visual information, multi/many-core programming, and hardware-software co-design. Vaader puts great efforts in building open tools and methods for system design automation. * Address: INSA Rennes, 20 Avenue des Buttes de Coësmes CS 70839 35708 Rennes Cedex 7 France ## Supervisors * Shuvra Bhattacharyya (University of Maryland, College Park, USA) - [Shuvra-Shikhar.Bhattacharyya@insa-rennes.fr](Shuvra-Shikhar.Bhattacharyya@insa-rennes.fr) * Jean-François Nezan (IETR, Equipe VAADER, Rennes) - [jnezan@insa-rennes.fr](jnezan@insa-rennes.fr) * Maxime Pelcat (IETR, Equipe VAADER, Rennes) - [maxime.pelcat@insa-rennes.fr](maxime.pelcat@insa-rennes.fr) ## Applications Send resume and application letter to [mpelcat@insa-rennes.fr](mailto:mpelcat@insa-rennes.fr?subject=Candidature%2021PhDCominLabs), [jnezan@insa-rennes.fr](mailto:jnezan@insa-rennes.fr?subject=Candidature%2021PhDCominLabs) ## References [ChairAML] [https://project.inria.fr/chairaml/](https://project.inria.fr/chairaml/) [Fuchs19] Fuchs, A., & Wentzlaff, D. (2019). The Accelerator Wall: Limits of Chip Specialization, 2019 IEEE HPCA. [Pelcat17] Pelcat, M., Mercat, A., Desnos, K., Maggiani, L., Liu, Y., Heulot, J., ... & Bhattacharyya, S. S. (2017). Reproducible evaluation of system efficiency with a model of architecture: From theory to practice. IEEE TCAD, 37(10), 2050-2063. [Serot18] https://hal.inria.fr/hal-03038307/