| Résumé : |
(auteur) Embedded Computer vision, as many real application scenarios other areas of artificial intelligence, is facing hardware and power constraints with the rising of edge computing applications. For instance, the object detection problem, consisting in finding different objects of specific classes (types) in an image, turns out to be quite complicated to embed near the image sensor as two complex tasks are required: multi-scale localization and multi-class classification (i.e. identifying bounding boxes that perfectly enclose each object, whatever its size, and labeling the type of the detected object). Today these tasks are mainly often performed on general-purpose desktop machines. Nevertheless, attractive applications like autonomous-driving, augmented reality or video surveillance are urging the need for low-power, low-latency and compact low power devices.The state of the art has approached this challenge by optimizing specific sections of the complete processing-pipeline for a comparable object detection performance. A typical example in the last decade corresponds to minimizing the computing precision, hence the power, to a minimal value. Diminishing the bit-depth or image size has then been studied while implementing pre-processing steps that increase robustness against the loss in bit and image resolution. An algorithm that doesn’t require that kind of pre-processing stage to be programmable is obviously desirable in order to simplify its implementation (e.g. no memory access to learned weights). Another strategy has been to reduce power due to I/O communications amongst different chips or devices thanks to a more exhaustive integration of specialized circuitry and thanks to more efficient memory accesses and mathematical operations.In that context of near-sensor computing, this work points towards a more energy efficient detection pipeline. We target several specific key aspects:1. We try to assess if a dedicated-class-agnostic region proposal algorithm, based on pre-processed low-level features, could replace the typical sliding window approach for object localization in integrated smart imaging systems, allowing to target more efficiently objects in the image. Then, we propose a pipeline that takes into account near image sensor features extraction for Region Proposals with an embedded version of an algorithm called EdgeBoxes.2. We try to assess an optimal type of pre-processing (based on an efficient architecture) that would allow extracting low level features (oriented gradients), and give the best trade-off between power consumption, hardware complexity and object detection performance. Specifically, while being this architecture is fully compatible with region proposal algorithms beyond the sliding window.3. Finally, we try to assess if non-standard, or neuromorphic, image acquisition techniques can be exploited in order to further increase the detection efficiency in real case scenarios.Our methodology relies on behavioral simulations carried out thanks to a custom framework written in Python and C++ code. We propose a hierarchical model (and code architecture) of different image acquisition and processing techniques, and we study their performance through specific metrics related to runtime, memory usage, hardware complexity, I/O data-rate, localization performance and classification performance. We provide comparison with the state of the art and several benchmarks giving guidance to choose one or another architecture depending on the specific needs, and we conclude by stating which one would give, from our perspective, the best trade-offs. |
Ajouter le résultat dans votre panier
Visionner les documents numériques
Affiner la recherche Interroger des sources externesStudy of an integrated pre-processing architecture for smart-imaging-systems, in the context of lowpower computer vision and embedded object detection (2021)
Improvement of photogrammetric accuracy by modeling and correcting the thermal effect on camera calibration in ISPRS Journal of photogrammetry and remote sensing, vol 148 (February 2019)
Du photon au pixel (2015)
Analyse fonctionnelle du futur système de repositionnement pour les fermetures de l'expérience ATLAS (2013)
