Centre de documentation
scientifique

Foundations of deep convolutional models through kernel methods / Alberto Bietti (2019)  

Public Titre : Foundations of deep convolutional models through kernel methods Type de document : Thèse/HDR Auteurs : Alberto Bietti, Auteur ; Julien Mairal, Directeur de thèse Editeur : Grenoble : Université de Grenoble Année de publication : 2019 Importance : 194 p. Format : 21 x 30 cm Note générale : bibliographie Thèse pour obtenir le grade de Docteur de la Communauté Université Grenoble Alpes, Spécialité : Mathématiques Appliquées Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Intelligence artificielle [Termes IGN] apprentissage automatique [Termes IGN] apprentissage profond [Termes IGN] approche hiérarchique [Termes IGN] classification par réseau neuronal convolutif [Termes IGN] espace de Hilbert [Termes IGN] état de l'art [Termes IGN] invariance [Termes IGN] jeu de données [Termes IGN] méthode fondée sur le noyau [Termes IGN] optimisation (mathématiques) [Termes IGN] Perceptron multicouche Index. décimale : THESE Thèses et HDR Résumé : (auteur) The increased availability of large amounts of data, from images in social networks, speech waveforms from mobile devices, and large text corpuses, to genomic and medical data, has led to a surge of machine learning techniques. Such methods exploit statistical patterns in these large datasets for making accurate predictions on new data. In recent years, deep learning systems have emerged as a remarkably successful class of machine learning algorithms, which rely on gradient-based methods for training multi-layer models that process data in a hierarchical manner. These methods have been particularly successful in tasks where the data consists of natural signals such as images or audio; this includes visual recognition, object detection or segmentation, and speech recognition.For such tasks, deep learning methods often yield the best known empirical performance; yet, the high dimensionality of the data and large number of parameters of these models make them challenging to understand theoretically. Their success is often attributed in part to their ability to exploit useful structure in natural signals, such as local stationarity or invariance, for instance through choices of network architectures with convolution and pooling operations. However, such properties are still poorly understood from a theoretical standpoint, leading to a growing gap between the theory and practice of machine learning. This thesis is aimed towards bridging this gap, by studying spaces of functions which arise from given network architectures, with a focus on the convolutional case. Our study relies on kernel methods, by considering reproducing kernel Hilbert spaces (RKHSs) associated to certain kernels that are constructed hierarchically based on a given architecture. This allows us to precisely study smoothness, invariance, stability to deformations, and approximation properties of functions in the RKHS. These representation properties are also linked with optimization questions when training deep networks with gradient methods in some over-parameterized regimes where such kernels arise. They also suggest new practical regularization strategies for obtaining better generalization performance on small datasets, and state-of-the-art performance for adversarial robustness on image tasks. Note de contenu : 1- Introduction 2- Invariance, Stability to deformations, and complexity of deep convolutional representations 3- A kernel perspective on regularization and robustness of deep neural networks 4- Links with optimization: inductive bias of neural tangent kernels 5- Invariance and stability through regularization: a stochastic optimization algorithm for data augmentation 6- Conclusion and perspectives Numéro de notice : 25833 Affiliation des auteurs : non IGN Thématique : INFORMATIQUE/MATHEMATIQUE Nature : Thèse française Note de thèse : Thèse de Doctorat : Mathématiques Appliquées : Grenoble Alpes : 2019 nature-HAL : Thèse DOI : sans En ligne : https://hal.archives-ouvertes.fr/tel-02543073/ document Format de la ressource électronique : URL Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=95171