18.06.2018 14:45 Michael Rauchensteiner (TUM):
Learning a two layer neural network by fewest samples (Master Thesis presentation)MI 03.10.011 (Boltzmannstr. 3, 85748 Garching)

We consider compositions of weighted sums of ridge functions, which are closely related to two layer feed-forward neural networks. The goal is the recovery of the ridge directions under mild smoothness assumptions of the function and for quasi-orthogonal ridge directions. The reconstruction can be divided into two steps. First, the identification of a matrix space spanned by the symmetric tensor products of the ridge directions. This space is approximated by evaluating the Hessian of the function on random points of the sphere and applying a dimension reduction on the span of the Hessians. Secondly, the recovery of the ridge directions expressed as rank-1 matrices by solving a non-linear program on the intersection of the reduced matrix space with the unit Frobenius sphere. The primary focus of my presentation is the analysis of the involved matrix spaces. We give bounds on the concentration of the span of the Hessians, that hold with high probability. This concentration is essential and enables us to apply the dimension reduction. This is complemented by numerical results for two-layer neural networks with sigmoidal activation functions.