Towards an understanding of CNNs: analysing the recovery of activation
pathways via Deep Convolutional Sparse Coding
Deep Convolutional Sparse Coding (D-CSC) is a framework reminiscent of deep
convolutional neural networks (DCNNs), but by omitting the learning of the
dictionaries one can more transparently analyse the role of the activation
function and its ability to recover activation paths through the layers.
Papyan, Romano, and Elad conducted an analysis of such an architecture,
demonstrated the relationship with DCNNs and proved conditions under which the
D-CSC is guaranteed to recover specific activation paths. A technical
innovation of their work highlights that one can view the efficacy of the ReLU
nonlinear activation function of a DCNN through a new variant of the tensor's
sparsity, referred to as stripe-sparsity. Using this they proved that
representations with an activation density proportional to the ambient
dimension of the data are recoverable. We extend their uniform guarantees to a
modified model and prove that with high probability the true activation is
typically possible to recover for a greater density of activations per layer.
Our extension follows from incorporating the prior work on one step
thresholding by Schnass and Vandergheynst.