"Noise Transforms Feed-Forward Networks into Sparse Coding Networks" looks like a cool #ICLR2023 submission! Gaussian noise added to the inputs of a ReLU MLP causes convergence to a sparse coding network with Gabor and center/surround receptive fields. openreview.net/pdf?id=P9yXPbf…
2
8
52
0
13
Download Image
@TrentonBricken we also found a relationship between noise and sparsity regularizers on activations back in 2014 :) arxiv.org/abs/1406.1831
@poolio Awesome! These seem very complimentary. Would it be fair to say this newer paper shows receptive fields appear for natural images and that sparsity remains even after noise is removed?