I'm sorry but this comment is just at odds with the current literature in deep learning.
A few very prominent researchers have been trying to use information theoretic concepts to explain generalization for a while now-see
the somewhat controversial branch Naftali Tishby developed...However many other papers apply 'actual information theory'....
I think the GP poster meant to contrast this paper not with other papers, but with the junk explanations for "how deep learning works" that science journalism comes up with.
trust me some of the junk explanations for how deep learning works originate from researchers. we really just don't have a convincing picture about why these networks work or don't work in a theoretical way rather than empirical/heuristic handwaves
if you are interested in this type of analysis are you familiar with the tishby papers? you will at the very least find them enjoyable/mind opening...
but some of his key claims are probably contradicted by empirical results.
He's a good speaker/teacher but I think there are serious problems with his work- mainly that a lot of his results collapse when tested on more general architectures.
