In Section 1 of [1], the author gives an hierarchical overview of neural networks. In the deep exponential families paper, the authors imitate the hierarchical structure in neural networks and replace each layer with some distributions from exponential families. The inference is by 'black box' variational inference.
[1]. Gal, Yarin. Uncertainty in deep learning. Diss. PhD thesis, University of Cambridge, 2016.