Notes on the Symmetries of 2-Layer ReLU-Networks.

NLDL(2020)

引用 0|浏览2
暂无评分
摘要
Symmetries in neural networks allow different weight configurations leading to the same network function. For odd activation functions, the set of transformations mapping between such configurations have been studied extensively, but less is known for neural networks with ReLU activation functions. We give a complete characterization for fully-connected networks with two layers. Apart from two well-known transformations, only degenerated situations allow additional transformations that leave the network function unchanged. Reduction steps can remove only part of the degenerated cases. Finally, we present a non-degenerate situation for deep neural networks leading to new transformations leaving the network function intact.
更多
查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络