Symmetry Discovery for Different Data Types
Abstract: Equivariant neural networks incorporate symmetries into their architecture, achieving higher generalization performance. However, constructing equivariant neural networks typically requires prior knowledge of data types and symmetries, which is difficult to achieve in most tasks. In this paper, we propose LieSD, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks. It characterizes equivariance and invariance (a special case of equivariance) of continuous groups using Lie algebra and directly solves the Lie algebra space through the inputs, outputs, and gradients of the trained neural network. Then, we extend the method to make it applicable to multi-channel data and tensor data, respectively. We validate the performance of LieSD on tasks with symmetries such as the two-body problem, the moment of inertia matrix prediction, and top quark tagging. Compared with the baseline, LieSD can accurately determine the number of Lie algebra bases without the need for expensive group sampling. Furthermore, LieSD can perform well on non-uniform datasets, whereas methods based on GANs fail.
- A generative model of symmetry transformations. arXiv preprint arXiv:2403.01946 .
- Learning invariances in neural networks from training data. Advances in Neural Information Processing Systems 33, 17605–17616.
- Group equivariant convolutional networks, in: International Conference on Machine Learning, PMLR. pp. 2990–2999.
- Steerable CNNs. arXiv preprint arXiv:1612.08498 .
- Automatic symmetry discovery with Lie algebra convolutional network. Advances in Neural Information Processing Systems 34, 2503–2515.
- Symmetry discovery with deep learning. Physical Review D 105, 096031.
- Classifying the classifier: dissecting the weight space of neural networks, in: ECAI 2020. IOS Press, pp. 1119–1126.
- A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups, in: International Conference on Machine Learning, PMLR. pp. 3318–3328.
- Hamiltonian neural networks. Advances in Neural Information Processing Systems 32.
- Neural ePDOs: Spatially adaptive equivariant partial differential operator based networks, in: The Eleventh International Conference on Learning Representations.
- Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 .
- Detecting symmetries with neural networks. Machine Learning: Science and Technology 2, 015010.
- Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324.
- Affine equivariant networks based on differential invariants, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5546–5556.
- Enabling equivariance for arbitrary Lie groups, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8183–8192.
- LieGG: Studying learned Lie group generators. Advances in Neural Information Processing Systems 35, 25212–25223.
- Equivariant architectures for learning in deep weight spaces, in: International Conference on Machine Learning, PMLR. pp. 25790–25816.
- Learning layer-wise equivariances automatically using gradients. Advances in Neural Information Processing Systems 36.
- Learning partial equivariances from data. Advances in Neural Information Processing Systems 35, 36466–36478.
- E (n) equivariant graph neural networks, in: International Conference on Machine Learning, PMLR. pp. 9323–9332.
- Self-supervised representation learning on neural network weights for model characteristic prediction. Advances in Neural Information Processing Systems 34, 16481–16493.
- PDO-eConvs: Partial differential operator based equivariant convolutions, in: International Conference on Machine Learning, PMLR. pp. 8697–8706.
- PDO-s3DCNNs: Partial differential operator based steerable 3D CNNs, in: International Conference on Machine Learning, PMLR. pp. 19827–19846.
- PDO-eS2CNNs: Partial differential operator based equivariant spherical CNNs, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 9585–9593.
- Self-supervised latent symmetry discovery via class-pose decomposition, in: NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations.
- Predicting neural network accuracy from weights. arXiv preprint arXiv:2002.11448 .
- General E (2)-equivariant steerable CNNs. Advances in Neural Information Processing Systems 32.
- 3D steerable CNNs: Learning rotationally equivariant features in volumetric data. Advances in Neural Information Processing Systems 31.
- Learning steerable filters for rotation equivariant CNNs, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 849–858.
- Adan: Adaptive nesterov momentum algorithm for faster optimizing deep models. IEEE Transactions on Pattern Analysis and Machine Intelligence .
- Latent space symmetry discovery. arXiv preprint arXiv:2310.00105 .
- Generative adversarial symmetry discovery, in: International Conference on Machine Learning, PMLR. pp. 39488–39508.
- Meta-learning symmetries by reparameterization. arXiv preprint arXiv:2007.02933 .
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.