2000 character limit reached
Symmetry Breaking in Symmetric Tensor Decomposition
Published 10 Mar 2021 in math.OC and cs.LG | (2103.06234v2)
Abstract: In this note, we consider the highly nonconvex optimization problem associated with computing the rank decomposition of symmetric tensors. We formulate the invariance properties of the loss function and show that critical points detected by standard gradient based methods are \emph{symmetry breaking} with respect to the target tensor. The phenomena, seen for different choices of target tensors and norms, make possible the use of recently developed analytic and algebraic tools for studying nonconvex optimization landscapes exhibiting symmetry breaking phenomena of similar nature.
- P. Comon, “Tensor decompositions,” Mathematics in Signal Processing V, pp. 1–24, 2002.
- P. Comon and M. Rajih, “Blind identification of under-determined mixtures based on the characteristic function,” Signal Processing, vol. 86, no. 9, pp. 2271–2281, 2006.
- L. De Lathauwer, B. De Moor, and J. Vandewalle, “A multilinear singular value decomposition,” SIAM Journal on Matrix Analysis and Applications, vol. 21, no. 4, pp. 1253–1278, 2000.
- T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Review, vol. 51, no. 3, pp. 455–500, 2009.
- N. D. Sidiropoulos, R. Bro, and G. B. Giannakis, “Parallel factor analysis in sensor array processing,” IEEE Transactions on Signal Processing, vol. 48, no. 8, pp. 2377–2388, 2000.
- John Wiley & Sons, 2005.
- J. Landsberg, Tensors: Geometry and Applications: Geometry and Applications. Volume 128 of Graduate Studies in Mathematics, American Mathematical Society, 2011.
- Y. Arjevani and G. Vinograd, “Symmetry & critical points for symmetric tensor decompositions problems,” arXiv preprint arXiv:2306.5319838, 2023.
- T. G. Kolda, “Numerical optimization for symmetric tensor decomposition,” Mathematical Programming, vol. 151, no. 1, pp. 225–248, 2015.
- J. Nie, “Generating polynomials and symmetric tensor decompositions,” Foundations of Computational Mathematics, vol. 17, no. 2, pp. 423–465, 2017.
- “Foundations of the PARAFAC procedure: models and conditions for an “explanatory” multimodal factor analysis,” UCLA Working Papers in Phonetics, vol. 16, pp. 1–84, 1970.
- L. De Lathauwer, J. Castaing, and J.-F. Cardoso, “Fourth-order cumulant-based blind identification of underdetermined mixtures,” IEEE Transactions on Signal Processing, vol. 55, no. 6, pp. 2965–2973, 2007.
- S. B. Hopkins, T. Schramm, and J. Shi, “A robust spectral algorithm for overcomplete tensor decomposition,” in Conference on Learning Theory, pp. 1683–1722, PMLR, 2019.
- J. Kileel and J. M. Pereira, “Subspace power method for symmetric tensor decomposition and generalized PCA,” arXiv preprint arXiv:1912.04007, 2019.
- S. Sherman and T. G. Kolda, “Estimating higher-order moments using symmetric tensor decomposition,” SIAM Journal on Matrix Analysis and Applications, vol. 41, no. 3, pp. 1369–1387, 2020.
- Y. Arjevani and M. Field, “On the principle of least symmetry breaking in shallow ReLU models,” arXiv preprint arXiv:1912.11939, 2019.
- Y. Arjevani and M. Field, “Symmetry & critical points for symmetric tensor decomposition problems,” 2022.
- Y. Arjevani and M. Field, “Symmetry & critical points for a model shallow neural network,” Physica D: Nonlinear Phenomena, vol. 427, p. 133014, 2021.
- Y. Arjevani and M. Field, “Analytic characterization of the hessian in shallow relu models: A tale of symmetry,” Advances in Neural Information Processing Systems, vol. 33, pp. 5441–5452, 2020.
- Y. Arjevani and M. Field, “Analytic study of families of spurious minima in two-layer relu neural networks: a tale of symmetry ii,” Advances in Neural Information Processing Systems, vol. 34, pp. 15162–15174, 2021.
- Y. Arjevani and M. Field, “Annihilation of spurious minima in two-layer relu networks,” Advances in Neural Information Processing Systems, vol. 35, pp. 37510–37523, 2022.
- Y. Arjevani and M. Field, “Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of s k,” Nonlinearity, vol. 35, no. 6, p. 2809, 2022.
- Y. Arjevani, “Hidden minima in two-layer relu networks,” arXiv preprint arXiv:2312.06234, 2023.
- Y. Arjevani, J. Bruna, M. Field, J. Kileel, M. Trager, and F. Williams, “Symmetry breaking in symmetric tensor decomposition,” arXiv preprint arXiv:2103.06234, 2021.
- L. Isserlis, “On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables,” Biometrika, vol. 12, no. 1/2, pp. 134–139, 1918.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.