The Multi-Dimensional Decomposition with Constraints

Abstract: We search for the best fit in Frobenius norm of $A \in {\mathbb C}^{m \times n}$ by a matrix product $B C^*$, where $B \in {\mathbb C}^{m \times r}$ and $C \in {\mathbb C}^{n \times r}$, $r \le m$ so $B = {b_{ij}}$, ($i=1, \dots, m$,~ $j=1, \dots, r$) definite by some unknown parameters $\sigma_1, \dots, \sigma_k$, $k << mr$ and all partial derivatives of $\displaystyle \frac{\delta b_{ij}}{\delta \sigma_l}$ are definite, bounded and can be computed analytically. We show that this problem transforms to a new minimization problem with only $k$ unknowns, with analytical computation of gradient of minimized function by all $\sigma$. The complexity of computation of gradient is only 4 times bigger than the complexity of computation of the function, and this new algorithm needs only $3mr$ additional memory. We apply this approach for solution of the three-way decomposition problem and obtain good results of convergence of Broyden algorithm.