Order Optimal Cascaded Code Distributed Computing With Low Complexity and Improved Flexibility

Abstract: Coded distributed computing (CDC), proposed by Li \emph{et al.}, offers significant potential for reducing the communication load in MapReduce computing systems. In cascaded CDC with $K$ nodes, $N$ input files, and $Q$ output functions, each input file will be mapped by $r\geq 1$ nodes and each output function will be computed by $s>1$ nodes such that coding techniques can be applied to generate multicast opportunities. However, a significant limitation of most existing coded distributed computing schemes is their requirement to split the original data into a large number of input files (or output functions) that grows exponentially with $K$, which significantly increases the coding complexity and degrades the system performance. In this paper, we focus on the case of $K/s\in\mathbb{N}$, deliberately designing the strategy of data placement and output functions assignment, such that a low-complexity CDC scheme is achievable. The main advantages of the proposed scheme include: 1) the multicast gains equal to $(r+s-1)(1-1/s)$ and $r+s-1$ which is approximately $r+s-1$ when $s$ is relatively large, and the communication load potentially better than the well-known scheme proposed by Li \emph{et al.}; 2) the proposed scheme requires significantly less input files and output functions; 3) all the operations are implemented over the binary field $\mathbb{F}_2$ with the one-shot fashion (i.e., each node can decode its requested content immediately upon receiving the multicast message during the current time slot). Finally, we derive a new information-theoretic converse bound for the cascaded CDC framework under the proposed strategies of data placement and output functions assignment. We demonstrate that the communication load of the proposed scheme is order optimal within a factor of $2$; and is also approximately optimal when $K$ is sufficiently large for a given $r$.