Sparse Random Matrices for Dimensionality Reduction

Abstract: The Johnson-Lindenstrauss (JL) theorem states that a set of points in high-dimensional space can be embedded into a lower-dimensional space while approximately preserving pairwise distances with high probability Johnson and Lindenstrauss (1984). The standard JL theorem uses dense random matrices with Gaussian entries. However, for some applications, sparse random matrices are preferred as they allow for faster matrix-vector multiplication. I outline the constructions and proofs introduced by Achlioptas (2003) and the contemporary standard by Kane and Nelson (2014). Further, I implement and empirically compare these sparse constructions with standard Gaussian JL matrices.