BeamClean: Language Aware Embedding Reconstruction

Abstract: In this work, we consider an inversion attack on the obfuscated input embeddings sent to a LLM on a server, where the adversary has no access to the LLM or the obfuscation mechanism and sees only the obfuscated embeddings along with the model's embedding table. We propose BeamClean, an inversion attack that jointly estimates the noise parameters and decodes token sequences by integrating a language-model prior. Against Laplacian and Gaussian obfuscation mechanisms, BeamClean always surpasses naive distance-based attacks. This work highlights the necessity for and robustness of more advanced learned, input-dependent methods.