Enhancing Drug Discovery: Autoencoder-Based Latent Space Augmentation for Improved Molecular Solubility Prediction using LatMixSol

Abstract: Accurate prediction of molecular solubility is a cornerstone of early-stage drug discovery, yet conventional machine learning models face significant challenges due to limited labeled data and the high-dimensional nature of molecular descriptors. To address these issues, we propose LatMixSol, a novel latent space augmentation framework that combines autoencoder-based feature compression with guided interpolation to enrich training data. Our approach first encodes molecular descriptors into a low-dimensional latent space using a two-layer autoencoder. Spectral clustering is then applied to group chemically similar molecules, enabling targeted MixUp-style interpolation within clusters. Synthetic samples are generated by blending latent vectors of cluster members and decoding them back to the original feature space. Evaluated on the Huuskonen solubility benchmark, LatMixSol demonstrates consistent improvements across three of four gradient-boosted regressors (CatBoost, LightGBM, HistGradientBoosting), achieving RMSE reductions of 3.2-7.6% and R-squared increases of 0.5-1.5%. Notably, HistGradientBoosting shows the most significant enhancement with a 7.6% RMSE improvement. Our analysis confirms that cluster-guided latent space augmentation preserves chemical validity while expanding dataset diversity, offering a computationally efficient strategy to enhance predictive models in resource-constrained drug discovery pipelines.