The Evolution of User-Selected Passwords: A Quantitative Analysis of Publicly Available Datasets

Abstract: The aim of this work is to study the evolution of password selection among users. We investigate whether users follow best practices when selecting passwords and identify areas in need of improvement. Four distinct publicly-available password datasets (obtained from security breaches, compiled by security experts, and designated as containing bad passwords) are employed. As these datasets were released at different times, the distributions characterizing these datasets suggest a chronological evolution of password selection. A similarity metric, Levenshtein distance, is used to compare passwords in each dataset against the designated benchmark of bad passwords. The resulting distributions of normalized similarity scores are then compared to each other. The comparison reveals an overall increase in the mean of the similarity distributions corresponding to more recent datasets, implying a shift away from the use of bad passwords. This conclusion is corroborated by the passwords' clustering behavior. An encoding capturing best practices maps passwords to a high dimensional space over which a $k$-means clustering (with silhouette coefficient) analysis is performed. Cluster comparison and character frequency analysis indicates an improvement in password selection over time with respect to certain features (length, mixing character types), yet certain discouraged practices (name inclusion, selection bias) still persist.