An in-depth analysis of guesser behavior

Abstract

We propose a methodology to perform an in-depth analysis on different password guessers and their guessing abilities. We devise new metrics and statistics that directly compare the types of passwords each guesser generates, extending analysis beyond number of passwords guessed which is the primary form of analysis in literature currently. This approach allows for a _ne-grained analysis where we compare the guesses produced by each guesser when trained on varied real-world datasets and under different conditions (e.g., limited training data, limited number of guesses, or dissimilar training and testing data). We find that similarity of training to testing data is more important than dataset size, and that some guessers are better equipped to deal with dissimilarity than others. We demonstrate that guessers often produce dissimilar guesses, even when trained on the same training data. This result is leveraged to show how guessers with lower resource requirements can be combined to guess a comparable number of passwords as more resource intensive tools. Our methodology can be applied in the future to better compare new guessing tools, and our insights allow us to provide concrete advice for systems administrators performing reactive checking with modern tools.