Discovering trade-offs between fairness and accuracy in machine learning systems: a multi-objective approach

Abstract

In recent years, the focus of machine learning has expanded from solely emphasizing accuracy to adopting a more comprehensive and human-centred perspective that includes privacy, fairness, and transparency. These aspects are frequently perceived as conflicting; for instance, there can be a trade-off between the accuracy and fairness of predictive models. Fairness analysis aims to ensure that machine learning models do not exhibit discrimination based on protected or sensitive attributes such as race, gender, age, or religion. However, multiple notions of fairness exist, and not all are compatible with each other. This thesis investigates the relationship between accuracy and various fairness metrics using three datasets, demonstrating that standard training techniques can lead to biased predictions. We examine the relationship between accuracy and six distinct fairness metrics through a multi-objective training approach designed to optimize both accuracy and fairness. Our findings reveal that while some scenarios exhibit a trade-off between accuracy and fairness, the multi-objective approach offers a range of models that balance these trade-offs. In other cases, the approach facilitates the development of models that are both accurate and fair, a result not achievable with single-objective methods. Consequently, our research highlights that explicitly incorporating fairness into the training process enables decision makers to access a spectrum of models meeting both accuracy and fairness criteria and identifies scenarios where a trade-off between fairness and accuracy is not necessary.