DemogPairs: quantifying the impact of demographic imbalance in deep face recognition

Although deep face recognition has achieved impressive results in recent years, there is increasing controversy regarding racial and gender bias of the models, questioning their trustworthiness and deployment into sensitive scenarios. In this work we present DemogPairs, a new validation set with 10.8K facial images and 58.3M identity verification pairs, distributed in demographically-balanced folds of Asian, Black and White females and males. We also propose a benchmark of experiments using DemogPairs over state-of-the-art deep face recognition models in order to analyze their cross-demographic behavior and potential demographic biases (see figure below).

Dataset download: DemogPairs can be downloaded from this Google Drive link (173MB).

Dataset usage rules: The DemogPairs dataset is available for non-commercial research purposes only. By downloading DemogPairs you agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the images and any portion of derived data. Please cite our paper if the DemogPairs dataset is useful to your research.