André Dörsch · Torsten Schlett · Peter Munch · Christian Rathgeb · Christoph Busch
This repository contains the official implementation of the ICPR 2024 - 2nd Workshop on Fairness in Biometric Systems paper Fairness measures for biometric quality assessment
This work proposes multiple fairness measures for evaluating quality components across demographic groups. This research contributes to the development of fair biometric systems by measuring whether quality components could incorporate demographic variables into their decision.
Quality assessment algorithms measure the quality of a captured biometric sample. Since the sample quality strongly affects the recognition performance of a biometric system, it is essential to only process samples of sufficient quality and discard samples of low-quality. Even though quality assessment algorithms are not intended to yield very different quality scores across demographic groups, quality score discrepancies are possible, resulting in different discard ratios. To ensure that quality assessment algorithms do not take demographic characteristics into account when assessing sample quality and consequently to ensure that the quality algorithms perform equally for all individuals, it is crucial to develop a fairness measure. In this work we propose and compare multiple fairness measures for evaluating quality components across demographic groups. Proposed measures, could be used as potential candidates for an upcoming standard in this important field.
Ensure you have Python >= 3.8.6 installed on your system.
To install the necessary dependencies for the usage of proposed fairness measures, run the following command in your terminal:
pip install numpy=1.23.4
If you are interested in re-running the experiments from the paper in the provided jupyter-notebook, you will need to install additional packages:
pip install pandas=1.1.5, seaborn=0.11.1, matplotlib=3.3.3
git clone https://github.com/dasec/QA-Fairness-Measures.git
# Imports
from src.fairness_measures.fairness_measure_gc import gini_coefficient
# Example: Mean Quality Scores from three different demographic groups
Mean_QS_Group_A, Mean_QS_Group_B, Mean_QS_Group_C = 35, 95, 89
# Calculate the Gini Coefficient
gini_score = gini_coefficient([35, 95, 89])
# Output: 0.273972602739726
# Calculate the Mean-GC-SQFR
Mean_GC_SQFR = 1 - gini_score
# Output: 0.726027397260274
# Calculate the Mean-GC-CSQFR
Mean_GC_CSQFR = (1 - gini_value) ** 3
# Output: 0.38270049894991737
# Imports
from src.fairness_measures.fairness_measure_gc import gini_coefficient
from src.fairness_measures.fairness_measure_lwm import low_weighted_mean_score
# Example: Quality Scores from two different demographic groups
Qd1, Qd2 = [1,2,3], [4,5,6]
QD = np.concatenate([Qd1,Qd2])
LWM_D1 = low_weighted_mean_score([Qd1, QD])
# Output: 1.8333333333333335
LWM_D2 = low_weighted_mean_score([Qd2, QD])
# Output: 4.333333333333333
LWM_GC_SQFR = 1 - gini_coefficient([LWM_D1, LWM_D2])
# Output: 0.5945945945945946
LWM_GC_CSQFR = (1 - gini_coefficient([LWM_D1, LWM_D2])**3)
# Output: 0.21021459735
# Imports
from src.fairness_measures.fairness_measure_mdg import mean_discard_gap
# Example: Quality Scores from three different demographic groups
group_a = np.array([76, 76, 77, 77, 78, 79, 80, 82, 82, 84])
group_b = np.array([84, 84, 84, 84, 85, 85, 87, 87, 88, 88])
group_c = np.array([82, 82, 82, 84, 84, 86, 87, 87, 88, 88])
groups = [group_a, group_b, group_c]
MGD_SQFR = 1 - mean_discard_gap(groups)
# Output: 0.44999999999999996
If you use this repository and found it useful for your research, please consider citing this paper:
@misc{Dörsch-FairnessMeasures-2024,
title={Fairness measures for biometric quality assessment},
author={A. Dörsch and T. Schlett and P. Munch and C. Rathgeb and C.Busch},
year={2024},
eprint={2408.11392},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2408.11392},
}