![[因果关系与鲁棒性视角下的算法公平性研讨会 Logo]](/v171/assets/images/logo-pmlr.svg){kind=logo}
[编辑]
Fair SA：面部识别中的公平性敏感性分析
Aparna R. Joshi，Xavier Suau Cuadros，Nivedha Sivakumar，Luca Zappella，Nicholas Apostoloff因果关系和鲁棒性视角下的算法公平性会议论文集，PMLR 171:40-58，2022。
摘要
随着深度学习在重大领域中的应用日益普及，评估模型的稳健性变得越来越重要。面部识别就是一个重要的领域，其现实世界的应用涉及受各种退化影响的图像，例如运动模糊或过度曝光。此外，在性别和种族等不同属性下捕获的图像也会对人脸识别算法的鲁棒性构成挑战。虽然传统的汇总统计数据表明人脸识别模型的整体性能不断提高，但这些指标并不能直接衡量模型的鲁棒性或公平性。视觉心理物理敏感性分析 (VPSA) (19) 提供了一种通过在数据中引入增量扰动来确定失败的个体原因的方法。然而，扰动可能会对不同子组产生不同的影响。在本文中，我们提出了一种基于鲁棒性的新型公平性评估，它以通用框架的形式扩展了 VPSA。通过该框架，我们可以分析模型对受扰动影响的群体不同子组进行公平评估的能力，并通过测量目标鲁棒性来确定子组的确切故障模式。随着对模型公平性的日益关注，我们以面部识别作为我们框架的一个示例应用，并提出通过 AUC 矩阵紧凑地可视化模型的公平性分析。我们分析了常用的人脸识别模型的性能，并经验性地表明，当图像受到扰动时，某些子组处于不利地位，从而揭示了在使用未受扰动的子组的模型性能中看不到的趋势。
引用本文
BibTeX
@InProceedings{pmlr-v171-joshi22a, title = {Fair {SA}: Sensitivity Analysis for Fairness in Face Recognition}, author = {Joshi, Aparna R. and Suau Cuadros, Xavier and Sivakumar, Nivedha and Zappella, Luca and Apostoloff, Nicholas}, booktitle = {Proceedings of The Algorithmic Fairness through the Lens of Causality and Robustness}, pages = {40--58}, year = {2022}, editor = {Schrouff, Jessica and Dieng, Awa and Rateike, Miriam and Kwegyir-Aggrey, Kweku and Farnadi, Golnoosh}, volume = {171}, series = {Proceedings of Machine Learning Research}, month = {13 Dec}, publisher = {PMLR}, pdf = {https://pmlr.com.cn/v171/joshi22a/joshi22a.pdf}, url = {https://pmlr.com.cn/v171/joshi22a.html}, abstract = {As the use of deep learning in high impact domains becomes ubiquitous, it is increasingly important to assess the resilience of models. One such high impact domain is that of face recognition, with real world applications involving images affected by various degradations, such as motion blur or high exposure. Moreover, images captured across different attributes, such as gender and race, can also challenge the robustness of a face recognition algorithm. While traditional summary statistics suggest that the aggregate performance of face recognition models has continued to improve, these metrics do not directly measure the robustness or fairness of the models. Visual Psychophysics Sensitivity Analysis (VPSA) (19) provides a way to pinpoint the individual causes of failure by way of introducing incremental perturbations in the data. However, perturbations may affect subgroups differently. In this paper, we propose a new fairness evaluation based on robustness in the form of a generic framework that extends VPSA. With this framework, we can analyze the ability of a model to perform fairly for different subgroups of a population affected by perturbations, and pinpoint the exact failure modes for a subgroup by measuring targeted robustness. With the increasing focus on the fairness of models, we use face recognition as an example application of our framework and propose to compactly visualize the fairness analysis of a model via AUC matrices. We analyze the performance of common face recognition models and empirically show that certain subgroups are at a disadvantage when images are perturbed, thereby uncovering trends that were not visible using the model’s performance on subgroups without perturbations.} }
Endnote
%0 会议论文 %T Fair SA：面部识别中的公平性敏感性分析 %A Aparna R. Joshi %A Xavier Suau Cuadros %A Nivedha Sivakumar %A Luca Zappella %A Nicholas Apostoloff %B 因果关系和鲁棒性视角下的算法公平性会议论文集 %C 机器学习研究会议论文集 %D 2022 %E Jessica Schrouff %E Awa Dieng %E Miriam Rateike %E Kweku Kwegyir-Aggrey %E Golnoosh Farnadi %F pmlr-v171-joshi22a %I PMLR %P 40--58 %U https://pmlr.com.cn/v171/joshi22a.html %V 171 %X As the use of deep learning in high impact domains becomes ubiquitous, it is increasingly important to assess the resilience of models. One such high impact domain is that of face recognition, with real world applications involving images affected by various degradations, such as motion blur or high exposure. Moreover, images captured across different attributes, such as gender and race, can also challenge the robustness of a face recognition algorithm. While traditional summary statistics suggest that the aggregate performance of face recognition models has continued to improve, these metrics do not directly measure the robustness or fairness of the models. Visual Psychophysics Sensitivity Analysis (VPSA) (19) provides a way to pinpoint the individual causes of failure by way of introducing incremental perturbations in the data. However, perturbations may affect subgroups differently. In this paper, we propose a new fairness evaluation based on robustness in the form of a generic framework that extends VPSA. With this framework, we can analyze the ability of a model to perform fairly for different subgroups of a population affected by perturbations, and pinpoint the exact failure modes for a subgroup by measuring targeted robustness. With the increasing focus on the fairness of models, we use face recognition as an example application of our framework and propose to compactly visualize the fairness analysis of a model via AUC matrices. We analyze the performance of common face recognition models and empirically show that certain subgroups are at a disadvantage when images are perturbed, thereby uncovering trends that were not visible using the model’s performance on subgroups without perturbations.
APA
Joshi, A.R., Suau Cuadros, X., Sivakumar, N., Zappella, L. & Apostoloff, N. (2022)。Fair SA：面部识别中的公平性敏感性分析。因果关系和鲁棒性视角下的算法公平性会议论文集，发表于机器学习研究会议论文集 171:40-58。可从 https://pmlr.com.cn/v171/joshi22a.html 获取。