Original
Measuring disparate impact in human and machine decisions
Abstract
Empirical analyses have grown increasingly important in discrimination litigation with the greater availability of detailed data on individuals and decisions. A popular analytic strategy is to estimate disparities after adjusting for observed covariates, typically with a regression model, in hopes of ferreting out discriminatory intent. This approach, however, is ill-suited to auditing algorithms that are now commonly used to aid decisions, which typically do not include race or other legally protected factors as inputs. Motivated by legal understandings of disparate impact, we introduce an approach that aims to measure "unjustified" disparities in both human and machine decisions. Our method, which we call risk-adjusted regression, proceeds in three steps. In the first step, we combine all available information in a machine learning model to estimate the value, or inversely, the risk, of taking a certain action, such as approving a loan application or hiring a job candidate. Second, we measure disparities in decisions after adjusting for these risk estimates alone. Finally, in the third step, we assess the sensitivity of results to potential mismeasurement of risk. We demonstrate this approach on a detailed dataset of 2.2 million police stops of pedestrians in New York City, and show that traditional statistical tests of discrimination can substantially understate the magnitude of (risk-adjusted) racial disparities.
中文
衡量人类与机器决策中的差别性影响
摘要
随着关于个人和决策的详细数据更易获得，实证分析在反歧视诉讼中变得越来越重要。一种流行的分析策略是在调整已观测协变量后估计差异，通常使用回归模型，以期发现歧视意图。然而，这种方法不适合审计目前常用于辅助决策的算法，因为这些算法通常不将种族或其他受法律保护的因素作为输入。受法律上对差别性影响的理解启发，我们提出一种方法，旨在衡量人类决策与机器决策中“不公正的”差异。我们将该方法称为风险调整回归，其分为三个步骤。第一步，我们在一个机器学习模型中整合所有可用信息，以估计采取某一行动的价值，或反过来说，风险，例如批准贷款申请或录用求职者。第二步，我们仅在这些风险估计进行调整后衡量决策中的差异。最后，第三步，我们评估结果对风险可能测量误差的敏感性。我们在纽约市220万次行人警察拦截的详细数据集上演示了该方法，并表明传统的歧视统计检验可能大幅低估（经风险调整的）种族差异的规模。
关键词
差别性影响、算法审计、风险调整回归、机器学习、种族差异、警察拦截、歧视、算法公平