Disparate impact of damage-detection on anonymous Wikipedia editors

Today, I'm writing briefly about a problem that I expect to be studying and trying to fix over the course of the next few weeks.

The problem: The damage detection models that ORES supports seems to be overly skeptical of edits by anonymous editors and newcomers.

I've been looking at this problem for a while, but I was recently inspired by by the framing of disparate impact. Thanks to Jacob Thebault-Spieker for suggesting I look at the problem this way.

In United States anti-discrimination law, the theory of disparate impact holds that practices in employment, housing, or other areas may be considered discriminatory and illegal if they have a disproportionate "adverse impact" on persons in a protected class.  via Wikipedia's Disparate Impact (CC-BY-SA 4.0)

So, let's talk about some terms and how I'd like to apply them to Wikipedia.

Disproportionate adverse impact.  The damage detection models that ORES supports are intended to focus attention on potentially damaging edits.  Still human judgement is not perfect and there's lot of fun research that suggests that "recommendations" like this can affect people's judgement.  So by encouraging Wikipedia's patrollers to look a particular edit, we are likely also making them more likely to find flaws in that edit than if it was not highlighted by ORES.  Having an edit rejected can demotivate the editor, but it may be even more concerning that the rejection of content from certain types of editors may lead to coverage biases as the editors most likely to contribute to a particular topic may be discouraged or prevented from editing Wikipedia

Protected class.  In US law, it seems that this term is generally reserved for race, gender, and ability.  In the case of Wikipedia, we don't know these demographics.  They could be involved and I think they likely are, but I think that anonymous editors and newcomers should also be considered a protected class in Wikipedia.  Generally, anonymous editors and newcomers are excluded from discussions and therefor subject to the will of experienced editors.  I think that this has been having a substantial, negative impact on the quality and coverage of Wikipedia.  To state it simply, I think that there are a collection of systemic problems around anonymous editors and newcomers that prevent them from contributing to the dominant store of human knowledge.

So, I think I have a moral obligation to consider the effect that these algorithms have in contributing to these issues and rectifying them.  The first and easiest thing I can do is remove the features user.age and user.is_anon from the prediction models.  So I did some testing.  Here's fitness measures (see AUC) all of the edit quality models with the current and without-user features included.

wiki	model	current AUC	no-user AUC	diff
dewiki	reverted	0.900	0.792	-0.108
enwiki	reverted	0.835	0.795	-0.040
enwiki	damaging	0.901	0.818	-0.083
enwiki	goodfaith	0.896	0.841	-0.055
eswiki	reverted	0.880	0.849	-0.031
fawiki	reverted	0.913	0.835	-0.078
fawiki	damaging	0.951	0.920	-0.031
fawiki	goodfaith	0.961	0.897	-0.064
frwiki	reverted	0.929	0.846	-0.083
hewiki	reverted	0.874	0.800	-0.074
idwiki	reverted	0.935	0.903	-0.032
itwiki	reverted	0.905	0.850	-0.055
nlwiki	reverted	0.933	0.831	-0.102
ptwiki	reverted	0.894	0.812	-0.082
ptwiki	damaging	0.913	0.848	-0.065
ptwiki	goodfaith	0.923	0.863	-0.060
trwiki	reverted	0.885	0.809	-0.076
trwiki	damaging	0.892	0.798	-0.094
trwiki	goodfaith	0.899	0.795	-0.104
viwiki	reverted	0.905	0.841	-0.064

So to summarize what this table tells us:  We'll lose between 0.05 and 0.10 AUC per model which brings us from beating the state of the art to not.  That makes the quantitative glands in my brain squirt some anti-dopamine out.  It makes me want to run the other way.  It's really cool to be able to say "we're beating the state of the art".  But on the other hand, it's kind of lame to know "we're doing it at the expense of users who are most sensitive and necessary."  So, I've convinced myself.  We should deploy these models that look less fit by the numbers, but also reduce the disparate impact on anons and new editors.  After all, the actual practical application of the model may very well actually be better despite what the numbers say.

But before I do anything, I need to convince my users.  They should have a say in this.  At the very least, they should know what is happening.  So, next week, I'll start a conversation laying out this argument and advocating for the switch.  

One final note.  This problem may be a blessing in disguise.  By reducing the fitness of our models, we have a new incentive to re-double our efforts toward finding alternative sources of signal to increase the fitness of our models.