A History of Technology Misuse in Recruiting — Act II: Amazon's Mirror

If there's a single story that crystallized the problem of algorithmic bias in recruiting, it's Amazon's.

In 2014, Amazon's engineering team built an AI tool designed to automate résumé screening — the kind of thing that, in theory, would cut through human subjectivity and surface the best candidates on merit alone. What it actually did was spend years studying a decade's worth of historical hiring data and conclude that the most successful candidates at Amazon were, overwhelmingly, men.

The algorithm quietly downgraded résumés that included the word "women's" (as in "women's chess club captain") having simply learned from 10 years of hiring data dominated by male candidates in technical roles. The tool wasn't taught to discriminate. It was taught to replicate. And it did exactly that, faithfully, at scale.

Amazon scrapped the tool. But the lesson it left behind is one that the industry has been slow to internalize: when training data reflects past discriminatory practices, such as the underrepresentation of women in technical roles, the AI learns those patterns as indicators of success, creating a detrimental feedback loop in which past biases are automated and projected into future hiring decisions.

This is the central irony of algorithmic hiring. The premise is that removing humans removes bias. But humans built the training data. Humans designed the models. And humans decided what "success" looked like in the first place. When trained on the résumés of a company's current employees, the algorithm reproduces the human bias that led to the company's workforce being predominantly white and male — further entrenching systemic inequities.

You don't eliminate bias by digitizing it. You just make it harder to see, and harder to challenge.