Amazon face recognition falsely matches 28 lawmakers with mugshots, ACLU says
Amazon’s facial recognition technology falsely identified 28 members of Congress as people who have been arrested for crimes, according to the American Civil Liberties Union (ACLU).
The ACLU of Northern California’s test of Amazon’s controversial Rekognition software also found that people of color were disproportionately misidentified in a mugshot database, raising new concerns about racial bias and the potential for abuse by law enforcement.
The report followed revelations in May that Amazon has been marketing and selling the Rekognition technology to police agencies, leading privacy advocates to urge CEO Jeff Bezos to stop providing the product to the government.
“Our test reinforces that face surveillance is not safe for government use,” Jacob Snow, a technology and civil liberties attorney at the ACLU Foundation of Northern California, said in a statement. “Face surveillance will be used to power discriminatory surveillance and policing that targets communities of color, immigrants, and activists. Once unleashed, that damage can’t be undone.”
The ACLU used the same facial recognition system that Amazon offers to the public, scanning for matches between images of faces. The group built a face database and search tool using 25,000 public arrest photos, then cross-referenced that data with public photos of every member of the US House and Senate.
Of the 28 people misidentified by the ACLU’s test, 11 of them were people of color, nearly 40% of those wrongly matched, despite the fact that people of color make up only 20% of those in Congress. Six members of the Congressional Black Caucus were included in the false matches, including the civil rights leader John Lewis.
“The ACLU’s test demonstrates beyond a shadow of a doubt what many of us already know – that Amazon’s facial recognition tool is discriminatory ... and therefore dangerous to democracy,” said Malkia Cyril, executive director of the Center for Media Justice. “They need to roll it back immediately.”
Amazon defended its technology in a statement, saying the ACLU’s results could “probably be improved” if the test had increased the “confidence thresholds”, meaning the likelihood that Rekognition found a match. The ACLU used an 80% confidence threshold, but Amazon said in its statement, “When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95% or higher.”
The ACLU, however, has pointed out that it used the default match settings that Amazon has set for Rekognition, and that the company currently references an 80% confidence metric for recognizing faces on its own website .
Amazon’s response seemed to admit that its technology would not work well in its default mode, Snow said in an interview: “Essentially, they are saying their product is broken out of the box.”
He also noted that there was no evidence Amazon was actually guiding police to use the higher threshold, and that even if the company was making those recommendations, there were no guarantees that police were using that standard. The technology allows law enforcement to change its thresholds.
“Amazon is failing to take responsibility for Rekognition and how Rekognition is being deployed in communities,” Snow said, adding that Congress should implement a moratorium on government use of facial recognition tools.
Three Congress members wrote to Bezos on Thursday requesting copies of any internal accuracy or bias assessments that Amazon had performed on Rekognition and a list of police or intelligence agencies that use the software or have been in contact with the company about possible acquisition.
The letter said the “efficacy and impact of the technology are not yet fully understood” and noted that “neither Congress nor state legislatures have passed laws explicitly authorizing the use of facial recognition by law enforcement”.
Studies have found that facial recognition technology misidentifies black people , women and young people at higher rates than older white men .
If police use Amazon’s technology on a large scale, “hundreds of thousands of people stand to be incorrectly identified”, said Cyril, noting that it would impact “people of color and particularly black people who are already discriminated against in our current US system of policing, who are already facing dangerous interactions with police”.
“Police will have a tool to aid that discrimination,” Cyril continued, adding that Amazon was “playing politics with people’s lives”.
An Amazon spokesperson asserted that Rekognition had beneficial applications, such as “preventing human trafficking” and “inhibiting child exploitation”, adding: “We remain excited about how image and video analysis can be a driver for good in the world, including in the public sector and law enforcement.”
Amazon has not disclosed the police agencies that may be using the software and did not respond to questions about its law enforcement partners on Thursday.
