Foodstuffs North Island’s facial recognition trial: do the numbers add up?

In announcing the preliminary results of its trial, FSNI claims facial recognition technology (FRT) has avoided an estimated 130 assault and abuse incidents in a six-month period in the 25 trial stores.

However, analysis of retail crime statistics produced by FSNI raises questions about these findings.

In the six months from October 2023 to March 2024, FSNI reported 130 incidents of assault or abuse across all 320 of their North Island stores.

Now, however, Foodstuffs are reporting that their trial of facial recognition has led to the avoidance of 130 incidents of assault and abuse in a six-month period in just 25 trial stores.

This is equivalent to the number of reported assault and abuse incidents across all their North Island stores in a six-month period.

Consumer NZ’s concerns with Foodstuffs North Island’s trial

Consumer NZ is concerned about privacy, accuracy and racial discrimination issues arising from the use of facial recognition in public spaces like supermarkets.

We are also troubled about the design of FSNI’s trial, which has gathered no information on the ethnicity of individuals mistakenly identified by the technology, and the way statistics on mistaken identity have been reported by the retailer.

We are concerned FSNI have not been transparent about their retail crime statistics in the context of the facial recognition trial.

We asked FSNI to share their retail crime statistics, as well as how many incidents they expected to take place in the 25 stores that trialled facial recognition. FSNI refused our request for data and did not answer our questions regarding how many events they’d expected in their trial stores.

While the results of the trial are being independently assessed, FSNI continue to use the technology in their 25 stores.

Foodstuffs North Island’s facial recognition trial

FSNI have been using facial recognition in their stores since at least 2018, and in November 2022, a Consumer NZ investigation revealed that they were using the technology in 29 stores across the North Island.

FSNI frame their use of facial recognition in the context of safety rather than theft, saying “everyone in New Zealand has the right to a safe workplace and a safe place to shop.”

Speaking in the wake of our investigation, Deputy Privacy Commissioner Liz MacPherson told Radio New Zealand, “we did ask [FSNI] whether they had actually been recording the effectiveness in keeping people safe. It appears they haven’t … We would not want to see them roll this out nationwide unless they’d done a trial to test the effectiveness of it in actually achieving safety.”

This means that if FSNI wanted to roll out FRT nationwide with the Office of the Privacy Commissioner’s blessing, they needed to demonstrate in a trial that the technology could increase safety.

Five cases of mistaken identity

FSNI reported that the 25 trial stores generated “1,747 alerts, with one well-reported case of misidentification and four cases where the identity of a customer was established after we spoke to them.”

However, the “four cases” FSNI referred to were also incidents where the identity of the customer was established after being spoken to by staff following an incorrect match by the FRT. That makes five cases of mistaken identity, not one.

And the rate of mistaken identity could be much higher.

FSNI report that out of 1747 alerts, they had one case of mistaken identity. About a 1 in 1750 chance of misidentification. 

But FSNI say that they only spoke to half of the individuals that generated an alert; we do not know how many of those matches may have turned out to be incorrect when individuals were approached by staff.

Of those who were approached by staff following a match by the facial recognition system – around 875 people – in five of these cases, they got the wrong person.

This means for 875 approaches made by staff, five were cases of mistaken identity. This is a 1 in 175 chance of misidentification. 10 times higher than in FSNI’s interpretation.

Consumer NZ is concerned that FSNI may be exaggerating facial recognition’s impact in reducing instances of assault and abuse.

FSNI announced the start of the trial in February 2024, with the Privacy Commissioner Michael Webster stating that he would use his inquiry powers to “keep a close eye” on the trial.

Racial bias

Speaking at the start of the trial, Webster laid out his concerns about the accuracy and bias of facial recognition.

“I am particularly worried about what this means for Māori, Pasifika, Indian and Asian shoppers, especially as the software is not trained on New Zealand’s population … I don’t want to see people incorrectly banned from their local supermarket and falsely accused,” Webster said.

In the matter of the widely reported case of mistaken identity, a Māori woman was trespassed from a New World Westend store in Rotorua, despite showing identification.

We do not know the ethnicity of the other four individuals FSNI staff misidentified because FSNI say that no ethnicity data was collected for privacy reasons.

Dr Karaitiana Taiuru, a Māori data and AI ethicist, can see why asking for ethnicity information puts FSNI in a tricky position but believes that they should have gathered this data.

“I see the issue. If I were in the supermarket and someone asked me if I was Māori or not, I’d be offended, right? But I think there’s ways around it without being offensive. They could have taken notes that said, ‘the person obviously wasn’t a Caucasian person’. They didn’t exercise that judgement,” he said.

This is particularly troubling because concerns about the discriminatory nature of FRT are so well founded, according to Dr Taiuru.

“We know that FRT can be biased because we know it hasn’t been trained on Māori and Pasifika, and it doesn’t seem like FSNI have done anything to try and address those issues.

“It certainly seems like they have no regard for protecting people of colour in this research by not documenting it. I feel like, if they had any sort of corporate responsibility, they would have come up with some sort of legal and moral way of finding out the misidentified people’s ethnicities,” he said.

What happens next

The Privacy Commissioner is now evaluating the results of FSNI’s trial of FRT. His office is reviewing its own data alongside the results from the independent evaluator who designed and oversaw the trial.

The Commissioner expects to announce his findings before the end of this year.

FSNI are yet to receive the final results of the trial from their independent evaluator and have gathered no evidence on whether their use of facial recognition is racially discriminative. Despite this, they are continuing to use the technology in 25 stores.

Foodstuffs North Island’s response

In response to the claims made in this article, FSNI explained that retail crime takes a serious toll on staff and customers as they work and shop.

A spokesperson for FSNI said, “Facial recognition is one tool we can use to help.  Our trial and independent review have shown it’s a tool that can improve safety and respect privacy and that’s something we feel accountable to do. It’s not a complete answer, but for every incident it helps prevent, it makes a huge difference to the lives of many people. This technology helps our teams identify repeat offenders when they enter stores again, allowing us to step in early and reduce the risk to our staff and customers. Keeping our people and customers safe while they work or shop with us is the least they should expect.”