Facial recognition at 29 Foodstuffs North Island stores

Foodstuffs North Island is the only major retailer in New Zealand using the technology. We surveyed the privacy policies of national retailers, including Bunnings, Mitre 10 and The Warehouse, for mention of biometrics or FRT. Mitre 10 admitted it had trialled the technology in one store, but that the store in question has not used it for two years.

Emma Wooster, Foodstuffs NZ’s head of public relations, said: “Facial recognition technology is one of the only tools we’ve identified that could help us to proactively target and reduce theft, burglary, robbery, assault and other aggressive, violent or threatening behaviour by repeat offenders. Facial recognition technology is only used for this specific and limited purpose.”

Wooster said the individual ownership and operation of New World, Pak’nSave and Four Square stores mean that security measures vary.

This is not the first time Foodstuffs North Island’s use of FRT has come under scrutiny. In 2018 the Otago Daily Times revealed that Foodstuffs had quietly introduced the technology. And in August 2020, the Papakura branch of New World made headlines by asking customers to remove their masks so they could be better scanned by FRT.

Wooster went on to say: “Supermarkets are on the front line of the rising trend of retail crime, and the sad reality is our teams are dealing with the consequences of this.”

That Foodstuffs North Island is the only major retailer using the technology does raise questions, however. If every other major retailer in New Zealand can get by without using FRT, why can’t they?

What is facial recognition technology?

FRT involves the identification of an individual based on an analysis of their facial features. This technology is powered by artificial intelligence programs capable of identifying and mapping facial features to create a ‘faceprint’ before comparing this faceprint to a database to find a matching individual.

Marcin Betkier, the chair of the Privacy Foundation, describes FRT as “remote biometrics, which means it can remotely identify the person and read their biometric information, without that person’s engagement or awareness.

“Up until now, any sort of biometric collection required some sort of interaction with people and their awareness,” Betkier said.

But in the age of remote biometrics, FRT-enabled cameras can turn our physiological attributes into data points from which we are instantly identifiable. And our data can be gathered and analysed as we walk through a city, or when we upload a picture to social media, without our knowledge or our permission.

In a retail context, it allows a business to scan everyone who enters the store and instantly identify individuals with a history of theft or violent or aggressive behaviour toward staff or customers. Security staff can ask that customer to leave, or closely monitor them in order to prevent crime.

How is it used in New Zealand?

Last year, our colleagues at Choice in Australia identified the use of FRT by Kmart, Bunnings and The Good Guys, with all three retailers pausing their use of the technology following Choice’s investigation. Our own investigation shows that while the technology is in its infancy in retail in New Zealand, there are a number of ways in which you are likely to have encountered the technology.

A landmark report by the Law Foundation of New Zealand published in 2020 identified a range of government uses of FRT. The Department of Internal Affairs (DIA) has used the technology in producing passports since 2012, while part of the process of applying for a RealMe credential is submitting a facial image. The Immigration Act 2009 empowered Immigration New Zealand (INZ) to collect biometric information including photographs, while SmartGates that use FRT are now commonplace at airport security.

Consumer applications of the technology are also widespread, and if you have an iPhone or Android device, there’s a good chance that you use FRT to unlock your phone and to log in to the apps on it.

Other attempts to implement the technology here show that the technology has got off to a fairly bumpy start. In 2020, the New Zealand Police contacted Clearview AI for a trial of its technology. Clearview AI is an American facial recognition company which claims to hold the “largest known database of 30+ billion facial images sourced from public-only web sources including news media, mugshot websites, public social media and other sources.”

However, the trial was a spectacular failure, generating only one match due to the lack of New Zealanders in Clearview’s dataset. It also drew criticism from the Office of the Privacy Commissioner (OPC).

What’s wrong with FRT?

“Passwords are the bane of our lives a lot of the time, but at least we can change them easily,” says Frith Tweedie, a principal privacy consultant at Simply Privacy.

“If your facial image or your fingerprints have been collected, then short of extensive plastic surgery, you can’t change those things. That makes it really sensitive information that needs to be protected.”

The value of biometric data that drives such hunger among businesses and governments is also likely to make it a target of cybercrime, and the storage of our biometric data by a range of organisations could represent a significant threat to privacy. No business and no-one is immune from this threat.

In October 2022, Australian health insurance provider Medibank revealed it had been hacked, resulting in the theft of 9.7 million people’s health information – with even the Prime Minister Anthony Albanese fearful that his data has been hacked. In November, the hackers began leaking lists of individuals who have had abortions, mental health issues or have accessed drug and alcohol addiction services.

With our facial data increasingly being used to access our smartphones, cross borders, and identify whether or not we might be criminals, breaches, misuse of data, or algorithmic bias could compromise our digital identities irreparably and permanently.

FRT means that databases of facial images have become a big business. The business model of Clearview AI, the company that landed NZ Police in hot water, relies on scraping individuals’ images from across the internet so that law enforcement (and potentially businesses) can match an individual’s face with their name and identifying details in real time.

There are also significant issues with the use of FRT on individuals in different ethnic groups. We remain in the earliest phases of the AI revolution but already there are countless examples of discrimination by algorithms, and FRT is no different.

“There are questions about the databases used to make comparisons, but what database are they using for comparison?” asks Betkier. “What does this mean for the accuracy of the system, and how accurate is it in comparing different minorities in our society?”

This is a real problem, according to Tweedie.

“If you’re using a third-party tool out of the US, what is that training data made up of? If it’s from a US population, it’s likely to struggle with recognising Māori and Pasifika people.”

In a 2021 report authored by Dr Andrew Chen and Dr Nessa Lynch in the wake of NZ Police’s Clearview AI trial, questions are raised about the role of facial tattoos in disrupting FRT systems. Far from making Māori or Pasifika populations immune to FRT, indigenous ethics commentator Karaitiana Taiuru expressed concern that these groups could be subjected to an increase in false arrests.

The slippery slope

At present, the use of FRT in New Zealand is largely limited to national security and crime prevention. But there is a wide range of other applications, some already being used in other countries, which are far less palatable.

“Technology has a propensity for ‘function creep’, where technological systems implemented for one goal can be used for another goal after they’ve been implemented,” Betkier said.

And this threat is real, with the companies that provide FRT services advertising a range of applications that focus less on security and more on profit.

NEC, a Japanese multinational ICT company, said: “In response to booming demand for varied biometrics, NEC has expanded its traditional range of face recognition applications from law-enforcement and security provision to new areas, and now boasts more than 1000 active systems in over 70 countries and regions spanning police department, immigration control agency, national ID, banking, entertainment, stadium, conference venue systems, and many more.”

NEC points to AdMov, a technology start-up in the Philippines which installed tablets with facial recognition technology in taxis. Based on an individual’s appearance and mood, the software would select the most appropriate advertisements to display. The technology can take things one step further, tracking eyeball movements in order to assess whether an individual has become disengaged.

In marketing its Welcome Kiosk, Sharp (another Japanese multinational) said: “Imagine being able to instantly identify all of your customers as they walk in the door, or being able to track employee productivity with real-time data. The possibilities are endless.”

How are you feeling about FRT now?

Appropriate use

There is no easy answer to the question of whether, and when, FRT can be deployed appropriately.

Betkier sees valid uses when FRT can surpass human performance, and when that use is proportionate.

“I think it can be a useful thing. Imagine that there is a person employed to recognise people’s faces with photographs at border control. That person may be sitting there for eight hours and is tired of doing the same thing. The job of that person could be automated, and done with more accuracy, by a machine.”

Despite being a staunch privacy advocate, Frith Tweedie sees the quandary faced by retailers.

“A lot of retailers are experiencing increasing levels of theft, violence and aggression in their stores, so it’s often quite a tricky balancing exercise. How do they keep their staff and customers safe? And how can they stop this stuff, because ultimately it increases costs for customers.”

But she says transparency is important.

“People need to know if they might be picked up by facial recognition when they walk into a shop so clear and prominent notices in stores are critical,” Tweedie said.

“Retailers also need to ensure information collected for safety and security purposes is only used for that purpose, and not for something else like marketing. Data must be held securely and not kept indefinitely.”

This still might not be enough for Betkier, however. He said the nature of the service should influence the way in which FRT is used, or whether it is used at all.

“It depends on the service. We need to eat, most of us need some sort of bank account or need energy in our homes. This shouldn’t be conditioned on providing biometrics because everyone needs these services.

“If there is a fancy bar, or a boutique shop where you can go and they want to use facial recognition then fine, you can go to a different bar.”

Starting the conversation

It’s difficult to gauge how consumers in New Zealand feel about the use of facial recognition by retailers because no-one has asked them. Research conducted by the Office of the Privacy Commissioner in April 2022 showed 49% of adults in New Zealand – and 51% of Māori adults – were concerned by the use of facial recognition in public spaces.

In Australia, 76% of people agree that “regulation is needed to protect consumers from harms caused by facial recognition use in retail settings”.

For Tweedie, while existing privacy laws can manage FRT to some extent, further regulation is likely.

“We need to make sure that we have the right frameworks and rules in place to try and manage the use of this technology safely. Other jurisdictions are grappling with this too, with certain states and cities in the US banning FRT.

“There’s also a big debate in the EU at the moment around the use of facial recognition by police in public places and whether it should be banned outright.”

In Australia, new laws have been proposed by the Human Technology Institute (HTI) at the University of Technology Sydney. This new regulation would introduce different rules for different uses of FRT, treating the use of the technology in retail as ‘high risk’ and prohibiting its use.

Missing so far from these debates in Aotearoa, however, is the opinion of millions of New Zealanders that could be subject to FRT when buying their groceries.

Both Tweedie and Betkier believe we need to have a public discussion about FRT before it is widely rolled out.

This dialogue has begun – albeit after the technology has been adopted – with the OPC carrying out a public consultation exercise on the use of biometrics, including FRT, in August 2022.

The consultation received more than 100 responses from individuals, businesses, government departments and advocacy groups, including Consumer. We support further privacy regulation of biometrics in Aotearoa.

But a public consultation is not a national conversation. We need to talk about FRT.