Home > Electrical/Electronics > Computer (I.T) > Hackers Compete To Fool Facial Recognition

Hackers Compete To Fool Facial Recognition

Facial-recognition technology is becoming increasingly prevalent in our lives, but its also highly vulnerable to attack. That’s why a group of researchers is appealing to hackers to take part in a new competition designed to expose facial recognition’s flaws and raise awareness of the potential risks.

The machine-learning security evasion competition has been a regular fixture of the Def Con hacking conference since 2019. Early iterations challenged researchers to sneak around the defenses of machine-learning-based malware detection systems. In 2021, organizers added a new track designed to uncover flaws in computer-vision models that use visual cues in order to detect phishing websites.

But this year the competition will also pit hackers against facial-recognition systems, challenging them to modify photographs of celebrities so that a machine-learning model misidentifies them. Zoltan Balazs, head of vulnerability research lab at software company Cujo AI, one of the organizers of the competition, says the addition was in response to the rapid expansion in the use of facial recognition and the seemingly lax approach to security among many vendors.

“We really hope that one of the conclusions of our competition will be that very special care should be taken into consideration whenever people are implementing facial-recognition systems,” he says. “Because they are not perfect. And the consequences can be bad.”

The facial-recognition challenge has been designed by AI security company Adversa AI, which knows exactly how vulnerable these kinds of machine-learning models are. The company regularly carries out “red teaming” exercises—in which it is hired by other firms to test their machine-learning systems for security flaws.

“It’s very easy and fast to do for attackers with enough motivation,” says Neelou. “Our engagements show that some of the best facial-recognition vendors demonstrate little to no security against adversarial input modifications.”

READ ALSO  Tech Firm Develops Advanced Bulk Messaging Platform

The organizers hope that their competition will highlight the current concerns around facial recognition. The winner is also required to publish its techniques, which should help the industry close potential gaps. The contest opened on 12 August and will run until 23 September. Entrants are given a set of 10 headshots of well-known celebrities and online access to a facial-recognition model that has been trained to recognize them.

Attackers are instructed to subtly alter the images so that the model misidentifies them. The goal is to trick the system into identifying each celebrity as each of the other celebrities, which means creating nine modified images for each headshot. These then need to be submitted to the competition organizers who will assess the effectiveness of the deception.

The most important criteria for judging each image is the confidence with which the model accepts the new identity. This is judged by the probability score the model gives the image, which ranges from 0 to 1. Images will also be rate on the “stealthiness” of the modifications—in other words, how difficult they are to spot. This will be judged based on the level structural similarity between the original and doctored image but will only be used as a tiebreaker if teams are level on confidence scores.

READ ALSO  Digital Switchover: Federal Govt Inaugurates Task Force

How entrants edit the images is up to them. While it is possible to modify them by hand, Neelou says that attacks on a machine-learning system typically use automated processes. In most cases, hackers won’t know anything about the model they are targeting and so will submit hundreds or thousands of images and use feedback from the model to iterate on their alterations. If they can glean some information about the model though, their job becomes even easier as they can tailor their approach to its particularities.

“There are various strategies to target different internal characteristics of neural networks,” says Neelou. “One attack technique may be the best against one network and the worst against another. That’s why there is no one-fits-all attack, but given enough time every AI system can be hacked.”

But what the industry really needs is a fundamental mind shift, so that security is taken into account earlier in the development process rather than being tagged on at the end, says Neelou. “The main reason AI is vulnerable to attacks is that AI is never built with security in mind,” he says, “As with many technologies, security is an afterthought.”

https://spectrum.ieee.org/media-library/a-photo-of-brad-pitt-on-the-left-and-an-ai-image-of-brad-pitt-on-the-right.jpg?id=31289379&width=1200&height=598

Source: spectrum

Total Views: 182 ,
0
0

Leave a Reply

Your email address will not be published. Required fields are marked *