Facial recognition is becoming more and more common, but ask anyone how to avoid it and they’ll say: easy, just wear a mask. In the future, though, that might not be enough. Facial recognition technology is under development that’s capable of identifying someone even if their face is covered up — and it could mean that staying anonymous in public will be harder than ever before.
The topic was raised this week after research published on the preprint server arXiv describing just such a system was shared in a popular AI newsletter. Using deep learning and a dataset of pictures of people wearing various disguises, researchers were able to train a neural network that could potentially identify masked faces with some reliability. Academic and sociologist Zeynep Tufekci shared the work on Twitter, noting that such technology could become a tool of oppression, with authoritarian states using it to identify anonymous protestors and stifle dissent.
The paper itself needs to be taken with a pinch of salt, though. Its results were far less accurate than industry-level standards (when someone was wearing a cap, sunglasses, and a scarf, for example, the system could only identify them 55 percent of the time); it used a small dataset; and experts in the field have criticized its methodology.
“It doesn’t strike me as a particularly convincing paper,” Patrik Huber, a researcher at the University of Surrey who specializes in face tracking and analysis, told The Verge. He pointed out that the system doesn’t actually match disguised faces to mugshots or portraits, but instead used something called “facial keypoints” (the distances between facial features like eyes, noses, lips, etc) as a proxy for someone’s identity.
But although the paper has its flaws, the challenge of recognizing people when their faces are covered is one that plenty of teams are working on — and making quick progress.
Facebook, for example, has trained neural networks that can recognize people based on characteristics like hair, body shape, and posture. Facial recognition systems that work on portions of the face have also been developed (although, again; not ready for commercial use). And there are other, more exotic methods to identify people. AI-powered gait analysis, for example, can recognize individuals with a high degree of accuracy, and even works with low-resolution footage — the sort you might get from a CCTV camera.
One system for identifying masked individuals developed at the University of Basel in Switzerland recreates a 3D model of the target’s face based on what it can see. Bernhard Egger, one of the scientists behind the work, told The Verge that he expected “lots of development” in this area in the near future, but thought that there would always be ways to fool the machine. “Maybe machines will outperform humans on very specific tasks with partial occlusions,” said Egger. “But, I believe, it will still be possible to not be recognized if you want to avoid this.”