Facial Recognition and Masks

Most people have been wearing masks since March of 2020, when the COVID pandemic first began. Masks were one of the first lines of defense against virus spread but had the unintended impact of getting in the way of facial recognition technology, from lower stakes day-to-day use of facial recognition for unlocking cell phones to higher-stakes security screening software at government facilities.

In response, many facial recognition software companies have looked at adapting their software to focus on recognition based on the eye region alone. Having software that can recognize an individual based on such a small portion of the face has raised concerns – both from a programmatic standpoint and from a privacy perspective.

“For many individuals and privacy advocates, it may be a comforting thought that a mask could offer some measure of invisibility from computerized surveillance,” writes Rachel Metz, for CNN Business. “But for facial recognition businesses, it poses a unique challenge at a moment when the technology appears to be in even greater demand.”

The benefit of facial recognition during a pandemic is it allows for distanced and contact-free screening. Privacy advocates worry, however, about the level of racial bias and potential for abuse of software that is using such a limited amount of data to identify individuals.

Generally speaking, facial recognition software does not do a great job of correctly identifying faces when a large portion of the face is covered by a mask. Though some software versions are coded to handle partly concealed face, the vast majority do not do well went put to the test.

Using pre-COVID-era software, in July 2020, federal researchers at the National Institute of Standards and Technology (NIST) tested the existing algorithms on how they recognized masked faces. The results showed that even the most accurate of the algorithms failed to match faces correctly between 5% and 50% of the time.

Facial recognition software companies turned to different models for how their algorithms functioned, including focusing analysis on the visible portions of the face instead of detecting items on the face, such as masks or sunglasses.

Focusing on the eye area has other benefits, says Marios Savvides, a professor at Carnegie Mellon University who studies biometric identification, in an interview with CNN.

“Savvides said the eye and eyebrow area (which is often referred to as the periocular region) is the part of the face that changes the least as you age, even if you gain weight,” writes Metz. “This means it's likely to look quite similar in different images of the same person, even if other parts of your face (your lips, for instance) grew or shrank.”

In the time since the pandemic began, updates to these systems have created algorithms that are far more adept at recognizing faces that are covered by a mask. This is viewed as a success for those in the security world while also raising additional concerns about personal privacy.

By January of 2021, the Department of Homeland Security (DHS) that a “controlled scenario test” showed that, “With masks, median system performance demonstrated a ~77% identification rate, with the best-performing system correctly identifying individuals ~96% of the time.”

The goal of the test was to reduce the need for individuals to have to remove their masks while at the airport, thus reducing the potential for spread of the COVID virus.

But others see a darker side, finding the ability of technology to recognize a mostly covered face as intrusive and even “too creepy,” as shared by the British advocacy group Privacy International. The group has even created a mask they “view as a symbol of resistance against the growing use of mass facial recognition.”

Want to learn about cybersecurity? View the full list of bachelor’s, master’s and doctorate degrees in cyber and information security. Many courses are available both on campus and online. To learn more about Capitol Tech’s degree programs, contact admissions@captechu.edu.