Tattoo Recognition Score Card: How Institutions Handled Unethical Biometric Surveillance Dataset

In response to an EFF campaign started last year, roughly a third  of institutions that we believe requested problematic and exploitive data as part of a government automated tattoo recognition challenge deleted the data or reported that they had never received or used it.

EFF has long been concerned with the many problems associated with efforts to use automated tattoo recognition, a form of biometric surveillance similar to face recognition that can use your body art to reveal your identity or personal information about you, such as your political, religious, familial, or cultural affiliations. We  have particular ethical concerns about an effort known as Tatt-C (also known as the Tattoo Recognition Challenge) that was managed by the National Institute of Standards and Technology (NIST) and the Federal Bureau of Investigation. NIST launched this tattoo recognition program in 2014 by creating an “open tattoo database” that institutions could use to test, train, and improve software that could recognize tattoos.

As part of the challenge, participants received a CD-ROM full of images to test their tattoo recognition software. However, this dataset was created with 15,000 images most of which were collected from prisoners, who were unaware of the intended use of the images.

After EFF filed a FOIA lawsuit in 2017 seeking access to NIST and FBI records about Tatt-C, we obtained documents that listed institutions and other parties that had requested the Tatt-C dataset. We then wrote to each entity and demanded that the research institutions, if they had received the Tatt-C dataset, delete the images, initiate an internal review of all research generated by using the dataset, and begin a review of the institution’s policies for training and using biometric data collected from individuals that did not consent to be photographed or to have their images used in such a way.  

As EFF wrote in the letter sent to the recipients of the dataset, there were is also considerable concern that there was no ethical review of the collection procedure that is generally required when conducting research with human subjects: 

“When compiling the Tatt-C dataset and conducting the Tatt-C challenge, NIST researchers failed to follow the Common Rule’s requirements for ethical review of the project. Specifically, the researchers did not seek a Human Subjects determination, which is required before assessing whether research must be reviewed by an independent review board (IRB). According to official reports, ‘the human subject’s determination was not known to be necessary for the proposed use of data.’

NIST discovered the oversight only after the Tatt-C research was conducted. At that stage, Information Technology Lab Director Charles H. Romine took the unusual step of retroactively determining that the research did not involve human subjects.  EFF’s evaluation of the application and determination found multiple red flags, including erroneous claims that the data was sufficiently de-identified from the subjects and downplaying NIST’s role in creating the data set. Many of the images reviewed by EFF contained personally identifying information, including people’s names, faces, and birth dates. Of great concern was the researchers’ omission that the data was collected from prisoners, which generally triggers greater scrutiny.”

Tattoos are also incredibly personal and often contain specific information and identifiers that could be used to track down a person even if their face and identity have been obscured. For example, even though the names of the inmates were removed from the Tatt-C metadata, the tattoos themselves sometimes contained personal information, such as life-like depictions of loved ones, names, and birth dates that all remain viewable to researchers.

Nearly all the entities that responded did confirm the data had been deleted. However, at least one university was still conducting research with the dataset five years later. The University of Campinas (UNICAMP) School of Engineering Computer Engineering in Brazil sent a letter that stated researchers are only required to seek ethics review for human data collected within Brazil. UNICAMP said that Prof. Léo Pini Magalhães would continue to use the tattoo images through the end of year and then would delete them. The university also refused to acknowledge that images contained personal information. UNICAMP also said that the professor is grabbing images of tattoos from the web, a practice that has increasingly come under fire from Congress in light of the Clearview AI face recognition scandal.

UNICAMP’s response illustrates the problem with Tatt-C: once the flawed dataset was shared, the government lost the ability to control how the sensitive images were used. Several institutions that did not respond to our letter are based abroad, including France, South Africa, Thailand, Mexico, and Peru. Therefore, it would be difficult, if not impossible, for U.S. authorities to hold these entities accountable if they misuse the information.

The table below details the institutions that NIST identified in government records as having requested the Tatt-C dataset and their responses to EFF’s letter.

For more information about tattoo recognition technology, check out EFF’s Street-Level Surveillance guide.