Tattoo Recognition Score Card: How Institutions Handled Unethical Biometric Surveillance Dataset

Fight Censorship, Share This Post!

In response to an EFF campaign started last year, roughly a third  of institutions that we believe requested problematic and exploitive data as part of a government automated tattoo recognition challenge deleted the data or reported that they had never received or used it.

EFF has long been concerned with the many problems associated with efforts to use automated tattoo recognition, a form of biometric surveillance similar to face recognition that can use your body art to reveal your identity or personal information about you, such as your political, religious, familial, or cultural affiliations. We  have particular ethical concerns about an effort known as Tatt-C (also known as the Tattoo Recognition Challenge) that was managed by the National Institute of Standards and Technology (NIST) and the Federal Bureau of Investigation. NIST launched this tattoo recognition program in 2014 by creating an “open tattoo database” that institutions could use to test, train, and improve software that could recognize tattoos.

As part of the challenge, participants received a CD-ROM full of images to test their tattoo recognition software. However, this dataset was created with 15,000 images most of which were collected from prisoners, who were unaware of the intended use of the images.

After EFF filed a FOIA lawsuit in 2017 seeking access to NIST and FBI records about Tatt-C, we obtained documents that listed institutions and other parties that had requested the Tatt-C dataset. We then wrote to each entity and demanded that the research institutions, if they had received the Tatt-C dataset, delete the images, initiate an internal review of all research generated by using the dataset, and begin a review of the institution’s policies for training and using biometric data collected from individuals that did not consent to be photographed or to have their images used in such a way.  

As EFF wrote in the letter sent to the recipients of the dataset, there were is also considerable concern that there was no ethical review of the collection procedure that is generally required when conducting research with human subjects: 

“When compiling the Tatt-C dataset and conducting the Tatt-C challenge, NIST researchers failed to follow the Common Rule’s requirements for ethical review of the project. Specifically, the researchers did not seek a Human Subjects determination, which is required before assessing whether research must be reviewed by an independent review board (IRB). According to official reports, ‘the human subject’s determination was not known to be necessary for the proposed use of data.’

NIST discovered the oversight only after the Tatt-C research was conducted. At that stage, Information Technology Lab Director Charles H. Romine took the unusual step of retroactively determining that the research did not involve human subjects.  EFF’s evaluation of the application and determination found multiple red flags, including erroneous claims that the data was sufficiently de-identified from the subjects and downplaying NIST’s role in creating the data set. Many of the images reviewed by EFF contained personally identifying information, including people’s names, faces, and birth dates. Of great concern was the researchers’ omission that the data was collected from prisoners, which generally triggers greater scrutiny.”

Tattoos are also incredibly personal and often contain specific information and identifiers that could be used to track down a person even if their face and identity have been obscured. For example, even though the names of the inmates were removed from the Tatt-C metadata, the tattoos themselves sometimes contained personal information, such as life-like depictions of loved ones, names, and birth dates that all remain viewable to researchers.

Nearly all the entities that responded did confirm the data had been deleted. However, at least one university was still conducting research with the dataset five years later. The University of Campinas (UNICAMP) School of Engineering Computer Engineering in Brazil sent a letter that stated researchers are only required to seek ethics review for human data collected within Brazil. UNICAMP said that Prof. Léo Pini Magalhães would continue to use the tattoo images through the end of year and then would delete them. The university also refused to acknowledge that images contained personal information. UNICAMP also said that the professor is grabbing images of tattoos from the web, a practice that has increasingly come under fire from Congress in light of the Clearview AI face recognition scandal.

UNICAMP’s response illustrates the problem with Tatt-C: once the flawed dataset was shared, the government lost the ability to control how the sensitive images were used. Several institutions that did not respond to our letter are based abroad, including France, South Africa, Thailand, Mexico, and Peru. Therefore, it would be difficult, if not impossible, for U.S. authorities to hold these entities accountable if they misuse the information.

The table below details the institutions that NIST identified in government records as having requested the Tatt-C dataset and their responses to EFF’s letter.

Institute of Legal Medicine and Forensic Sciences/Instituto de Medicina Legal y Ciencias Forenses, Peru

No response to our request 

Bureau of Prisons

No response to our request 

LTU Technologies, France

No longer retains any of the Tatt-C data

Mahidol University International College, Thailand 

No response to our request 

John Hopkins University Applied Physics Laboratory

No response to our request 

Commissariat à l’Énergie Atomique

No response to our request 

Purdue University, School of Electrical and Computer Engineering

After the university deemed that the dataset did not comprise human subject research, tests were conducted and reported back to NIST. The data was subsequently deleted.

Noblis

No response to our request 

Nanyang Technological University, ForSe Lab/Cybersecurity Lab, Singapore

No response to our request 

University of Queensland

The dataset was used to test algorithms and then was deleted.  

Progeny Systems

No response to our request 

West Virginia University – Multispectral Imagery Lab 

The dataset was received, used, and destroyed. WVU has initiated a new review of the resulting research. 

Fraunhofer Institute of Optronics, Systems Technology and Image Exploitation, Germany 

The dataset was received, used, and destroyed. 

Flex Analytics

The dataset was received, used, and destroyed. The company is now defunct, however the former owner found three remaining images on his computer and deleted them after receiving our letter.  

Kitware Inc.

The dataset was received and destroyed. 

University of Campinas

Has refused to delete the dataset and will continue to conduct research with it until the “end of the current year.” 

Compass Technical Consulting 

No response to our request 

MITRE Corporation

No response to our request 

United Technologies Research Centre Ireland, Ireland

The dataset was received and destroyed, but no tests were ever conducted using it.

Council for Scientific and Industrial Research, South Africa

No response to our request 

Instituto Politecnico Nacional, Mexico

No response to our request 
Center for Data Innovation

“You can count us as a non-response to this inquiry.”

Neurotechnology, Lithuania 

No response to our request 
St. Louis Crime Strategies Unit, St. Louis Circuit Attorney’s Office

No response to our request 

Carnegie Mellon University, Dept. of Electrical and Computer Engineering, CyLab

CMU had no records of ever having requested or participated in Tatt-C.

Stanford University, Department of Computer Science

Stanford could find no remaining evidence of the dataset and the Research Compliance Office (RCO) was informed of our letter. 

For more information about tattoo recognition technology, check out EFF’s Street-Level Surveillance guide.


Fight Censorship, Share This Post!

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.