Facial biometrics are widely used to reliably and conveniently recognize people in photos, in videos, or from real-time webcam streams. It is therefore of fundamental importance to detect synthetic faces in images in order to reduce the vulnerability of biometrics-based security systems. Furthermore, manipulated images of faces can be intentionally shared on social media to spread fake news related to the targeted individual. This paper shows how fake face recognition models may mainly rely on the information contained in the background when dealing with generated faces, thus reducing their effectiveness. Specifically, a classifier is trained to separate fake images from real ones, using their representation in a latent space. Subsequently, the faces are segmented and the background removed, and the detection procedure is performed again, observing a significant drop in classification accuracy. Finally, an explainability tool (SHAP) is used to highlight the salient areas of the image, showing that the background and face contours crucially influence the classifier decision.

Tanfoni, M., Ceroni, E.G., Marziali, S., Pancino, N., Maggini, M., Bianchini, M. (2024). Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images. ELECTRONICS, 13(16) [10.3390/electronics13163161].

Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images

Tanfoni, Marco
;
Ceroni, Elia Giuseppe
;
Marziali, Sara;Pancino, Niccolò;Maggini, Marco;Bianchini, Monica
2024-01-01

Abstract

Facial biometrics are widely used to reliably and conveniently recognize people in photos, in videos, or from real-time webcam streams. It is therefore of fundamental importance to detect synthetic faces in images in order to reduce the vulnerability of biometrics-based security systems. Furthermore, manipulated images of faces can be intentionally shared on social media to spread fake news related to the targeted individual. This paper shows how fake face recognition models may mainly rely on the information contained in the background when dealing with generated faces, thus reducing their effectiveness. Specifically, a classifier is trained to separate fake images from real ones, using their representation in a latent space. Subsequently, the faces are segmented and the background removed, and the detection procedure is performed again, observing a significant drop in classification accuracy. Finally, an explainability tool (SHAP) is used to highlight the salient areas of the image, showing that the background and face contours crucially influence the classifier decision.
2024
Tanfoni, M., Ceroni, E.G., Marziali, S., Pancino, N., Maggini, M., Bianchini, M. (2024). Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images. ELECTRONICS, 13(16) [10.3390/electronics13163161].
File in questo prodotto:
File Dimensione Formato  
electronics-13-03161-v2.pdf

accesso aperto

Tipologia: PDF editoriale
Licenza: Creative commons
Dimensione 7.09 MB
Formato Adobe PDF
7.09 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11365/1269074