SIDBench: A Python Framework for Reliably Assessing Synthetic Image Detection Methods
3rd ACM International Workshop on Multimedia AI against Disinformation(2024)
Abstract
The generative AI technology offers an increasing variety of tools for
generating entirely synthetic images that are increasingly indistinguishable
from real ones. Unlike methods that alter portions of an image, the creation of
completely synthetic images presents a unique challenge and several Synthetic
Image Detection (SID) methods have recently appeared to tackle it. Yet, there
is often a large gap between experimental results on benchmark datasets and the
performance of methods in the wild. To better address the evaluation needs of
SID and help close this gap, this paper introduces a benchmarking framework
that integrates several state-of-the-art SID models. Our selection of
integrated models was based on the utilization of varied input features, and
different network architectures, aiming to encompass a broad spectrum of
techniques. The framework leverages recent datasets with a diverse set of
generative models, high level of photo-realism and resolution, reflecting the
rapid improvements in image synthesis technology. Additionally, the framework
enables the study of how image transformations, common in assets shared online,
such as JPEG compression, affect detection performance. SIDBench is available
on https://github.com/mever-team/sidbench and is designed in a modular manner
to enable easy inclusion of new datasets and SID models.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined