MUN:Image Forgery Localization Based on M3 Encoder and UN Decoder

Abstract

Image forgeries can entirely change the semantic informa tion of an image, and can be used for unscrupulous purposes. In this paper, we propose a novel image forgery localization network named as MUN, which consists of an M3 encoder and a UN decoder. Firstly, the M3 encoder is constructed based on a Multi-scale Max-pooling query module to ex tract Multi-clue forged features. Noiseprint++ is adopted to assist the RGB clue, and its deployment methodology is dis cussed. A Multi-scale Max-pooling Query (MMQ) module is proposed to integrate RGB and noise features. Secondly, a novel UN decoder is proposed to extract hierarchical features from both top-down and bottom-up directions, reconstruct ing both high-level and low-level features at the same time. Thirdly, we formulate an IoU-recalibrated Dynamic Cross Entropy (IoUDCE) loss to dynamically adjust the weights on forged regions according to IoU which can adaptively balance the influence of authentic and forged regions. Last but not least, we propose a data augmentation method, i.e., Deviation Noise Augmentation (DNA), which acquires accessible prior knowledge of RGB distribution to improve the generalization ability. Extensive experiments on publicly available datasets show that MUN outperforms the state-of-the-art works.