SODA-SR (CVPR2024)

This repository is an official PyTorch implementation of the paper Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer.

Yuang Ai^1,2, Xiaoqiang Zhou^1,3, Huaibo Huang^1,2, Lei Zhang^4,5, Ran He^1,2,6

¹MAIS & CRIPAC, Institute of Automation, Chinese Academy of Sciences
²School of Artificial Intelligence, University of Chinese Academy of Sciences
³University of Science and Technology of China ⁴OPPO Research Institute
⁵The Hong Kong Polytechnic University ⁶ShanghaiTech University

Abstract

Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image SuperResolution (SR) by accessing both the source and target data. Considering privacy policies or transmission restrictions of source data in practical scenarios, we propose a SOurce-free Domain Adaptation framework for image SR (SODA-SR) to address this issue, i.e., adapt a source-trained model to a target domain with only unlabeled target data. SODA-SR leverages the source-trained model to generate refined pseudo-labels for teacher-student learning. To better utilize pseudo-labels, we propose a novel wavelet-based augmentation method, named Wavelet Augmentation Transformer (WAT), which can be flexibly incorporated with existing networks, to implicitly produce useful augmented data. WAT learns low-frequency information of varying levels across diverse samples, which is aggregated efficiently via deformable attention. Furthermore, an uncertainty-aware self-training mechanism is proposed to improve the accuracy of pseudo-labels, with inaccurate predictions being rectified by uncertainty estimation. To acquire better SR results and avoid overfitting pseudo-labels, several regularization losses are proposed to constrain target LR and SR images in the frequency domain. Experiments show that without accessing source data, SODA-SR outperforms state-of-the-art UDA methods in both synthetic→real and real→real adaptation settings, and is not constrained by specific network architectures.

Results

Quantitative Results

Visual Comparisons

Evaluation

You can download the pre-trained models from the Huggingface.

 python3 main_test_sodasr.py --task classical_sr --scale 4 \
 --model_path ... --folder_lq ... --folder_gt ... --cuda 0 --save_name ...

Acknowledgement

Our code is built upon KAIR, Deformable-DETR and CDC. We thank the authors for their awesome work.

Contact

If you have any questions, please feel free to contact with me at [email protected].

Citation

@inproceedings{ai2024uncertainty,
  title={Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer},
  author={Ai, Yuang and Zhou, Xiaoqiang and Huang, Huaibo and Zhang, Lei and He, Ran},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={8142--8152},
  year={2024}
}