RaindropClarity (ECCV'2024)

Introduction

Raindrop Clarity: A Dual-Focused Dataset for Day and Night Raindrop Removal
European Conference on Computer Vision (ECCV'2024)

[Poster] [Slides] [Video]

Abstract

Existing raindrop removal datasets have two shortcomings. First, they consist of images captured by cameras with a focus on the background, leading to the presence of blurry raindrops. To our knowledge, none of these datasets include images where the focus is specifically on raindrops, which results in a blurry background. Second, these datasets predominantly consist of daytime images, thereby lacking nighttime raindrop scenarios. Consequently, algorithms trained on these datasets may struggle to perform effectively in raindrop-focused or nighttime scenarios. The absence of datasets specifically designed for raindrop-focused and nighttime raindrops constrains research in this area. In this paper, we introduce a large-scale, real-world raindrop removal dataset called Raindrop Clarity. Raindrop Clarity comprises 15,186 high-quality pairs/triplets (raindrops, blur, and background) of images with raindrops and the corresponding clear background images. There are 5,442 daytime raindrop images and 9,744 nighttime raindrop images. Specifically, the 5,442 daytime images include 3,606 raindrop- and 1,836 background-focused images. While the 9,744 nighttime images contain 4,838 raindrop- and 4,906 background-focused images. Our dataset will enable the community to explore background-focused and raindrop-focused images, including challenges unique to daytime and nighttime conditions.

RaindropClarity Dataset

Day_Train	Dropbox	BaiduPan code:j9ay
Day_Test	Dropbox	BaiduPan code:dten
Night_Train	Dropbox	BaiduPan code:hmsw
Night_Test	Dropbox	BaiduPan code:nten

Pre-trained Models: BaiduPan code:i3dg and Results code:outp

Model Name	Model Dropbox	Model BaiduPan	Results Dropbox	Results BaiduPan
Raindrop + Restoration	Dropbox	BaiduPan code:i3dg	Dropbox	BaiduPan code:outp

Evaluation

python calculate_psnr_ssim_sid.py

please change base_path, time_of_day, model_name accordingly.

Test

bash run_eval_diffusion_day.sh

bash run_eval_diffusion_night.sh

Inside script, please change model_name accordingly.

CUDA_VISIBLE_DEVICES=7 python eval_diffusion_day_dit.py --sid "$sid"

CUDA_VISIBLE_DEVICES=6 python eval_diffusion_day_rdiffusion.py --sid "$sid"

CUDA_VISIBLE_DEVICES=2 python eval_diffusion_day_restomer.py --sid "$sid"

CUDA_VISIBLE_DEVICES=1 python eval_diffusion_day_uformer.py --sid "$sid"

CUDA_VISIBLE_DEVICES=2 python eval_diffusion_day_onego.py --sid "$sid"

CUDA_VISIBLE_DEVICES=1 python eval_diffusion_day_idt.py --sid "$sid"

CUDA_VISIBLE_DEVICES=2 python eval_diffusion_day_icra.py --sid "$sid"

CUDA_VISIBLE_DEVICES=1 python eval_diffusion_day_atgan.py --sid "$sid"

Train

CUDA_VISIBLE_DEVICES=1,2 python train.py --config daytime_64.yml --test_set Raindrop_DiT

please change daytime_64.yml,daytime_128.yml,daytime_256.yml according to model_name and image_size.

Acknowledgments

Code is implemented based WeatherDiffusion, we would like to thank them.

License

The code and models in this repository are licensed under the MIT License for academic and other non-commercial uses.
For commercial use of the code and models, separate commercial licensing is available. Please contact:

• Yeying Jin ([email protected])
• Jonathan Tan ([email protected])

Citation

If this work is useful for your research, please cite our paper.

@article{jin2024raindrop,
  title={Raindrop Clarity: A Dual-Focused Dataset for Day and Night Raindrop Removal},
  author={Jin, Yeying and Li, Xin and Wang, Jiadong and Zhang, Yan and Zhang, Malu},
  journal={arXiv preprint arXiv:2407.16957},
  year={2024}
}