Exploring the potential of channel interactions for image restoration

Yuning Cui, Alois Knoll

(Training details can be found in the supplementary material, which can also be downloaded from this link.)

Image restoration aims to reconstruct a clear image from a degraded observation. Convolutional neural networks have achieved promising performance on this task. The usage of Transformer has recently made significant advancements in state-of-the-art performance by modeling long-range dependencies. However, these deep architectures primarily concentrate on enhancing representation learning for the spatial dimension, neglecting the significance of channel interactions. In this paper, we explore the potential of channel interactions for restoring images through our proposal of a dual-domain channel attention mechanism. To be specific, channel attention in the spatial domain allows each channel to amass valuable signals from adjacent channels under the guidance of learned dynamic weights. In order to effectively exploit the significant difference in infrequency between degraded and clean image pairs, we develop the implicit frequency domain channel attention to facilitate the integration of information from different frequencies. Extensive experiments demonstrate that the proposed network, dubbed ChaIR, achieves state-of-the-art performance on 13 benchmark datasets for five image restoration tasks, including image dehazing, image motion/defocus deblurring, image desnowing, and image deraining.

Installation

The project is built with PyTorch 3.8, PyTorch 1.8.1. CUDA 10.2, cuDNN 7.6.5 For installing, follow these instructions:

conda install pytorch=1.8.1 torchvision=0.9.1 -c pytorch
pip install tensorboard einops scikit-image pytorch_msssim opencv-python

Install warmup scheduler:

cd pytorch-gradual-warmup-lr/
python setup.py install
cd ..

Training and Evaluation

Please refer to respective directories.

Results (ChaIR)

Images here

Citation

If you find this project useful for your research, please consider citing:

@article{cui2023exploring,
  title={Exploring the potential of channel interactions for image restoration},
  author={Cui, Yuning and Knoll, Alois},
  journal={Knowledge-Based Systems},
  volume={282},
  pages={111156},
  year={2023},
  publisher={Elsevier}
}

Contact

Should you have any question, please contact Yuning Cui.