HVI-CIDNet
HVI-CIDNet copied to clipboard
"You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement"
You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement
Yixu Feng∗ , Cheng Zhang∗ , Pei Wang , Peng Wu , Qingsen Yan , Yanning Zhang
News 💡
-
2024.04.28 Synchronize all Baidu Pan datasets, outputs, and weights to One Drive. 🌎
-
2024.04.18 Full version Code, models, visual comparison have been released. We promise that all the weights only trained on train sets, and each weights is reproducible. Hope it will help your future work. If you have trained a better result, please contact us. We look forward to subsequent work based on the HVI color "space"! 💎
-
2024.04.14 Update test fine tuning result and weights on LOLv1 dataset. 🧾
-
2024.03.04 Update five unpaired datasets (DICM, LIME, MEF, NPE, VV) visual results. ✨
-
2024.03.03 Update pre-trained weights and output results on our HVI-CIDNet using Baidu Pan. 🧾
-
2024.02.08 Update HVI-CIDNet in Arxiv. The new code, models and results will be uploaded. 🎈
⚙ Proposed HVI-CIDNet
HVI-CIDNet pipeline:
CID dual-branch UNet structure:
🖼 Visual Comparison
LOL-v1, LOL-v2-real, and LOL-v2-synthetic:
DICM, LIME, MEF, NPE, and VV:
LOL-Blur:
Weights and Results 🧾
All the weights that we trained on different datasets is available at [Baidu Pan] (code: yixu
) and [One Drive] (code: yixu
). Results on DICM, LIME, MEF, NPE, and VV datasets can be downloaded from [Baidu Pan] (code: yixu
) and [One Drive] (code: yixu
).
Bolded fonts represent impressive metrics.
- The metrics of HVI-CIDNet on paired datasets are shown in the following table, all the link code is
yixu
:
Folder (test datasets) | PSNR | SSIM | LPIPS | GT Mean | Results | Weights Path |
---|---|---|---|---|---|---|
(LOLv1) v1 w perc loss/ wo gt mean |
23.8091 | 0.8574 | 0.0856 | Baidu Pan and One Drive | LOLv1/w_perc.pth | |
(LOLv1) v1 w perc loss/ w gt mean |
27.7146 | 0.8760 | 0.0791 | √ | ditto | LOLv1/w_perc.pth |
(LOLv1) v1 wo perc loss/ wo gt mean |
23.5000 | 0.8703 | 0.1053 | Baidu Pan and One Drive | LOLv1/wo_perc.pth | |
(LOLv1) v1 wo perc loss/ w gt mean |
28.1405 | 0.8887 | 0.0988 | √ | ditto | LOLv1/wo_perc.pth |
(LOLv2_real) v2 wo perc loss/ wo gt mean |
23.4269 | 0.8622 | 0.1691 | Baidu Pan and One Drive | (lost) | |
(LOLv2_real) v2 wo perc loss/ w gt mean |
27.7619 | 0.8812 | 0.1649 | √ | ditto | (lost) |
(LOLv2_real) v2 best gt mean |
28.1387 | 0.8920 | 0.1008 | √ | Baidu Pan and One Drive | LOLv2_real/w_prec.pth |
(LOLv2_real) v2 best Normal |
24.1106 | 0.8675 | 0.1162 | Baidu Pan and One Drive | (lost) | |
(LOLv2_real) v2 best PSNR |
23.9040 | 0.8656 | 0.1219 | Baidu Pan and One Drive | LOLv2_real/best_PSNR.pth | |
(LOLv2_real) v2 best SSIM |
23.8975 | 0.8705 | 0.1185 | Baidu Pan and One Drive | LOLv2_real/best_SSIM.pth | |
(LOLv2_real) v2 best SSIM/ w gt mean |
28.3926 | 0.8873 | 0.1136 | √ | None | LOLv2_real/best_SSIM.pth |
(LOLv2_syn) syn wo perc loss/ wo gt mean |
25.7048 | 0.9419 | 0.0471 | Baidu Pan and One Drive | LOLv2_syn/wo_perc.pth | |
(LOLv2_syn) syn wo perc loss/ w gt mean |
29.5663 | 0.9497 | 0.0437 | √ | ditto | LOLv2_syn/wo_perc.pth |
(LOLv2_syn) syn w perc loss/ wo gt mean |
25.1294 | 0.9388 | 0.0450 | Baidu Pan and One Drive | LOLv2_syn/w_perc.pth | |
(LOLv2_syn) syn w perc loss/ w gt mean |
29.3666 | 0.9500 | 0.0403 | √ | ditto | LOLv2_syn/w_perc.pth |
Sony_Total_Dark | 22.9039 | 0.6763 | 0.4109 | Baidu Pan and One Drive | SID.pth | |
LOL-Blur | 26.5719 | 0.8903 | 0.1203 | Baidu Pan and One Drive | LOL-Blur.pth | |
SICE-Mix | 13.4235 | 0.6360 | 0.3624 | √ | Baidu Pan and One Drive | SICE.pth |
SICE-Grad | 13.4453 | 0.6477 | 0.3181 | √ | Baidu Pan and One Drive | SICE.pth |
- Performance on five unpaired datasets are shown in the following table:
metrics | DICM | LIME | MEF | NPE | VV |
---|---|---|---|---|---|
NIQE | 3.79 | 4.13 | 3.56 | 3.74 | 3.21 |
BRISQUE | 21.47 | 16.25 | 13.77 | 18.92 | 30.63 |
- While we don't recommend that you perform finetuning on the test set, in order to demonstrate the effectiveness of our model, we also provide here the results of test finetuning training on the LOLv1 dataset. Using the fine tuning technique on the test set does make the PSNR metrics higher, but other metrics are not found to be significantly changed on CIDNet, which may result in a lower generalization of the model, so we do not recommend you do this.
Folder (test datasets) | PSNR | SSIM | LPIPS | GT Mean | Results | Weights Path |
---|---|---|---|---|---|---|
(LOLv1) v1 test finetuning |
25.4036 | 0.8652 | 0.0897 | Baidu Pan and One Drive | LOLv1/test_finetuning.pth | |
(LOLv1) v1 test finetuning |
27.5969 | 0.8696 | 0.0869 | √ | ditto | ditto |
1. Get Started 🌑
Dependencies and Installation
- Python 3.7.0
- Pytorch 1.13.1
(1) Create Conda Environment
conda create --name CIDNet python=3.7.0
conda activate CIDNet
(2) Clone Repo
git clone [email protected]:Fediory/HVI-CIDNet.git
(3) Install Dependencies
cd HVI-CIDNet
pip install -r requirements.txt
Data Preparation
You can refer to the following links to download the datasets. Note that we only use low_blur
and high_sharp_scaled
subsets of LOL-Blur
dataset.
- LOLv1
- LOLv2
- LOL-Blur: Baidu Pan(code:
yixu
) and One Drive(code:yixu
) - DICM,LIME,MEF,NPE,VV: Baidu Pan(code:
yixu
) and One Drive(code:yixu
) - SICE: Baidu Pan(code:
yixu
) and One Drive(code:yixu
) - Sony-Total-Dark(SID): Baidu Pan(code:
yixu
) and One Drive(code:yixu
)
Then, put them in the following folder:
datasets (click to expand)
├── datasets
├── DICM
├── LIME
├── LOLv1
├── our485
├──low
├──high
├── eval15
├──low
├──high
├── LOLv2
├── Real_captured
├── Train
├── Low
├── Normal
├── Test
├── Low
├── Normal
├── Synthetic
├── Train
├── Low
├── Normal
├── Test
├── Low
├── Normal
├── LOL_blur
├── eval
├── high_sharp_scaled
├── low_blur
├── test
├── high_sharp_scaled
├── 0012
├── 0017
...
├── low_blur
├── 0012
├── 0017
...
├── train
├── high_sharp_scaled
├── 0000
├── 0001
...
├── low_blur
├── 0000
├── 0001
...
├── MEF
├── NPE
├── SICE
├── Dataset
├── eval
├── target
├── test
├── label
├── train
├── 1
├── 2
...
├── SICE_Grad
├── SICE_Mix
├── SICE_Reshape
├── Sony_total_dark
├── eval
├── long
├── short
├── test
├── long
├── 10003
├── 10006
...
├── short
├── 10003
├── 10006
...
├── train
├── long
├── 00001
├── 00002
...
├── short
├── 00001
├── 00002
...
├── VV
2. Testing 🌒
Download our weights from [Baidu Pan] (code: yixu
) and put them in folder weights
:
├── weights
├── LOLv1
├── w_perc.pth
├── wo_perc.pth
├── test_finetuning.pth
├── LOLv2_real
├── best_PSNR.pth
├── best_SSIM.pth
├── w_perc.pth
├── LOLv2_syn
├── w_perc.pth
├── wo_perc.pth
├── LOL-Blur.pth
├── SICE.pth
├── SID.pth
-
You can test our method as followed, all the results will saved in
./output
folder:
# LOLv1
python eval.py --lol --perc # weights that trained with perceptual loss
python eval.py --lol # weights that trained without perceptual loss
# LOLv2-real
python eval.py --lol_v2_real --best_GT_mean # you can choose best_GT_mean or best_PSNR or best_SSIM
# LOLv2-syn
python eval.py --lol_v2_syn --perc # weights that trained with perceptual loss
python eval.py --lol_v2_syn # weights that trained without perceptual loss
# SICE
python eval.py --SICE_grad # output SICE_grad
python eval.py --SICE_mix # output SICE_mix
# Sony-Total-Dark
python eval_SID_blur --SID
# LOL-Blur
python eval_SID_blur --Blur
# five unpaired datasets DICM, LIME, MEF, NPE, VV.
# We note that: you can choose one weights in ./weights folder, and set the alpha float number (defualt=1.0) as illumination scale of the datasets.
# You can change "--DICM" to the other unpaired datasets "LIME, MEF, NPE, VV".
python eval.py --unpaired --DICM --unpaired_weights --alpha
# e.g.
python eval.py --unpaired --DICM --unpaired_weights ./weights/LOLv2_syn/w_perc.pth --alpha 0.9
- Also, you can test all the metrics mentioned in our paper as follows:
# LOLv1
python measure.py --lol
# LOLv2-real
python measure.py --lol_v2_real
# LOLv2-syn
python measure.py --lol_v2_syn
# Sony-Total-Dark
python measure_SID_blur.py --SID
# LOL-Blur
python measure_SID_blur.py --Blur
# SICE-Grad
python measure.py --SICE_grad
# SICE-Mix
python measure.py --SICE_mix
# five unpaired datasets DICM, LIME, MEF, NPE, VV.
# You can change "--DICM" to the other unpaired datasets "LIME, MEF, NPE, VV".
python measure_niqe_bris.py --DICM
# Note: Following LLFlow, KinD, and Retinxformer, we have also adjusted the brightness of the output image produced by the network, based on the average value of GroundTruth (GT). This only works in paired datasets. If you want to measure it, please add "--use_GT_mean".
#
# e.g.
python measure.py --lol --use_GT_mean
- Evaluating the Parameters, FLOPs, and running time of HVI-CIDNet:
python net_test.py
3. Training 🌓
- We put all the configurations that need to be adjusted in the
./data/options.py
folder and explained them in the file. We apologize that some of the training parameters we are no longer able to provide and share with you, but we guarantee that all the weights are trainable by parameter tuning. You can train our HVI-CIDNet by:
python train.py
- All weights are saved to the
./weights/train
folder and are saved in steps of the checkpoint set in theoptions.py
asepoch_*.pth
where*
represent the epoch number. - Also, for every weight saved, metrics are measured for the validation set and printed to the command line. Finally, the results of all weights' test metrics on the validation set and options in
./data/options.py
will be saved to./results/training/metrics.md
. - In each epoch, we save an output (test) and GT image to the
./results/training
folder to facilitate the visualization of the training results and progress of each epoch, as well as to detect the generation of gradient explosion in advance. - After each checkpoint, we save all the validation set outputs for this time in the
./results
folder to the corresponding folder. Note that we use a replacement strategy for different checkpoints for each dataset. That is, we do not save the plots of all checkpoints, but only the weights of each checkpoint.
4. Contacts 🌔
If you have any questions, please contact us or submit an issue to the repository!
Yixu Feng ([email protected]) Cheng Zhang ([email protected])
5. Citation 🌕
If you find our work useful for your research, please cite our paper
@misc{feng2024hvi,
title={You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement},
author={Yixu Feng and Cheng Zhang and Pei Wang and Peng Wu and Qingsen Yan and Yanning Zhang},
year={2024},
eprint={2402.05809},
archivePrefix={arXiv},
primaryClass={cs.CV}
}