Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/147030
|
| Title: | 自監督式學習的單張影像除雨技術
Self-supervised Single Image Deraining |
| Authors: | 李偉華
Li, Wei-Hua |
| Contributors: | 彭彥璁
Peng, Yan-Tsung
李偉華
Li, Wei-Hua |
| Keywords: | 影像處理
影像除雨
自監督式學習
Image processing
Image deraining
Self-supervised learning |
| Date: | 2023 |
| Issue Date: | 2023-09-01 15:23:59 (UTC+8) |
| Abstract: | 單一影像除雨 (Single Image Deraining) 的任務目標在於去除單一影 像中的雨紋，該領域近年來引起了許多關注。近期在這個主題的研 究，主要集中在深度學習中的監督式學習方法上，該方法使用下雨場 景影像與其相對應的乾淨影像來訓練模型。然而，收集成對影像的工 作相當花費時間與人力成本。因此，我們提出了 Rain2Avoid (R2A)， 一個只需要一張下雨場景影像就可以除雨的自監督式學習模型。
我們也提出一個參考局部影像梯度來預測潛在雨紋的模組，在自監 督的訓練過程中我們會略過雨紋像素，參考區域相似的像素來產生較 乾淨的背景影像，並直接對輸入下雨影像進行自監督式訓練。可以預 期的是自監督式的 R2A 表現可能不如有使用乾淨影像作為參考的監 督式學習模型。但是當訓練的成對影像是無法取得時，R2A 就會有優 勢，R2A 可以只使用一張下雨場景影像進行自監督學習。實驗結果顯 示，我們所提出的方法表現得比最先進的小樣本除雨和自監督降噪方 法還要良好。
It is common to take pictures outside; however, the weather may not be good. If we shoot the picture on a rainy day, we might capture rain streaks in the image. Image deraining is one of the image processing tasks, trying to remove the rain streaks on the image. Most works in these years apply a supervised image-deraining method, which relies on rainy-clean image pairs to train. However, collecting such pairwise images is strenuous and time- consuming. Therefore, some works generated synthetic rainy images, making it easier to get lots of pairwise images. However, using synthetic images to train a deraining model may not work well on real rainy images.
We present a novel self-supervised method based on locally dominant gra- dient prior (LDGP) and non-local self-similarity stochastic sampling (NSSS) which can respectively extract the potential rain streak and generate the stochas- tic derain reference. With the help of LDGP and NSSS, we can self-supervise only one single image for image deraining. Extensive experiments on syn- thetic and real image datasets validate the potential of our self-supervised image-deraining method. |
| Reference: | [1] Yunhao Ba, Howard Zhang, Ethan Yang, Akira Suzuki, Arnold Pfahnl, Chethan Chinder Chandrappa, Celso M de Melo, Suya You, Stefano Soatto, Alex Wong, et al. Not just streaks: Towards ground truth for single image deraining. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII, pages 723–740. Springer, 2022.
[2] Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In Proc. Int’l Conf. Machine Learning. PMLR, 2019.
[3] Xiang Chen, Hao Li, Mingqiang Li, and Jinshan Pan. Learning a sparse transformer network for effective image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5896–5905, 2023.
[4] Xiang Chen, Jinshan Pan, Kui Jiang, Yufeng Li, Yufeng Huang, Caihua Kong, Longgang Dai, and Zhentao Fan. Unpaired deep image deraining using dual contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2017–2026, 2022.
[5] Yi-Lei Chen and Chiou-Ting Hsu. A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In Proc. Conf. Computer Vision and Pattern Recognition, 2013.
[6] Liang-Jian Deng, Ting-Zhu Huang, Xi-Le Zhao, and Tai-Xiang Jiang. A directional 53 global sparse model for single image rain removal. Applied Mathematical Modelling, 2018.
[7] David Eigen, Dilip Krishnan, and Rob Fergus. Restoring an image taken through a window covered with dirt or rain. In Proc. Int’l Conf. Computer Vision, 2013.
[8] Kshitiz Garg and Shree K Nayar. Photorealistic rendering of rain streaks. ACM Transactions on Graphics (TOG), 2006.
[9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. Conf. Computer Vision and Pattern Recognition, 2016.
[10] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
[11] Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbor2neighbor: Self-supervised denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14781–14790, 2021.
[12] Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Baojin Huang, Yimin Luo, Jiayi Ma, and Junjun Jiang. Multi-scale progressive fusion network for single image deraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8346–8355, 2020.
[13] Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, Liang-Jian Deng, and Yao Wang. Fastderain: A novel video rain streak removal method using directional gradient priors. IEEE Trans. on Image Processing, 2018. 54
[14] Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-learning denoising from single noisy images. In Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[15] Wooseok Lee, Sanghyun Son, and Kyoung Mu Lee. Ap-bsn: Self-supervised denoising for real-world images via asymmetric pd and blind-spot network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17725–17734, 2022.
[16] Junyi Li, Zhilu Zhang, Xiaoyu Liu, Chaoyu Feng, Xiaotao Wang, Lei Lei, and Wangmeng Zuo. Spatially adaptive self-supervised learning for real-world image denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9914–9924, 2023.
[17] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proceedings of the European conference on computer vision (ECCV), pages 254–269, 2018.
[18] Yizhou Li, Yusuke Monno, and Masatoshi Okutomi. Single image deraining network with rain embedding consistency and layered lstm. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4060–4069, 2022.
[19] Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, and Michael S Brown. Rain streak removal using layer priors. In Proc. Conf. Computer Vision and Pattern Recognition, 2016.
[20] Yu Luo, Yong Xu, and Hui Ji. Removing rain from a single image via discriminative sparse coding. In Proc. Int’l Conf. Computer Vision, 2015. 55
[21] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. Int’l Conf. Computer Vision, 2001.
[22] Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assessment in the spatial domain. IEEE Trans. on Image Processing, 2012.
[23] Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making a “completely blind”image quality analyzer. IEEE Signal processing letters, 2012.
[24] Yan-Tsung Peng and Wei-Hua Li. Rain2avoid: Self-supervised single image deraining. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023.
[25] Shyam Nandan Rai, Rohit Saluja, Chetan Arora, Vineeth N Balasubramanian, Anbumani Subramanian, and CV Jawahar. Fluid: Few-shot self-supervised image deraining. In Proc. of the IEEE/CVF Winter Conf. on Applications of Computer Vision, 2022.
[26] Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. Progressive image deraining networks: A better and simpler baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3937–3946, 2019.
[27] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015. 56
[28] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In CVPR, pages 9446–9454, 2018.
[29] Hong Wang, Qi Xie, Qian Zhao, Yuexiang Li, Yong Liang, Yefeng Zheng, and Deyu Meng. Rcdnet: An interpretable rain convolutional dictionary network for single image deraining. arXiv preprint arXiv:2107.06808, 2021.
[30] Yinglong Wang, Shuaicheng Liu, Chen Chen, and Bing Zeng. A hierarchical approach for rain or snow removing in a single color image. IEEE Trans. on Image Processing, 2017.
[31] Zejin Wang, Jiazheng Liu, Guoqing Li, and Hua Han. Blind2unblind: Self-supervised image denoising with visible blind spots. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2027–2036, 2022.
[32] Zichun Wang, Ying Fu, Ji Liu, and Yulun Zhang. Lg-bpn: Local and global blind-patch network for self-supervised real-world denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18156–18165, 2023.
[33] Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, and Ying Wu. Semi-supervised transfer learning for image rain removal. In Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[34] Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. Deep joint rain detection and removal from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1357–1366, 2017. 57
[35] Rajeev Yasarla, Vishwanath A Sindagi, and Vishal M Patel. Syn2real transfer learning for image deraining using gaussian processes. In Proc. Conf. Computer Vision and Pattern Recognition, 2020.
[36] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5728–5739, 2022.
[37] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14821–14831, 2021.
[38] Dan Zhang, Fangfang Zhou, Yuwen Jiang, and Zhengming Fu. Mm-bsn: Self-supervised image denoising for real-world with multi-mask based on blind-spot network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4188–4197, 2023.
[39] He Zhang, Vishwanath Sindagi, and Vishal M Patel. Image de-raining using a conditional generative adversarial network. IEEE transactions on circuits and systems for video technology, 2019.
[40] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26(7):3142–3155, 2017.
[41] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. Conf. Computer Vision and Pattern Recognition, 2018 |
| Description: | 碩士
國立政治大學
資訊科學系
110753106 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0110753106 |
| Data Type: | thesis |
| Appears in Collections: | [資訊科學系] 學位論文
|
Files in This Item:
| File |
Description |
Size | Format | |
|---|
| 310601.pdf | | 41740Kb | Adobe PDF2 | 113 | View/Open |
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
