U2PNet: an Unsupervised Underwater Image-Restoration Network Using Polarization

The authors of this paper include Linghao Shen, Haisheng Xia, Xun Zhang, Yongqiang Zhao, Ning Li, Seong G. Kong, Binglu Wang, and Zhijun Li. Linghao Shen, Xun Zhang, Yongqiang Zhao, and Ning Li are from the School of Automation at Northwestern Polytechnical University, with research interests involving image processing, computational optical imaging, and intelligent optoelectronic perception. Haisheng Xia's research focus is on rehabilitation wearable robotics technology. Seong G. Kong is a visiting scholar at Seoul National University and Purdue University, with research interests in intelligent signal processing and pattern recognition. Binglu Wang's research direction is computer vision and digital signal processing. Zhijun Li is a professor at Tongji University, with research covering adaptive control and mobile robotics technology.

1. Introduction

• Challenges of Underwater Robot Vision Sensing
• Physical Model of Underwater Image Degradation
• Polarization-Based Underwater Image Restoration Methods
• Necessity of Unsupervised Underwater Image Restoration

2. Related Work

• Physics-Based Methods
• Image Enhancement-Based Methods
• Deep Learning-Based Methods

3. Proposed Method

• Underwater Image Formation Model
• Non-Reference Constraints of Transmittance Map
• Non-Reference Constraints of Restored Image
• Polarization-Based Network U2PNet

4. Experiments

• Implementation Details
  • Datasets
  • Experimental Settings
  • Benchmark Methods
  • Evaluation Metrics
• Evaluation and Comparison on Simulated Datasets
  • Qualitative Comparison
  • Quantitative Comparison
• Method Comparison on Real Datasets
  • Qualitative Comparison
  • Quantitative Comparison
• Ablation Study
• Running Time Comparison
• Convergence and Pre-Training Effect Analysis
• Generalization Performance

5. Conclusion

Q: What research methods were specifically used in the paper?

• Underwater Image Formation Model: The Jaffe-McGlamery model was used to describe the degradation process of underwater images, which decomposes the image into direct transmission and backward scattering components.
• Non-Reference Constraints:
  • Local Gradient Constraint of Transmission Map t: By analyzing the relationship between transmission map t and polarization degree, a loss function was designed to ensure the recovery of image details and color.
  • Global Intensity, Local Color, and Gradient Constraints of Restored Image J: By analyzing the reasons for intensity changes in underwater environmental imaging, a loss function was designed to estimate the intensity of the restored image and ensure its details and color.
• U2PNet Network:
  • T-Net Subnetwork: Used for estimating transmission map t, taking two polarized images I‖ and I⊥ as input.
  • B∞-Net Subnetwork: Used for estimating the light intensity at infinity B∞, taking intensity image I as input.
  • Loss Function: Combining the above constraints, a non-reference loss function was designed, including spatial consistency loss, intensity loss, color loss, etc., for training the network and ensuring recovery effects.

Q: What are the main research findings and achievements?

• The U2PNet network achieved state-of-the-art performance on both simulated and real underwater polarized images.
• The U2PNet network does not require pre-training and can perform underwater image restoration with only a single blurred stitched polarized image.
• The U2PNet network has a lightweight network structure and efficient loss functions, with high computational efficiency.
• The U2PNet network has good generalization capabilities and can be applied to image restoration in scenes such as fog and sandstorms.

Q: What are the current limitations of this research?

• There is a lack of underwater polarized image data and a benchmark dataset.
• The U2PNet network contains multiple tunable parameters and assumes that backward scattering has a higher polarization degree.
• When the polarization degree of the scene is much greater than that of the water, the method may fail.