Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

Images taken in low light conditions have motion blur and photon shot noise.
State-of-the-art image restoration networks are limited to well-illuminated scenes and don’t perform well with photon shot noise.
A new blur estimation technique is proposed to address photon-limited conditions.
The proposed method uses a gradient-based backpropagation method to estimate the blur kernel.
The blur kernel is modeled using a low-dimensional representation with key points on the motion trajectory.
This low-dimensional representation provides improved kernel estimates and better deconvolution performance.

Paper studies photon-limited blind deconvolution problem
Blur kernel and latent clean image are recovered from blurred image
Forward model includes blurred-noisy image, latent clean image, and blur kernel
Deep neural networks have been used to solve blind image deconvolution
Iterative kernel estimation method proposed to update blur kernel
Kernel modeled in low-dimensional space by specifying key points of motion trajectory
Iterative framework used to prove regularity of kernel estimation problem
Better blur kernel estimates in photon-limited regimes

Traditional Blind Deconvolution uses joint optimization framework with kernel and image priors
Levin et. al pointed out that this framework favours the no-blur degenerate solution
Some methods model the blur kernel in terms of the camera trajectory and then recover both the trajectory and the clean image
Poisson deconvolution problem has been studied for decades
Deep Learning Methods have been proposed for the blind image deblurring task
Neural Networks and Iterative Methods have shown promising results

Camera motion blur can be modeled as a latent clean image convolved with a blur kernel.
The blur kernel is generated from a two-dimensional trajectory.
A differentiable and low-dimensional representation of the blur kernel is proposed.
A neural network is used to learn the transformations from the key points to the blur kernels.

Initial estimate of direction and magnitude of blur is generated
Initialization uses rectilinear motion kernel, parameterized by length and orientation
Iterative scheme is divided into three stages
Stage I searches for appropriate kernel representation in latent space
Stage II fine-tunes kernel obtained from Stage I
Initialization uses minor variation of kernel estimation in PolyBlur
Cost function is evaluated using non-blind solver and Kernel-Trajectory Network
Cost function includes reblurring loss and 1-norm sparsity prior
Networks are retrained using synthetic and GoPro datasets
Poisson noise is added to blurred images for retraining

End-to-end trained methods compared to approach in supplementary document
Figure 1 and 2 show examples from RealBlur dataset
Figure 3 shows examples from PLDD dataset
Figure 4 shows reconstruction results on synthetically blurred images from Levin dataset

Performance of the scheme improves when increasing the number of key points used for representing kernels.
Using a Kernel Trajectory Network improves the performance of the scheme significantly.

We use an iterative framework for the photon-limited blind deconvolution problem.
We use a non-blind solver to deconvolve Poisson corrupted and blurred images.
We outperform end-to-end trained image restoration networks.
We use a Poisson noise model for photon-limited settings.
The signal-to-noise ratio for the measurements is low in the photon-limited regime.
We explore photon-limited deconvolution for α ∈ [10,40].
We use a kernel initialization method to estimate a rectilinear kernel.
We use a KTN architecture to represent the motion blur.
We use symmetric boundary conditions for the inverse problem scheme.
We set the initial step size as δ = 10^5 and δ = 2.0 for Stage I and II respectively.
We limit the number of iterations to 150.
We use a Nvidia TitanX GPU for the experiments.
We view the blur kernel as the continuous camera trajectory reduced to K key points.
We learn a differentiable representation from the vectorized K key points to a blur kernel.
We calculate the reblurring loss using symmetric boundary conditions.