Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

Revisit the rotation averaging problem applied in global Structure-from-Motion pipelines.
Current methods have a weakly connected cost function with the input data.
Proposed method directly propagates uncertainty from point correspondences into the rotation averaging.
Integrate a variant of the MAGSAC loss into the rotation averaging problem.
Results superior to baselines in terms of accuracy on large-scale public benchmarks.

Building 3D reconstructions from unordered image collections is important for crowd-sourced mapping.
Structure-from-Motion is used to estimate camera parameters and scene geometry.
There are two types of Structure-from-Motion: incremental and global.
Incremental methods are more accurate but have high computational cost.
Global methods are less accurate but have lower computational cost.
Rotation averaging is used to estimate globally consistent rotations.
Current methods use a cost function that is weakly connected to the input data.
This paper proposes to better model the underlying noise distributions.
This paper also proposes to integrate a variant of the MAGSAC loss.

Rotation averaging is a long standing problem in computer vision
Govindu [16] proposed an approximation to make the problem linear
Martinec and Pajdla [22] parameterized the problem in terms of a 3x3 matrix
Wilson et al. [36] investigated conditions for when the problem is easy
Eriksson et al. [11] proposed a block-coordinate descent method
Dellaert et al. [9] proposed an optimization scheme based on higher-dimensional rotations
Hartley et al. [18] proposed a Weiszfeld-algorithm to minimize the L1-loss
Gao et al. [14] proposed an iterative scheme for solving the rotation averaging problem
Wilson and Snavely [37] solved the translation averaging problem using an outlier filter

Proposed a way to leverage uncertainties in rotation averaging
Rotation averaging is a component of global SfM algorithms
Formalized as a graph optimization problem
Vertices are image poses, edges are relative poses
Primary goal is to use constraints to find absolute poses of cameras
Uncertainty-aware rotation averaging considers quality of relative poses
Uncertainty is propagated from features to relative rotation
Covariances are plugged into optimization procedure to consider pair-wise uncertainties

MAGSAC++ is a robust loss function designed to reduce dependence on manually set inlier-outlier threshold parameter
MAGSAC++ loss assumes inlier residuals follow a χ2-distribution and does not make assumptions about outliers
Data points are a set of relative rotations, and the models to estimate are the global orientations of the cameras
Inlier weight is calculated using a formula with noise standard deviation, residual, and prior distribution of σ
Residuals are assumed to be normally distributed with the same variance
Weight of each edge is calculated by passing its residual to a formula
Loss is minimized by IRLS and each iteration guarantees a non-increase in the loss

Tested global SfM implemented in Theia library with and without considering uncertainties
Evaluated popular loss functions, including Soft L 1
Used angle-axis rotation parameterization to minimize relative rotation error
Tested on 1DSfM and ETH3D datasets
Tuned hyper-parameters on Madrid Metropolis scene
Compared weighting schemes using inlier number and proposed covariance
Used Cauchy loss and did not use MAGSAC to avoid bias
MAGSAC with covariance method outperformed all other methods
Proposed algorithm led to more than 10 points increase in AUC score
Proposed pipeline more accurate than baseline when images are downsampled
Lowest median orientation error achieved by proposed algorithm
Global approaches run-time two orders of magnitude lower than COLMAP

Problem of rotation averaging applied in global Structure-from-Motion pipelines
Leveraging uncertainties in two-view epipolar geometries and investigating robust losses
Integrating covariance matrices of uncertainties directly in optimization procedure improves reconstruction quality
Carefully choosing robust loss also gives a boost in accuracy
Source code will be made publicly available