Inertia Sensor Aided Alignment for Burst Pipeline in Low Light Conditions

Abstract: Merging short-exposure frames can provide an image with reduced noise in low light conditions. However, how best to align images before merging is an open problem. To improve the performance of alignment, we propose an inertia-sensor aided algorithm for smartphone burst photography, which takes rotation and out-plane relative movement into account. To calculate homography between frames, a three by three rotation matrix is calculated from gyro data recorded by smartphone inertia sensor and three-dimensional translation vector are estimated by matched feature points detected from two frames. The rotation matrix and translations are combined to form the initial guess of homography. An unscented Kalman filter is utilized to provide a more accurate homography estimation. We have tested the algorithm on a variety of different scenes with different camera relative motions. We compare the proposed method to benchmark single-image and multi-image denoising methods with favorable results.