DOI number:  10.1111/cgf.13566

Affiliation of Author(s):  Department of Computer Science and Technology Tsinghua University

Journal:  Computer Graphics Forum

Abstract:  Abstract Video stabilization is necessary for many hand‐held shot videos. In the past decades, although various video stabilization methods were proposed based on the smoothing of 2D, 2.5D or 3D camera paths, hardly have there been any deep learning methods to solve this problem. Instead of explicitly estimating and smoothing the camera path, we present a novel online deep learning framework to learn the stabilization transformation for each unsteady frame, given historical steady frames. Our network is composed of a generative network with spatial transformer networks embedded in different layers, and generates a stable frame for the incoming unstable frame by computing an appropriate affine transformation. We also introduce an adversarial network to determine the stability of apiece of video. The network is trained directly using the pair of steady and unsteady videos. Experiments show that our method can produce similar results as traditional methods, moreover, it is capable of handling challenging unsteady video of low quality, where traditional methods fail, such as video with heavy noise or multiple exposures. Our method runs in real time, which is much faster than traditional methods.

Note:  Other ISSNs: 1467-8659

Co-author:  Jun Hu,Miao Wang,Tai-Jiang Mu,Shi-Min Hu

First Author:  Sen-Zhe Xu

Indexed by:  Journal Article

Discipline:  Engineering

Document Type:  J

Volume:  37

Issue:  7

Page Number:  267–276

ISSN No.:  0167-7055

Translation or Not:  no

Date of Publication:  2018-10-24

Links to published journals:  https://onlinelibrary.wiley.com/doi/10.1111/cgf.13566