视频修补.pptVIP

Video Repairing: Inference of Foreground and background Under Severe Occlusion Repot: Gui Yan 09.6.19 Outline Introduction Static background repairing Moving foreground repairing The video repairing system Introduction The problem is stated as follows: Given a video with severe damage or occlusion, a synthesis approach will be proposed to infer missing background and foreground to achieve a visually acceptable video restoration. Static background repairing Input preparation (unnecessary ) 1)User can manually remove the object by video mask 2)Tracking and locating automatically the mean shift tracking algorithm Layer segmentation and propagation Layer representation is the key to maintain temporal consistency without distortion. The layer boundary needs not be very accurate; The segmentation boundaries in key frames will be automatically propagated to other frames by mean shift tracking algorithm again; Different layers in frame are stitched to generate layer mosaics (to guarantee temporal coherence among frames); The foreground layer (the flower bed) The background layer (the house and the sky) The overlapping boundary layer (green stripe) An iterative approach is used to estimate perspective transform; (using Phase Correlation to produce a translation matrix) Repairing the mosaic by image repairing. Homography blending The details are as follows: 1)Choosing k layered reference mosaic; 2)Foreground (M1), Background (M2) and overlapping boundary layer (M3) 3)To warp the repaired frame back. Procedure Layer propagation Layered mosaics Frame repairing Homography blending Moving foreground repairing Sampling Alignment one reasonable constraint motion is periodicity. Movel sampling: sample movel and damage movel To sample a movel: collecting the mean and variance of the luminance of all pixels (x, y) in the filed of view; detecting moving pixels; constructing connected components for the