Dynamic Point Cloud Compression for Free Viewpoint Video.pdf

1Dynamic Point Cloud Compression for Free Viewpoint Video Edouard Lamboray Michael Waschbüsch Stephan Würmlin Hanspeter Pfister* Markus Gross Computer Graphics Laboratory, Swiss Federal Institute of Technology (ETH), Zurich, Switzerland *MERL - Mitsubishi Electric Research Laboratories, Cambridge, MA, USA {lamboray, waschbuesch, wuermlin, grossm}@inf.ethz.ch pfister@ Abstract In this paper, we present a coding framework addressing the compression of dynamic 3D point clouds which repre- sent real world objects and which result from a video acquisition using multiple cameras. The encoding is per- formed as an off-line process and is not time-critical. The decoding however, must allow for real-time rendering of the dynamic 3D point cloud. We introduce a compression framework which encodes multiple attributes like depth and color of 3D video fragments into progressive streams. The reference data structure is aligned on the original camera input images and thus allows for easy view-depen- dent decoding. The separate encoding of the objects sil- houette allows the use of shape-adaptive compression algorithms. A novel differential coding approach permits random access in constant time throughout the complete data set and thus enables true free viewpoint video. 1. Introduction In the past, many 3D reconstruction methods have been investigation but no method was presented which is tai- lored to streaming and compression of free viewpoint video. In this paper, we present a compression framework for 3D video fragments which is a dynamic point based representation tailored for real-time streaming and display of free viewpoint videos [14]. Our representation general- izes 2D video pixels towards 3D irregular point samples and thus we combine the simplicity of conventional 2D video processing with the power of more complex polygo- nal representations for free viewpoint video. 3D video fragments are generic in the sense that they work with any real-time 3D reconstruction method wh