3D & ROI Video

Synthesis of integral imaging based 3D-images and 3D-video

By using MegaPOV the scene description language of POV-Ray is fully incorporated into the tool, which allows for exact definitions of arbitrarily complex 3D-scenes. Compared to experimental research the evaluation of different static and dynamic II-techniques is greatly facilitated.

Read more about synthesis for 3D integral imaging

Pseudo video sequence coding of integral imaging based 3D-images

There is a significant amount of inherent redundancy in an integral image. Applying a 2D-image coding scheme to an integral image fail to extract and reduce this redundancy efficiently. We have proposed a pre-processing and compression scheme that aims to enhance the compression efficiency of integral images. The scheme first transforms the still integral image into a pseudo video sequence consisting of sub-images, which is then compressed using an H.264 video encoder. Thereby leveraging on the efficient temporal and spatial coding tools of the H.264-standard for coding integral images.

The improvement in compression efficiency of using this scheme is evaluated and presented. Different parameterizations are also investigated, which affects the coding efficiency at low range bitrates. An average PSNR increase of 5.7 dB or more, compared to JPEG 2000, is observed on a set of reference images. Compared to other coding schemes the introduced artifacts are distributed more homogenously within the depicted 3D-volume.

Read more about pseudo video sequence coding for 3D integral imaging

Spatio-temporal pre-processing methods for region-of-interest video coding

Region of interest (ROI) video coding uses the information most interest to the viewer to control the allocation of bits to the background and the ROI. The aim is to increase the quality in the ROI at the expense of the quality in the background. We have proposed and investigated three pre-processing filters (independent codec) based on previous approaches: The spatial filter that only modifies the background within a single frame and the temporal filter that uses information from the previous frame. These two filters are also combined into a spatiotemporal filter.

The abilities of these filters to reduce the number of bits necessary to encode the background and to successfully re-allocate these to the ROI have been investigated. The computational complexity they introduce have also been analysed. The spatio-temporal filter was shown to give the best performance. This was verified in qualitative tests where the spatio-temporal filter gives an improvement in PSNR of the ROI of more than 1.32 dB or a reduction in bit-rate of 31% compared to the encoding of the original sequence. The spatial filter gives a similar or slightly worse result. However, the computational complexity of the spatio-temporal filter is lower and so has the better performance. Additional subjective tests and an analysis of motion vector statistics produced a similar result.

Get video sequences using spatio-temporal pre-filter

Publications

1. L. S. Karlsson and M. Sjöström, "A preprocessing approach to ROI Video Coding using Variable Gaussian Filters and Variance in Intensity", Proceedings of 47th International Symposium ELMAR-2005 focused on Multimedia Systems and Applications, 8-10 June 2005, Zadar, Croatia, pp. 65-68, June 2005
2. L. S. Karlsson and M. Sjöström, "Improved ROI Video Coding using Variable Gaussian Pre-Filters and Variance in Intensity", Proceedings of IEEE International Conference on Image Processing 2005, Vol. II, pp. 313-316, 2005
3. Olsson, R. and Xu, Y., “A Ray-Tracing Based Simulation Environment for Generating Integral Imaging Source Material”, Proceedings of RadioVetenskap och Kommunikation, June 2005
4. Olsson, R. and Xu, Y., “A ray-tracing based interactive simulation environment for generating integral imaging video sequences”, Proceedings of Optics East - ITCOM - Three-Dimensional TV, Video, and Display IV, October 2005 
5. Olsson, R.; Sjöström, M. and Xu, Y., “A Combined Pre-processing and H.264-compression Scheme for 3D Integral Images”, Proceedings of ICIP 2006, October 2006 
6. L. S. Karlsson, R. Olsson and M. Sjöström: Temporal Filter with Bilinear Interpolation for ROI Video Coding, MUCOM Technical Report, 2006
7. L. S. Karlsson, M. Sjöström and R. Olsson: "Spatio-Temporal Filter for ROI Video Coding", EUSIPCO 06, 2006
8. Olsson, R.; Sjöström, M. and Xu, Y., “Evaluation of Combined Pre-Processing and H.264-Compression Schemes for 3D Integral Images”, Proceedings of Electronic Imaging - VCIP, January 2007
9. Olsson, R. and Sjöström, M., ”A Depth Dependent Quality Metric for Evaluation of Coded Integral Imaging based 3D-images", Proceedings of 3DTV-Conference 2007, May 2007
10. L. S. Karlsson, “Spatio-temporal pre-processing methods for region-of-interest video coding", Licentiate Thesis no 21, Mid Sweden University, Sundsvall, ISBN 978-91-85317-45-5, ISNN 1652-8948, April 2007
11. Olsson, R. and Sjöström, M., ”A novel quality metric for evaluating depth distribution of artifacts in coded still 3D images", To be presented at Electronic Imaging -- Stereoscopic Displays and Applications, January 2008
12. Karlsson L.S. and Sjöström M., "A Spatio-Temporal Filter for Region-of-Interest Video Coding", Signal processing: Image communication Journal, submitted, 2007
13. Karlsson L.S. and Sjöström M., "Region-of-Interest 3D Video Coding based on Depth Images", To be submitted.