Towards Reliable Stereoscopic 3D Quality Evaluation: Subjective Assessment and Objective Metrics

Abstract

Stereoscopic three-dimensional (3D) services have become more popular recently amid promise of providing immersive quality of experience (QoE) to the end-users with the help of binocular depth. However, various arisen artifacts in the stereoscopic 3D processing chain might cause discomfort and severely degrade the QoE. Unfortunately, although the causes and nature of artifacts have already been clearly understood, it is impossible to eliminate them under the limitation of current stereoscopic 3D techniques. Moreover, their influence on the perceived quality is not well understood. Therefore, quality evaluation, both subjective assessment and objective metrics are necessarily required to understand, measure and eventually, model and predict stereoscopic 3D quality. The thesis, composed of six collected papers, contributes to the field of quality evaluation of stereoscopic 3D media in three aspects.

First, quality evaluation of crosstalk perception was carried out on polarized stereoscopic display, since crosstalk is one of the most annoying artifacts in the visualization stage of stereoscopic 3D and can not be completely eliminated with current technologies. The subjective tests were customized for crosstalk perception with varying independent parameters of scene content, camera baseline and crosstalk level. An objective metric for crosstalk perception was proposed based on our findings of perceptual attributes of crosstalk perception, namely shadow degree, separation distance and spatial position of crosstalk. Furthermore, subjective crosstalk assessment methodologies for auto-stereoscopic displays at arbitrary viewing positions in a specified area were suggested, supported by a head and score tracking system.

Second, an extension from crosstalk perception to QoE in the simplest stereoscopic system was studied, since QoE is often referred as a criterion for the acceptance of any commercial system and determines the success or not. In addition to crosstalk level, other requisite factors of the simplest stereoscopic system, including scene content, camera baseline, screen size and viewing position have also been investigated on their relationships to perceptual attributes of QoE. Specifically, those perceptual attributes are crosstalk perception and depth enabled visual comfort, which cover the main negative and positive aspects of stereoscopic QoE. By modeling these perceptual attributes separately and combining them thereafter, an objective QoE metric was proposed.

Third, further work on the complex stereoscopic system was carried out by investigating the influence of coding artifacts on QoE. This work was done under the context of assessing 3D video compression technologies within MPEG’s effort for standardizing 3D video coding techniques. However, the subjective scores can be used as ground truth dataset for proposing QoE model which will incorporate both the coding artifacts and configurations of the complex stereoscopic system. Meanwhile, since the subjective tests were conducted at 13 laboratories around the world with large amount of test sessions, it can be used for defining a process for certification of subjective test campaigns.