Selecting scenes for 2D and 3D subjective video quality tests

This paper presents recommended techniques for choosing video sequences for subjective experiments. Subjective video quality assessment is a well-understood field, yet scene selection is often driven by convenience or content availability. Three-dimensional testing is a newer field that requires new considerations for scene selection. The impact of experiment design on best practices for scene selection will also be considered. A semi-automatic selection process for content sets for subjective experiments will be proposed.

• Publication year

  2013

• Venue

  EURASIP Journal on Image and Video Processing

• Publication date

  2013-08-28

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1186/1687-5281-2013-50
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• 2D VS. 3D VISUAL QUALITY EVALUATION: THE DEPTH FACTOR

  2013cited by this paper
• Subjective assessment of visual discomfort induced by binocular disparity and stimulus width in stereoscopic image

  2013cited by this paper
• RMIT3DV: Pre-announcement of a creative commons uncompressed HD 3D video database

  2012cited by this paper
• Impact of mobile devices and usage location on perceived multimedia quality

  2012cited by this paper
• Video quality evaluation in the cloud

  2012cited by this paper
• Analysis of Public Image and Video Databases for Quality Assessment

  2012cited by this paper
• NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences

  2012cited by this paper
• Study of depth bias of observers in free viewing of still stereoscopic synthetic stimuli

  2012cited by this paper
• What Makes a Professional Video? A Computational Aesthetics Approach

  2012influential reference
• Audiovisual Quality Components

  2011cited by this paper
• New stereoscopic video shooting rule based on stereoscopic distortion parameters and comfortable viewing zone

  2011cited by this paper
• Study on visual discomfort induced by stimulus movement at fixed depth on stereoscopic displays using shutter glasses

  2011cited by this paper
• Author manuscript, published in "International Workshop on Quality of Multimedia Experience QoMEX, Mechelen: Belgium (2011)" THE INFLUENCE OF RELATIVE DISPARITY AND PLANAR MOTION VELOCITY ON VISUAL DISCOMFORT OF STEREOSCOPIC VIDEOS

  2011cited by this paper
• NEW REQUIREMENTS OF SUBJECTIVE VIDEO QUALITY ASSESSMENT METHODOLOGIES FOR 3DTV

  2010cited by this paper
• A comprehensive database and subjective evaluation methodology for quality of experience in stereoscopic video

  2010cited by this paper
• Performance comparisons of subjective quality assessment methods for mobile video

  2010cited by this paper
• ANALYSIS OF FREELY AVAILABLE SUBJECTIVE DATASET FOR HDTV INCLUDING CODING AND TRANSMISSION DISTORTIONS

  2010cited by this paper
• The Consumer Digital Video Library

  2010cited by this paper
• 3D Television (3dtv) Technology, Systems, and Deployment: Rolling Out the Infrastructure for Next-Generation Entertainment

  2010cited by this paper
• Measuring visual discomfort associated with 3D displays

  2009cited by this paper
• 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen

  2009cited by this paper
• A New EDI-based Deinterlacing Algorithm

  2007cited by this paper
• Effect of disparity and motion on visual comfort of stereoscopic images

  2006cited by this paper
• An effective de-interlacing technique using two types of motion information

  2003cited by this paper
• Issues in vision modeling for perceptual video quality assessment

  1999cited by this paper
• Perceptual Effects of Noise in Digital Video Compression

  1998cited by this paper
• Two Criteria for Video Test Scene Selection

  1994cited by this paper

• Descriptor: Benchmark Dataset for Animated Multimedia Quality Assessment (BeDAMQA)

  2025cites this paper
• Subjective and Objective Quality Assessment Methods of Stereoscopic Videos with Visibility Affecting Distortions

  2024cites this paper
• A Spherical Light Field Database for Immersive Telecommunication and Telepresence Applications

  2024cites this paper
• AVT-VQDB-UHD-2-HDR: An open 8K HDR source dataset for video quality research

  2024cites this paper
• Why No Reference Metrics for Image and Video Quality Lack Accuracy and Reproducibility

  2023cites this paper
• Subjective Test Environments: A Multifaceted Examination of Their Impact on Test Results

  2023cites this paper
• Impact of Content on Subjective Quality of Experience Assessment for 3D Video

  2022cites this paper
• On the Link between Subjective Score Prediction and Disagreement of Video Quality Metrics

  2021cites this paper
• Wide Color Gamut Image Content Characterization: Method, Evaluation, and Applications

  2021cites this paper
• Subjective Assessment Experiments That Recruit Few Observers With Repetitions (FOWR)

  2021cites this paper
• Perceptual Quality of Audio-Visual Content with Common Video and Audio Degradations

  2021cites this paper
• Subjective Assessment of 360° Image Projection Formats

  2020cites this paper
• Virtual Reality Laboratories: A Way Forward for Schools?

  2020cites this paper
• Towards Better Quality Assessment of High-Quality Videos

  2020cites this paper
• Colornet - Estimating Colorfulness in Natural Images

  2019cites this paper
• Evaluating experiment design with unrepeated scenes for video quality subjective assessment

  2019cites this paper
• A three layer system for audio-visual quality assessment

  2019influential citation
• Improving relevant subjective testing for validation: Comparing machine learning algorithms for finding similarities in VQA datasets using objective measures

  2019cites this paper
• Study of Subjective Quality and Objective Blind Quality Prediction of Stereoscopic Videos

  2019cites this paper
• Performance assessment of three-dimensional video codecs in mobile terminals

  2018cites this paper
• A Testbed for Evaluating Video Streaming Services in LTE

  2017cites this paper
• Evaluation of the effects of transmission impairments on perceived video quality by exploiting ReTRiEVED dataset

  2017cites this paper
• Characterization and selection of light field content for perceptual assessment

  2017influential citation
• Descriptive statistical analysis of subjective experiments in mobile imaging

  2017cites this paper
• Quality of experience measurement of compressed multi-view video

  2017cites this paper
• Proof-of-concept: role of generic content characteristics in optimizing video encoders

  2017cites this paper
• Hybrid simulation of ethernet based intra-vehicle networks

  2016cites this paper
• Exemplary Test Design and Evaluation of an Autostereoscopic 3DTV considering Display Operating Parameters

  2016cites this paper
• Content-aware adaptive multiple description coding scheme

  2016cites this paper
• What's on TV: A large scale quantitative characterisation of modern broadcast video content

  2016cites this paper
• On the Optimal Presentation Duration for Subjective Video Quality Assessment

  2016cites this paper
• Support for reduced presentation durations in subjective video quality assessment

  2016cites this paper
• Image Quality of Experience: A Subjective Test Targeting the Consumer's Experience

  2016cites this paper
• Evaluating the influence of packet loss on visual quality of perception for high bandwidth automotive networks

  2016cites this paper
• Badanie korelacji pomiędzy technicznymi parametrami obrazu wideo stereoskopowego (3D) a subiektywną oceną treści wideo

  2016cites this paper
• Hybrid Sequencing of Uncompressed and Compressed 3D Stereoscopic Video: A Preliminary Quality Evaluation

  2016cites this paper
• Objective Video Quality Assessment - Towards Large Scale Video Database Enhanced Model Development

  2015cites this paper
• USING VIDEO IN EDUCATIONAL PROCESS IS AN URGENT NEED

  2015cites this paper
• Assessment of IP packet loss influence on perceptual quality of streaming video

  2015cites this paper
• Characterization of the HEVC Coding Efficiency Advance Using 20 Scenes, ITU-T Rec. P.913 Compliant Subjective Methods, VQM, and PSNR

  2015cites this paper
• Video Quality Assessment: Subjective testing of entertainment scenes

  2015cites this paper
• The impact of complexity in the rate-distortion optimization: A visualization tool

  2015cites this paper
• Study of the effects of video content on quality of experience

  2015cites this paper
• Subjective Quality Assessment of Longer Duration Video Sequences Delivered Over HTTP Adaptive Streaming to Tablet Devices

  2014cites this paper
• A new method for immersive audiovisual subjective testing

  2014cites this paper
• A study on the effects of quality of service parameters on perceived video quality

  2014cites this paper
• Audiovisual quality in multimedia services Subjective assessment methods for 3 D video quality

  year unknowncites this paper