3DGS-VBench+ and 3DGSV-Assessor: A Multi-Dimensional Subjective Benchmark and a Reference-Flexible Objective Model for 3D Gaussian Splatting Video Quality Assessment

📌 Highlights

The paper combines a large subjective benchmark with a reference-flexible quality model for compressed 3DGS videos. Its central engineering idea is to jointly predict overall, geometry, and color quality, then optionally refine the prediction using one captured training view rather than requiring an aligned ground-truth video.

The strongest reported result is the reduced-reference 3DGSV-Assessor, which reaches SRCC values of 0.9714, 0.9691, and 0.9663 for overall, geometry, and color quality on 3DGS-VBench+. The no-reference variant remains close, with SRCC values of 0.9617, 0.9604, and 0.9590.

• 3DGS-VBench+ contains 3,036 rendered videos from 33 scenes and six compression algorithms.
• The benchmark provides 9,108 MOS labels across overall, geometry, and color dimensions.
• The assessor uses one generation pass for all three scores.
• The reduced-reference mode improves over the no-reference mode, while no-reference assessment remains competitive.
• The main caveat is that deployment cost and arbitrary-camera-path robustness are not quantified in the main body.

🎯 Introduction

Novel-view synthesis reconstructs a scene from limited captured viewpoints and renders arbitrary new views. 3DGS replaces implicit volume rendering with explicit anisotropic Gaussians, enabling real-time, high-fidelity rendering, but its large storage footprint motivates compression. Training and compression introduce characteristic artifacts such as structural omissions, floating, fogging, and temporal flicker along continuous camera paths.

The paper identifies three gaps in existing NVS quality assessment: most datasets are small, compression strategies and severity levels are not systematically covered, and annotations usually provide only an overall score. Because arbitrary rendered trajectories rarely have aligned ground-truth videos, full-reference evaluation is often impractical, while a captured training view can provide a weak reduced reference. The paper therefore targets a multi-dimensional model that supports both no-reference and reduced-reference assessment.

🔬 Methodology

3DGS-VBench+ uses 22 synthetic scenes from NeRF Synthetic and Explicit-NeRF-QA-Synthetic, plus 11 real-world scenes from Tanks & Temples, Deep Blending, Mip-NeRF360, and PKU-DyMVHumans. Synthetic scenes use native validation trajectories with 300 viewpoints. For real-world scenes, the authors construct smooth 360-degree orbital paths and sample 600 viewpoints. Six compression methods are evaluated: CompGS, Compact-3DGS, LightGaussian, C3dgs, Scaffold-GS, and HAC.

Each scene is paired with 92 reconstruction settings. Multi-parameter methods use controlled single-parameter ablations, cross combinations, and a highest-quality default; single-parameter methods traverse eight compression levels. The resulting videos are encoded at 30 fps with libx265 and constant rate factor 10. Fifteen subjects rate every sequence on overall, geometry, and color quality, and the ratings are normalized into MOS values on a 0–100 scale.

Stage 1 samples uniformly spaced video frames and prompts Qwen3.5 (9B) to produce five-level textual judgments for the three dimensions. Hidden states at the answer positions are concatenated into a shared representation and passed to independent regression heads. LoRA adapts both the visual encoder and language model, while cross-entropy and L1 regression are trained together. Stage 2 embeds a captured training view and same-scene video frames using a shared SigLIP encoder and trains an intra-scene ranking cue. Stage 3 forms a relational cue from reference and video embeddings, their absolute difference, and cosine similarity, then applies bounded feature modulation to the frozen Stage-1 representation.

📊 Experiments

The evaluation uses SRCC, PLCC, and KRCC, where higher values indicate better agreement with human MOS. The main benchmark is 3DGS-VBench+. Zero-shot transfer is evaluated on GSC-QA, NeRF-VSQA, and ENeRF-QA. Traditional metrics are applied directly, LMM baselines are evaluated zero-shot and with selected fine-tuning, and deep VQA baselines are retrained using the same split. The proposed model uses a within-scene 4:1 training/testing split averaged over five-fold cross-validation; external datasets are used only for zero-shot testing.

On 3DGS-VBench+, both proposed variants outperform the compared methods across all nine reported metric-dimension combinations. The reduced-reference variant reaches SRCC/PLCC/KRCC of 0.9714/0.9726/0.8529 for overall quality, 0.9691/0.9727/0.8461 for geometry quality, and 0.9663/0.9703/0.8385 for color quality. The no-reference variant reaches 0.9617/0.9628/0.8367, 0.9604/0.9628/0.8316, and 0.9590/0.9615/0.8211 respectively.

The zero-shot results show that the advantage transfers beyond the training benchmark. The reduced-reference model reaches SRCC values of 0.9333 on GSC-QA, 0.8180 on NeRF-VSQA, and 0.9644 on ENeRF-QA, while the no-reference model reaches 0.9130, 0.7437, and 0.9471. The reference cue is most useful where the no-reference model is weaker, especially on NeRF-VSQA.

The macro-component ablation shows that adapting both the visual tower and language model is stronger than adapting either alone, and that the reference-aware cue improves SRCC by approximately 0.8–1.0% on the in-domain benchmark. The LightGaussian analysis also reports monotonic MOS declines as compression becomes stronger, with pruning generally offering a more favorable storage-quality trade-off than the tested color quantization setting.

figure:teasor1

Caption: Overview of the proposed benchmark and model. Left: 3DGS-VBench+, a multi-dimensional subjective VQA benchmark for compressed 3DGS. Right: 3DGSV-Assessor scores the overall, geometry, and color quality of a rendered video, exploiting a training-viewpoint reference when available and operating no-reference otherwise.

Why It Matters: It summarizes the benchmark outputs and the assessor's optional-reference design.

fig:real_dataset_exp

Caption: Statistical analysis of the 11 real-world scenes in 3DGS-VBench+ from content complexity and quality perspectives. (a) Scene distribution across SI and TI dimensions. (b)-(d) MOS distribution for overall quality, geometry quality, and color quality.

Why It Matters: It shows the real-world content and perceptual-score coverage used to support dataset diversity.

fig:syn_dataset_exp

Caption: Statistical analysis of the 22 synthetic scenes in 3DGS-VBench+ from content complexity and quality perspectives. (a) Scene distribution across SI and TI dimensions. (b)-(d) MOS distribution for overall quality, geometry quality, and color quality.

Why It Matters: It shows the synthetic content distribution and the coverage of the three subjective quality dimensions.

fig:mos-size

Caption: MOS vs. Storage (MB) for LightGaussian on representative real-world and synthetic scenes in 3DGS-VBench+. (a) 5 CLs of prune for real-world scenes. (b) 5 CLs of (c-ratio, c-size) for real-world scenes. (c) 5 CLs of prune for synthetic scenes. (d) 5 CLs of (c-ratio, c-size) for synthetic scenes.

Why It Matters: It connects compression settings, storage, and perceived quality, including different behavior for real and synthetic scenes.

fig:method

Caption: Framework of 3DGSV-Assessor, an LMM-based method for all-in-one, reference-flexible 3DGS VQA. Left: Stage~1 is an LMM-based no-reference regressor that scores all three dimensions jointly. Right: Stage~2 encodes a captured view and same-scene videos into a reference-agreement cue, trained by ranking to pull higher-rated renderings closer to the reference. Middle: Stage~3 turns the cue into a bounded gate that refines the Stage-1 representation when a reference is available and is bypassed otherwise, so a single model serves both RR and NR assessment.

Why It Matters: It clarifies how the no-reference regressor, reference cue, and bounded fusion stage share one assessment framework.

🔮 Conclusion

The paper presents a substantially expanded 3DGS video-quality benchmark and a model designed around the practical reference conditions of novel-view synthesis. The benchmark expands coverage across scene types, compression mechanisms, distortion strengths, and three perceptual dimensions. The assessor's shared representation and separate heads provide joint scoring, while the optional reference gate improves performance without requiring the reference to enter the LMM generation context.

The reported results support the central design: explicit continuous regression is more perceptually sensitive than direct textual scoring alone, adapting both visual and language components is beneficial, and a same-scene reference cue provides additional value, particularly in cross-dataset evaluation.

🛠️ Future Research Improvements

The next technical priority is to test robustness beyond the prescribed orbital trajectories and the single captured-view reduced-reference setting. Evaluations could vary the reference viewpoint, reference count, camera motion, scene scale, and the balance between geometry-heavy and color-heavy distortions while keeping the three MOS targets separate.

For deployment, the method would benefit from explicit measurements of inference latency, frame-token budget, memory use, and quality degradation under smaller backbones or fewer sampled frames. A further useful study would connect the three predicted dimensions directly to compression optimization and determine whether dimension-aware control improves storage-quality trade-offs over optimizing overall MOS alone.

🏭 Potential Industry Use Scenarios

A 3DGS compression service could use the three scores to select operating points that preserve geometry while allowing more aggressive color compression, or to reject models whose overall quality is acceptable but whose geometry quality is poor. The reported LightGaussian analysis suggests that such dimension-aware decisions can expose different storage-quality trade-offs for pruning and color quantization.

A rendering platform could run the no-reference mode on arbitrary user-generated camera paths and activate reduced-reference refinement when one captured training view is available. This supports asset acceptance testing, regression monitoring after compression, and quality-aware cataloging of interactive 3D scenes without requiring an aligned reference video.

💬 Critical Analysis

The strongest aspect is the alignment between the problem setting and the model interface. The benchmark explicitly captures geometry and color degradation, while the assessor separates those dimensions and accepts a weak reference that is realistically available during 3DGS reconstruction. The controlled compression settings and the 33-scene mixture make the dataset more useful for model training than a small collection of isolated distortions.

The results are compelling but should be interpreted with the evaluation protocol in mind. The largest gains are measured on within-scene splits of the authors' benchmark, and the external datasets are evaluated zero-shot after training on 3DGS-VBench+. The reference cue is trained with same-scene relative ordering, which is well matched to the available supervision but leaves open how stable absolute score calibration is across substantially different scene distributions.

Reproduction requires more than implementing the final gate: the benchmark construction, compression parameter calibration, video trajectories, subjective-label normalization, frame sampling, LoRA setup, and split protocol all influence the result. The main body gives substantial training detail, but it does not establish runtime cost or production-scale throughput, so those measurements should be treated as necessary follow-up evidence before deployment.