SCIENCE@home

Real-Time Physically-Based Relighting and Composition of Radiance Fields with Proxy Meshes

Yang Xu,Jinyang Bo,Junfeng Wang,He Tian,Yuhe Zhang,Kang Li,Guohua Geng

Published 2025 in International Symposium on Mixed and Augmented Reality

ABSTRACT

Radiance fields, such as neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS), are the new primitives to represent 3D scenes. Relighting and composition of radiance fields are critical for modeling the complex 3D world in computer graphics. However, it is difficult to relight and composite radiance fields because traditional physically-based rendering techniques, such as path tracing, cannot be directly applied to radiance fields. We propose a physically-based relighting and composition method for radiance fields with proxy meshes. A unified framework is presented to enable us to use radiance fields as the traditional assets in computer graphics. We generate proxy meshes of the radiance fields by reconstructing the geometries of the scenes using Gaussian-based surface reconstruction and the materials using physically-based differentiable rendering. We leverage differential rendering, which is previously used in augmented reality (AR) and mixed reality (MR), to evaluate the radiance change on the proxy meshes introduced by the changing lighting condition, the inserted radiance fields, or the inserted mesh models. Proxy meshes can help us utilize hardwareaccelerated ray tracing to perform real-time path tracing. Experimental results show that our method outperforms the baselines in terms of relighting performance and can achieve photorealistic relighting and composition of radiance fields in real-time.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-59 of 59 references · Page 1 of 1

CITED BY

  • No citing papers are available for this paper.

Showing 0-0 of 0 citing papers · Page 1 of 1