Authors:
(1) Hoon Kim, Beeble AI, and contributed equally to this work;
(2) Minje Jang, Beeble AI, and contributed equally to this work;
(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;
(4) Jisoo Lee, Beeble AI, and contributed equally to this work;
(5) Donghyun Na, Beeble AI, and contributed equally to this work;
(6) Sanghyun Woo, New York University, and contributed equally to this work.
We present a detailed video demonstration of our SwitchLight framework. Initially, we use real-world videos from Pexels [1] to showcase its robust generalizability and practicality. Then, for state-of-the-art comparisons, we utilize the FFHQ dataset [25] to demonstrate its advanced relighting capabilities over previous methods. The presentation includes several key components:
De-rendering: This stage demonstrates the extraction of normal, albedo, roughness, and reflectivity attributes from any given image, a process known as inverse rendering.
Neural Relighting: Leveraging these intrinsic properties, our system adeptly relights images to align with a new, specified target lighting environment.
Real-time Physically Based Rendering (PBR): Utilizing the Three.js framework and integrating derived intrinsic properties with the Cook-Torrance reflectance model, we facilitate real-time rendering. This enables achieving 30 fps on a MacBook Pro with an Apple M1 chip (8-core CPU and 8-core GPU) and 16 GB of RAM.
Copy Light: Leveraging SwitchLight’s ability to predict lighting conditions of a given input image, we explore an intriguing application. This process involves two images, a source and a reference. We first extract their intrinsic surface attributes and lighting conditions. Then, by combining the source intrinsic attributes with the reference lighting condition, we generate a new, relit image. In this image, the source foreground remains unchanged, but its lighting is altered to match that of the reference image.
State-of-the-Art Comparisons: We benchmark our framework against leading methods, specifically Total Relight [34] and Lumos [52], to highlight substantial performance improvements over these approaches.
/cdn.vox-cdn.com/uploads/chorus_asset/file/25784221/247333_EOY_Package_Check_In_CVirginia_PRIME.jpg)
