WeatherEdit

Controllable Weather Editing with 4D Gaussian Field

Chenghao Qian1,*, Wenjing Li1,†, Yuhu Guo2, Gustav Markkula1
University of Leeds1, Carnegie Mellon University2
Paper Code

Controllable Weather Type

Multi-view with 4D Editing

Controllable Weather Severity

Abstract

In this work, we present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects with controllable types and severity in 3D scenes. Our approach is structured into two key components: weather background editing and weather particle construction. For weather background editing, we introduce an all-in-one adapter that integrates multiple weather styles into a single pretrained diffusion model, enabling the generation of diverse weather effects in 2D image backgrounds. During inference, we design a Temporal-View (TV-) attention mechanism that follows a specific order to aggregate temporal and spatial information, ensuring consistent editing across multi-frame and multi-view images. To construct the weather particles, we first reconstruct a 3D scene using the edited images and then introduce a dynamic 4D Gaussian field to generate snowflakes, raindrops and fog in the scene. The attributes and dynamics of these particles are precisely controlled through physical-based modelling and simulation, ensuring realistic weather representation and flexible severity adjustments. Finally, we integrate the 4D Gaussian field with the 3D scene to render consistent and highly realistic weather effects. Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity, highlighting its potential for autonomous driving simulation in adverse weather.

Method Overview

Synthetic Image

(a) Weather Background Editing: An all-in-one adapter enables the synthesis of diverse weather effects (e.g., snow, rain, fog) from text prompts and segmentation maps. Inference takes multi-frame and multi-view images as input, using temporal-view (TV) attention to ensure consistency. (b) Weather Particle Construction: A 4D Gaussian field, initialized from 3D Gaussians, undergoes attribute modeling and dynamic simulation, allowing user-controlled weather intensity (e.g., light, moderate, heavy). Local field alignment ensures seamless integration of the reconstructed 3D scene with dynamic weather effects.

BibTeX


@article{qian2025wedit,
      title={WeatherEdit: Controllable Weather Editing with 4D Gaussian Field}, 
      author={Chenghao Qian and Wenjing Li and Yuhu Guo and Gustav Markkula},
      year={2025},
      eprint={2505.20471},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2505.20471},}