Publications

A fixed classifier-free guidance scale cannot adapt to how the balance between condition alignment, realism, and diversity changes during sampling. UniGuide formulates guidance as a sequential decision problem and learns a preference-conditioned policy that selects separate guidance parameters for the denoised estimate and the noise residual.

The policy uses compact features describing the solver context, denoiser agreement, and changes in guidance effects. A preference vector lets the same policy explore different alignment–diversity trade-offs at inference time, without manual schedule design or retraining the diffusion backbone. Evaluation includes held-out alignment and diversity measures.

Standard diffusion posterior guidance differentiates an observation likelihood through the denoiser at every reverse step, adding substantial memory and runtime costs. DING instead evaluates the denoiser at an auxiliary reverse sample, decoupling its input from the state being corrected by the observation likelihood.

For masked linear observations, this likelihood surrogate yields a closed-form Gaussian update: missing coordinates follow the reference transition, while observed coordinates receive a likelihood correction. The Gaussian update is exact for the surrogate conditional, rather than the true posterior transition. The method is evaluated on zero-shot inpainting under practical compute and memory budgets.

This follow-up studies the approximation behind DING and extends its guidance principle to large-scale image and video editing. Text-conditioned edits are formulated through masked inpainting, combining a pretrained generative prior with a correction based on the observed content.

The approach applies guidance at inference time, without backpropagating through the frozen denoiser. This extends zero-shot inpainting to image and video editing without task-specific model training.