Skip to content

Reset the APG momentum buffer before each sample - #598

Open
0z5a wants to merge 1 commit into
NVlabs:mainfrom
0z5a:apg-reset-momentum
Open

0z5a wants to merge 1 commit into
NVlabs:mainfrom
0z5a:apg-reset-momentum

Conversation

@0z5a

@0z5a 0z5a commented Nov 19, 2026

Copy link
Copy Markdown

Summary

Refs #597.

AdaptiveProjectedGuidance now clears its momentum state at the start of each sample, and adaptive_projected_guidance_momentum=None turns momentum off as the docstring says. The change is in diffusion/guiders/adaptive_projected_guidance.py and diffusion/scheduler/dpm_solver.py, plus a new tests/test_adaptive_projected_guidance.py.

Why

  • The guider built its MomentumBuffer once in __init__. inference_sana_video.py builds one guider for the whole run and passes it to every DPMS(...) call, so each video started from the momentum of the previous one.
  • __init__ always built MomentumBuffer(None) for the default None, so the first call raised a TypeError.

The fix adds a reset() method:

def reset(self):
    if self.adaptive_projected_guidance_momentum is None:
        self.momentum_buffer = None
    else:
        self.momentum_buffer = MomentumBuffer(self.adaptive_projected_guidance_momentum)

__init__ calls it, and the DPMS() factory calls apg.reset() before it builds the solver. The script calls DPMS() once per video, so the momentum still runs across the steps of one video, as before. diffusers' AdaptiveProjectedGuidance also creates a new buffer at step 0 of each run.

How I found it

I was reading the guider and saw that momentum_buffer is set in __init__ and only updated after that. I then checked how the video script creates and reuses it. See #597 for a short script.

Where this can show up in practice

inference_sana_video.py --guidance_type adaptive_projected_guidance --sampling_algo flow_dpm-solver with more than one prompt. Every video after the first came out slightly different from a run with that prompt alone. That makes seeds harder to reproduce and comparisons between prompt lists inexact. My estimate; I have not seen a field report.

Compatibility

  • The first video of a run is unchanged. Later videos now match what a new process would give for the same prompt and seed.
  • Code that calls the guider directly and wants a fresh start can call guider.reset().
  • FlowEuler is not changed. Its APG branch stops earlier with a shape error at timestep * noise_pred, so I could not test a reset there. With --bs greater than 1, the APG branch in diffusion/model/dpm_solver.py stops with a shape error too. Both are outside this PR.

Testing

  • New tests/test_adaptive_projected_guidance.py, three tests on CPU:
    • reset() gives the same output as a new guider.
    • momentum=None leaves the buffer unset and gives the same output on repeated calls.
    • Two DPMS(...) samples with one guider, the same toy model, prompt and noise, give the same latents.
  • On main all three fail (AttributeError, AssertionError, and latents that differ by 5e-4). With the fix all three pass.
  • I ran them with stub modules for the diffusion package __init__ files, since my machine does not have the full model stack. I did not run the other tests or a real checkpoint.
  • black 25.12.0 and isort 7.0.0 (the versions in .pre-commit-config.yaml) report no changes for the three files.

macOS arm64, CPU only, Python 3.12.13, torch 2.15.0, diffusers 0.43.0.

AdaptiveProjectedGuidance created its MomentumBuffer once in __init__,
so a guider reused across samples carried momentum from one sample into
the next. Add reset(), call it from DPMS(), and leave the buffer unset
when the momentum is None, as the docstring says.

Refs NVlabs#597

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant