Repository navigation
Conversation
AdaptiveProjectedGuidance created its MomentumBuffer once in __init__, so a guider reused across samples carried momentum from one sample into the next. Add reset(), call it from DPMS(), and leave the buffer unset when the momentum is None, as the docstring says. Refs NVlabs#597
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #597.
AdaptiveProjectedGuidancenow clears its momentum state at the start of each sample, andadaptive_projected_guidance_momentum=Noneturns momentum off as the docstring says. The change is indiffusion/guiders/adaptive_projected_guidance.pyanddiffusion/scheduler/dpm_solver.py, plus a newtests/test_adaptive_projected_guidance.py.Why
MomentumBufferonce in__init__.inference_sana_video.pybuilds one guider for the whole run and passes it to everyDPMS(...)call, so each video started from the momentum of the previous one.__init__always builtMomentumBuffer(None)for the defaultNone, so the first call raised aTypeError.The fix adds a
reset()method:__init__calls it, and theDPMS()factory callsapg.reset()before it builds the solver. The script callsDPMS()once per video, so the momentum still runs across the steps of one video, as before. diffusers'AdaptiveProjectedGuidancealso creates a new buffer at step 0 of each run.How I found it
I was reading the guider and saw that
momentum_bufferis set in__init__and only updated after that. I then checked how the video script creates and reuses it. See #597 for a short script.Where this can show up in practice
inference_sana_video.py --guidance_type adaptive_projected_guidance --sampling_algo flow_dpm-solverwith more than one prompt. Every video after the first came out slightly different from a run with that prompt alone. That makes seeds harder to reproduce and comparisons between prompt lists inexact. My estimate; I have not seen a field report.Compatibility
guider.reset().FlowEuleris not changed. Its APG branch stops earlier with a shape error attimestep * noise_pred, so I could not test a reset there. With--bsgreater than 1, the APG branch indiffusion/model/dpm_solver.pystops with a shape error too. Both are outside this PR.Testing
tests/test_adaptive_projected_guidance.py, three tests on CPU:reset()gives the same output as a new guider.momentum=Noneleaves the buffer unset and gives the same output on repeated calls.DPMS(...)samples with one guider, the same toy model, prompt and noise, give the same latents.mainall three fail (AttributeError,AssertionError, and latents that differ by 5e-4). With the fix all three pass.diffusionpackage__init__files, since my machine does not have the full model stack. I did not run the other tests or a real checkpoint.black25.12.0 andisort7.0.0 (the versions in.pre-commit-config.yaml) report no changes for the three files.macOS arm64, CPU only, Python 3.12.13, torch 2.15.0, diffusers 0.43.0.