Repository navigation
Conversation
app/sana_pipeline_inpaint.py calls scheduler.sample_mask(), but the method was dropped from diffusion/model/dpm_solver.py when sample_frame_aware() was added, so the flow_dpm-solver inpaint path stops with AttributeError. Bring back the method as it was in 48ec430. inverse() and the multistep updates it uses are unchanged since then. Refs NVlabs#356
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #356.
app/sana_pipeline_inpaint.pycallsscheduler.sample_mask(...)on theflow_dpm-solverpath.DPM_Solverno longer has that method, so inpainting with that sampler stops with:This PR adds
sample_mask()back todiffusion/model/dpm_solver.py, unchanged from 48ec430 (the inpainting PR, #296).Why
The method was added in #296 and dropped in #313, when
sample_frame_aware()was added. Nothing else calls it, so the inpaint app was the only thing left without it. The parts it uses (inverse(),get_time_steps(),multistep_dpm_solver_update()) are still there.inverse()is the same as in 48ec430.I put it back as it was, rather than rewriting it, so the inpaint output should match what #296 gave.
How I found it
I read the open issue and checked the call against the solver.
git log -S'def sample_mask'shows the two commits above.Where this can show up in practice
Anyone who runs the inpaint demo (
app/app_sana_inpaint.py) orSanaPipelineInpaintwithflow_dpm-solver. That is the defaultvis_sampler, and the demo's default config uses it. The run stops at the sampling step with the error above.flow_eulerdoes not use this method. My estimate; I have not seen a field report.Testing
A small CPU script loads
diffusion/model/dpm_solver.pyon its own, with a toy noise model and the same arguments the inpaint pipeline passes (order=2,skip_type="time_uniform_flow",method="multistep",flow_shift=3.0, 10 steps, latents[1, 4, 8, 8]):main(27303b8):AttributeError: 'DPM_Solver' object has no attribute 'sample_mask'.latent_maskall zeros: the output equalsimg_latentexactly (max abs difference 0.0).latent_mask= 1 on the left half: the right half equalsimg_latent. The left half is sampled and finite.I did not run the full inpaint pipeline with a checkpoint, since I do not have the weights on this machine.
macOS arm64, CPU only, Python 3.11, torch 2.15.0.