Experimental Offline Standalone Mac Installer for Stable Diffusion Web UI (unofficial) 20230411

[brkirch]

Changelog

• Based on the brkirch/stable-diffusion-webui branch, which adds to AUTOMATIC1111@a9fed7c in order to use Gradio 3.16.2 which has better performance and stability
• Adds option to remove negative conditioning at low sigma values ((Optimization) Option to remove negative conditioning at low sigma values AUTOMATIC1111/stable-diffusion-webui#9177) and defaults to a negative guidance minimum sigma of 3. Also adds this information to infotext.
• Adds token merging (Integrate optional speed and memory improvements by token merging (via dbolya/tomesd) AUTOMATIC1111/stable-diffusion-webui#9256). Defaults to using token merging for hires fix.
• Uses custom PyTorch build (brkirch/pytorch@df3b3f9) based on kulinseth/pytorch. Without this, removing negative conditioning at low sigma values doesn't work correctly. This PyTorch build may also be less likely to produce NaN related errors. Unfortunately, this build also currently has bugs that cause occasional (random) Python crashes.
• Add option to disable removing negative conditioning at low sigma values when token merging is active. This is enabled by default.
• Modify sub-quadratic attention default value to use a fixed amount of RAM (just enough for 512x512 generations without chunking) but no more than that. In cases that this reduces RAM usage, it doesn't appear to have a notable effect on performance.
• Implement torch_gc() for MPS. This should reduce overall memory usage.
• Added modified DPM++ 2M sampler by @hallatore
• Add most of the recently merged PRs from AUTOMATIC1111/stable-diffusion-webui
• Add update option to installer. Any local changes made will be carried over to the updated files if possible. If changes can't be carried over then those changes will be left stashed (with git stash).
• Move changed defaults to webui-macos-env.sh. This will allow for updating to work in the future without creating conflicts in the webui-user.sh file.

---

This discussion was created from the release Experimental Offline Standalone Mac Installer for Stable Diffusion Web UI (unofficial) 20230411.

[jrittvo]

On a clean install it starts fine. No issues getting extensions. With tomesd enabled, the speed increases are as expected. Overall speed matches exactly what I get on my regular Auto1111 install with tomesd added in there manually. That install also uses your new k-diffusion git commit and some of your other new tweaks, but uses torch 2.1.0.dev20230407 and gradio: 3.23.0.

In this new 0411 build, there are a number of non-fatal warnings thrown when enabling Hires. fix, but the generate button still starts generation which completes (with tomesd and hires fix actually working). There are also warnings when enabling Restore faces, but those crash the app. They seem to be related to some of the changes made for tomesd, but they trigger even with tomesd disabled.

Do you want specific bug reports for this version, or is it more a proof of concept while you fine tune and you already know what the issues are?

[sonluong1987]

Hi @jrittvo, are you able to get hires fix working consistently? I get NaNs / black images or this error quite often:

Traceback (most recent call last): File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/call_queue.py", line 56, in f res = list(func(*args, **kwargs)) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/call_queue.py", line 37, in f res = func(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/txt2img.py", line 56, in txt2img processed = process_images(p) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/processing.py", line 519, in process_images res = process_images_inner(p) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/processing.py", line 674, in process_images_inner samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/processing.py", line 966, in sample samples = self.sampler.sample_img2img(self, samples, noise, conditioning, unconditional_conditioning, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_samplers_kdiffusion.py", line 346, in sample_img2img samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs)) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_samplers_kdiffusion.py", line 247, in launch_sampling return func() File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_samplers_kdiffusion.py", line 346, in <lambda> samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs)) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/k-diffusion/k_diffusion/sampling.py", line 623, in sample_dpmpp_2m_v2 denoised = model(x, sigmas[i] * s_in, **extra_args) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_samplers_kdiffusion.py", line 155, in forward x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b])) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/k-diffusion/k_diffusion/external.py", line 114, in forward eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/k-diffusion/k_diffusion/external.py", line 140, in get_eps return self.inner_model.apply_model(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_hijack_utils.py", line 17, in <lambda> setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs)) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_hijack_utils.py", line 26, in __call__ return self.__sub_func(self.__orig_func, *args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/modules/sd_hijack_unet.py", line 45, in apply_model return orig_func(self, x_noisy.to(devices.dtype_unet), t.to(devices.dtype_unet), cond, **kwargs).float() File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, in apply_model x_recon = self.model(x_noisy, t, **cond) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 1335, in forward out = self.diffusion_model(x, t, context=cc) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1538, in _call_impl result = forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 797, in forward h = module(h, emb, context) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 84, in forward x = layer(x, context) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/modules/attention.py", line 334, in forward x = block(x, context=context[i]) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/modules/attention.py", line 269, in forward return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 121, in checkpoint return CheckpointFunction.apply(func, len(inputs), *args) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/torch/autograd/function.py", line 506, in apply return super().apply(*args, **kwargs) # type: ignore[misc] File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 136, in forward output_tensors = ctx.run_function(*ctx.input_tensors) File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/tomesd/patch.py", line 54, in _forward x = u_a(self.attn1(m_a(self.norm1(x)), context=context if self.disable_self_attn else None)) + x File "/Volumes/Work/SDiffusion/stable-diffusion-webui4/venv/lib/python3.10/site-packages/tomesd/merge.py", line 104, in merge dst = dst.scatter_reduce(-2, dst_idx.expand(n, r, c), src, reduce=mode) RuntimeError: index -1 is out of bounds for dimension 1 with size 6144

[jrittvo]

It throws a bunch of errors, but it still works as expected when I click generate after the errors. The errors I get seem to be warnings and not fatal crashes. This is -without- Restore faces enabled. That setting stops generation after the Hires fix step completes but before Restore faces runs, and no final file is saved. (You can't always trust the preview to show if a generation was successful. Use autosave images, because this version has images saved that don't make to final preview.)

The index -1 is out bounds for dimension 1 with size xxxx I think is an error when using SD 2.1 type models without the Upcast cross attention layer to float32 box checked, up by the model selection at the top of the page.

?? @brkirch ?? The system seems to know it is an SD 2.1 model on its own because it changes to v-prediction, so why not make the toggle of Upcast cross attention automatic as well?

[brkirch]

?? @brkirch ?? The system seems to know it is an SD 2.1 model on its own because it changes to v-prediction, so why not make the toggle of Upcast cross attention automatic as well?

This was originally my plan, but the detection in its current state can only tell if the model is 2.X, not 2.1 specifically. Thinking about it a little more though, it won't really matter much if SD 2.0 v-prediction models need upcasting attention disabled manually as leaving it on unnecessarily won't do any harm anyway (not to mention SD 2.0 is likely not used much anyway - AFAIK most models using SD 2 are based on SD 2.1 instead of SD 2.0). I'll go ahead and add it, at least it should minimize the number of NaN errors.

Also I figured out what is likely the other most common cause of Unet NaN errors, and I have a fix for it. The fix makes removing negative conditioning at low sigma values work without the problematic commits I had previously included in PyTorch to get it working. So the next experimental build should likely work a lot better, and without a lot of the stability issues.

[AnotherUser33dot]

[jrittvo]

I wonder if that was really a fix, or if the underlying issue is something else intermittent/random. I ask because I don't think this install uses diffusers. (I don't see diffusers in either of the python site-packages folders.) I think it uses k-diffusion (in the repositories folder. Maybe there is some sort of aliasing going on between the 2 names and the pip update actually did change the pipeline. ?? @brkirch ??