1. 25 Jan, 2023 2 commits
    • brkirch's avatar
      Add UI setting for upcasting attention to float32 · e3b53fd2
      brkirch authored
      Adds "Upcast cross attention layer to float32" option in Stable Diffusion settings. This allows for generating images using SD 2.1 models without --no-half or xFormers.
      
      In order to make upcasting cross attention layer optimizations possible it is necessary to indent several sections of code in sd_hijack_optimizations.py so that a context manager can be used to disable autocast. Also, even though Stable Diffusion (and Diffusers) only upcast q and k, unfortunately my findings were that most of the cross attention layer optimizations could not function unless v is upcast also.
      e3b53fd2
    • brkirch's avatar
      Add option for float32 sampling with float16 UNet · 84d9ce30
      brkirch authored
      This also handles type casting so that ROCm and MPS torch devices work correctly without --no-half. One cast is required for deepbooru in deepbooru_model.py, some explicit casting is required for img2img and inpainting. depth_model can't be converted to float16 or it won't work correctly on some systems (it's known to have issues on MPS) so in sd_models.py model.depth_model is removed for model.half().
      84d9ce30
  2. 24 Jan, 2023 14 commits
  3. 23 Jan, 2023 24 commits