• brkirch's avatar
    Add UI setting for upcasting attention to float32 · e3b53fd2
    brkirch authored
    Adds "Upcast cross attention layer to float32" option in Stable Diffusion settings. This allows for generating images using SD 2.1 models without --no-half or xFormers.
    
    In order to make upcasting cross attention layer optimizations possible it is necessary to indent several sections of code in sd_hijack_optimizations.py so that a context manager can be used to disable autocast. Also, even though Stable Diffusion (and Diffusers) only upcast q and k, unfortunately my findings were that most of the cross attention layer optimizations could not function unless v is upcast also.
    e3b53fd2
Name
Last commit
Last update
.github Loading commit data...
configs Loading commit data...
embeddings Loading commit data...
extensions Loading commit data...
extensions-builtin Loading commit data...
html Loading commit data...
javascript Loading commit data...
localizations Loading commit data...
models Loading commit data...
modules Loading commit data...
scripts Loading commit data...
test Loading commit data...
textual_inversion_templates Loading commit data...
.gitignore Loading commit data...
.pylintrc Loading commit data...
CODEOWNERS Loading commit data...
LICENSE.txt Loading commit data...
README.md Loading commit data...
environment-wsl2.yaml Loading commit data...
launch.py Loading commit data...
requirements.txt Loading commit data...
requirements_versions.txt Loading commit data...
screenshot.png Loading commit data...
script.js Loading commit data...
style.css Loading commit data...
v2-inference-v.yaml Loading commit data...
webui-macos-env.sh Loading commit data...
webui-user.bat Loading commit data...
webui-user.sh Loading commit data...
webui.bat Loading commit data...
webui.py Loading commit data...
webui.sh Loading commit data...