Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Contribute to GitLab
Sign in / Register
Toggle navigation
S
stable-diffusion-webui
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Administrator
stable-diffusion-webui
Commits
17b60490
Unverified
Commit
17b60490
authored
Sep 18, 2022
by
AUTOMATIC1111
Committed by
GitHub
Sep 18, 2022
Browse files
Options
Browse Files
Download
Plain Diff
Merge pull request #635 from C43H66N12O12S2/attention
Move scale multiplication to the front
parents
8ff6f093
18d6fe43
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
sd_hijack.py
modules/sd_hijack.py
+2
-2
No files found.
modules/sd_hijack.py
View file @
17b60490
...
...
@@ -50,7 +50,7 @@ def split_cross_attention_forward(self, x, context=None, mask=None):
q_in
=
self
.
to_q
(
x
)
context
=
default
(
context
,
x
)
k_in
=
self
.
to_k
(
context
)
k_in
=
self
.
to_k
(
context
)
*
self
.
scale
v_in
=
self
.
to_v
(
context
)
del
context
,
x
...
...
@@ -85,7 +85,7 @@ def split_cross_attention_forward(self, x, context=None, mask=None):
slice_size
=
q
.
shape
[
1
]
//
steps
if
(
q
.
shape
[
1
]
%
steps
)
==
0
else
q
.
shape
[
1
]
for
i
in
range
(
0
,
q
.
shape
[
1
],
slice_size
):
end
=
i
+
slice_size
s1
=
einsum
(
'b i d, b j d -> b i j'
,
q
[:,
i
:
end
],
k
)
*
self
.
scale
s1
=
einsum
(
'b i d, b j d -> b i j'
,
q
[:,
i
:
end
],
k
)
s2
=
s1
.
softmax
(
dim
=-
1
,
dtype
=
q
.
dtype
)
del
s1
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment