Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix aclnnRepeatInterleaveIntWithDim error on NPU for get_1d_rotary_pos_embed #10820

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ZhengKai91
Copy link

@ZhengKai91 ZhengKai91 commented Feb 18, 2025

What does this PR do?

While running diffusers on NPU, the following error was encountered:
RuntimeError: call aclnnRepeatInterleaveIntWithDim failed, detail:EZ1001: 2025-02-18-15:02:05.865.233 self not implemented for DT_DOUBLE, should be in dtype support list [DT_UINT8,DT_INT8,DT_INT16,DT_INT32,DT_INT64,DT_BOOL,DT_FLOAT16,DT_FLOAT,DT_BFLOAT16,].
This error occurs specifically on NPU due to the unsupported data type DT_DOUBLE for the repeat_interleave operation. The error is triggered in the get_1d_rotary_pos_embed function in diffusers/models/embeddings.py
specifically in the following line:
freqs_cos = freqs.cos().repeat_interleave(2, dim=1).float() # [S, D]

Solution
To resolve this issue, I explicitly cast freqs to float before the cos operation. This ensures that the data type is supported by the repeat_interleave operation on NPU. The modified code is as follows:
freqs = torch.outer(pos, freqs).float() # type: ignore # [S, D/2]
if use_real and repeat_interleave_real:
# flux, hunyuan-dit, cogvideox
freqs_cos = freqs.cos().repeat_interleave(2, dim=1).float() # [S, D]
This change ensures compatibility with the data types supported by NPU

Testing
I have tested the modified code on NPU and confirmed that this change resolves the error without affecting the normal operation of other functionalities.
Documentation Updates
No documentation updates were made, as this is a bug fix specific to NPU and does not introduce new features or significant changes.
Dependencies
No new dependencies are introduced by this change.
Issue Fixed
Fixes the aclnnRepeatInterleaveIntWithDim error on NPU for the get_1d_rotary_pos_embed function.

Pre-Submission Checklist
[x] This PR fixes a bug specific to NPU.
[x] I have read the contributor guideline.
[x] I have read the philosophy doc.
[x] No discussion or approval via GitHub issue or forum was needed for this change.
[x] No documentation updates were necessary.
[x] No new tests were added, as this is a bug fix.
Reviewers
@yiyixuxu @sayakpaul

@ZhengKai91 ZhengKai91 marked this pull request as draft February 18, 2025 13:59
@ZhengKai91 ZhengKai91 marked this pull request as ready for review February 18, 2025 14:02
@ZhengKai91 ZhengKai91 marked this pull request as draft February 18, 2025 14:04
@ZhengKai91 ZhengKai91 marked this pull request as ready for review February 18, 2025 14:08
Copy link
Collaborator

@hlky hlky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ZhengKai91. We are aiming to improve alternative device type support and appreciate any contributions towards that goal!

@@ -1151,7 +1151,7 @@ def get_1d_rotary_pos_embed(
/ (theta ** (torch.arange(0, dim, 2, dtype=freqs_dtype, device=pos.device)[: (dim // 2)] / dim))
/ linear_factor
) # [D/2]
freqs = torch.outer(pos, freqs) # type: ignore # [S, D/2]
freqs = torch.outer(pos, freqs).float() # type: ignore # [S, D/2]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
freqs = torch.outer(pos, freqs).float() # type: ignore # [S, D/2]
freqs = torch.outer(pos, freqs) # type: ignore # [S, D/2]
is_npu = freqs.device.type == "npu"
if is_npu:
freqs = freqs.float()

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants