Fix aclnnRepeatInterleaveIntWithDim error on NPU for get_1d_rotary_pos_embed #10820
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
While running diffusers on NPU, the following error was encountered:
RuntimeError: call aclnnRepeatInterleaveIntWithDim failed, detail:EZ1001: 2025-02-18-15:02:05.865.233 self not implemented for DT_DOUBLE, should be in dtype support list [DT_UINT8,DT_INT8,DT_INT16,DT_INT32,DT_INT64,DT_BOOL,DT_FLOAT16,DT_FLOAT,DT_BFLOAT16,].
This error occurs specifically on NPU due to the unsupported data type DT_DOUBLE for the repeat_interleave operation. The error is triggered in the get_1d_rotary_pos_embed function in diffusers/models/embeddings.py
specifically in the following line:
freqs_cos = freqs.cos().repeat_interleave(2, dim=1).float() # [S, D]
Solution
To resolve this issue, I explicitly cast freqs to float before the cos operation. This ensures that the data type is supported by the repeat_interleave operation on NPU. The modified code is as follows:
freqs = torch.outer(pos, freqs).float() # type: ignore # [S, D/2]
if use_real and repeat_interleave_real:
# flux, hunyuan-dit, cogvideox
freqs_cos = freqs.cos().repeat_interleave(2, dim=1).float() # [S, D]
This change ensures compatibility with the data types supported by NPU
Testing
I have tested the modified code on NPU and confirmed that this change resolves the error without affecting the normal operation of other functionalities.
Documentation Updates
No documentation updates were made, as this is a bug fix specific to NPU and does not introduce new features or significant changes.
Dependencies
No new dependencies are introduced by this change.
Issue Fixed
Fixes the aclnnRepeatInterleaveIntWithDim error on NPU for the get_1d_rotary_pos_embed function.
Pre-Submission Checklist
[x] This PR fixes a bug specific to NPU.
[x] I have read the contributor guideline.
[x] I have read the philosophy doc.
[x] No discussion or approval via GitHub issue or forum was needed for this change.
[x] No documentation updates were necessary.
[x] No new tests were added, as this is a bug fix.
Reviewers
@yiyixuxu @sayakpaul