[SYCL] Enable level zero v2 in DPC++ #16656

omarahmed1111 · 2025-01-16T10:30:02Z

This makes a simple way to use the experimental level_zero_v2 in dpc++. Fixes #16613 .

pbalcer · 2025-01-16T11:36:53Z

My opinion is that we should build and ship both adapters by default, and make it togglable which one is used at runtime through an env variable (like I suggested here).
Having both is often useful for performance comparisons, or simply to have a fallback if one of the adapters is not behaving correctly.

omarahmed1111 · 2025-01-16T14:51:34Z

My opinion is that we should build and ship both adapters by default, and make it togglable which one is used at runtime through an env variable (like I suggested here). Having both is often useful for performance comparisons, or simply to have a fallback if one of the adapters is not behaving correctly.

Good point! changed that.

jbrodman · 2025-01-16T17:16:46Z

By default, I think we should only build 1 adapter. We don't want to confuse users by enumerating the same device even more times than we already do (OpenCL vs Level Zero). I think it would be nice to support building both and toggling for the reasons stated above, but I'd also like the option to ONLY build v1 or v2.

pbalcer · 2025-01-17T09:17:36Z

By default, I think we should only build 1 adapter. We don't want to confuse users by enumerating the same device even more times than we already do (OpenCL vs Level Zero). I think it would be nice to support building both and toggling for the reasons stated above, but I'd also like the option to ONLY build v1 or v2.

The current proposal, and, from what I can tell, the implementation, is that both adapters will be physically present in the libs directory, however, only one of them will be active at a time. There will be a new environment variable (UR_ADAPTER_LEVEL_ZERO_V2=1, although, long-term, I'd have preferred for this to be an option for ONEAPI_DEVICE_SELECTOR), that users will be able to set to toggle which adapter is used. This ensures that L0 devices are not enumerated twice, but makes it very simple for users to experiment with the new adapter.

Having to build SYCL from scratch with a special option might be challenging for developers of higher-level frameworks or applications that usually use a prebuilt compiler.

sarnex · 2025-01-17T16:08:28Z

buildbot/configure.py

@@ -68,6 +68,7 @@ def do_configure(args):

    if sys.platform != "darwin":
        sycl_enabled_backends.append("level_zero")
+        sycl_enabled_backends.append("level_zero_v2")


Couple of comments

Can you explain what L0 V2 is and how it releases to the current L0/L0 adapter

I agree with James, we shouldn't build two plugins by default. If we have some configure switch to build two, that's fine.

README. In short, it's a new L0 adapter with re-implemented performance-critical APIs. See here for performance validation results.

Why though? As I noted, with the current implementation, users won't be affected by the inclusion of the two adapters. And not including it by default would mean that it's going to be much more difficult for application and framework developers to experiment with the new adapter, which is the whole point of including it in SYCL right now in the first place. In practice, it will likely mean we might need to provide the users that want to use it an alternative prebuilt compiler package.

@igchor ping

Out of curiosity, why "v2" instead of incremental refactoring?

Got it, thanks

At this point it seems like the v2 adapter is only intended to be used by DPCPP/UR developers, and IMO features like that should not be built by default. If the feature is ready enough for downstream clients, should we switch over by default?

Out of curiosity, why "v2" instead of incremental refactoring?

The existing L0 adapter code base has accumulated a lot of complexity over the years as L0 and the driver evolved. It has become very difficult to change. For example, there are 4 major queue modes that the adapter supports - "batched in order", "batched out of order", "immediate in order", "immediate out of order". In the existing adapter, they are all implemented together, with various conditions sprinkled all throughout to make it work. In practice, batched mode is all but legacy for 99% of scenarios, and "in order" and "out of order" L0 paths are so different (different types of L0 events, different way of synchronization), that there's not much to gain from sharing all the code between them.
Refactoring this incrementally, without breaking existing code, would be challenging. So the team has decided that a partial re-implementation (we still reuse all that's practical) of the performance critical paths is the best path forward. This also means that we have the opportunity to easily leverage new driver features and do a clean break away from the accumulated legacy tech debt.

At this point it seems like the v2 adapter is only intended to be used by DPCPP/UR developers, and IMO features like that should not be built by default. If the feature is ready enough for downstream clients, should we switch over by default?

In terms of quality, the v2 adapter has higher UR CTS passrate than the current one, and passes the vast majority of SYCL e2e tests (big exception being tests that hard-code exact sequence of operations expected from the adapter). However, this is still work-in-progress.

sycl/cmake/modules/FetchUnifiedRuntime.cmake

igchor · 2025-01-29T21:59:28Z

@omarahmed1111 will this approach enable us to mark specific e2e tests with XFAIL for v2 only, or would that require some extra changes?

omarahmed1111 · 2025-02-21T14:48:03Z

@intel/dpcpp-devops-reviewers Gentle ping on this!

pbalcer · 2025-02-25T08:51:59Z

@intel/dpcpp-devops-reviewers please review. This is blocking our progress on properly enabling v2 in tests and the compiler packages.
Thanks!

aelovikov-intel

Please add links to the in-tree documentation describing rationale and short/mid/long-term plans for this.

igchor · 2025-02-25T21:14:35Z

Please add links to the in-tree documentation describing rationale and short/mid/long-term plans for this.

I believe @pbalcer explained the rationale in his previous comment. As for the plans, we want to provide v2 as an experimental adapter in the 2025.2 release to gather feedback from the users and, if needed, add missing features (v2 does not implement support for all env variables that are used to tweak the legacy adapter). In 2026.0, we plan to make the v2 adapter the default one for L0.

I guess we can add this information to https://github.com/intel/llvm/blob/sycl/unified-runtime/source/adapters/level_zero/v2/README.md

aelovikov-intel · 2025-02-25T21:42:27Z

I believe @pbalcer explained the rationale in his previous comment.

I didn't ask for the explanation itself, but to put the link to it as a comment in the code, so that it would be immediately available for anyone reading the code in future.

aelovikov-intel

Please add links to the in-tree documentation describing rationale and short/mid/long-term plans for this

You've only added the former and not the latter... Also, having links to a comments thread in PR review is weird at the very least.

Anyhow, someone else gave you an approval for the SYCL RT, so I'm only LGTM'ing the configure.py change following those pre-approved changes.

omarahmed1111 · 2025-02-27T19:18:25Z

Please add links to the in-tree documentation describing rationale and short/mid/long-term plans for this

You've only added the former and not the latter... Also, having links to a comments thread in PR review is weird at the very least.

Anyhow, someone else gave you an approval for the SYCL RT, so I'm only LGTM'ing the configure.py change following those pre-approved changes.

Apologies, I misunderstood that, I modified the comment to reference the page @igchor mentioned and added a small comment about the timeline. I didn't want to add more than that as I think L0_V2 team could reference this document explanations better. So, I just will leave this for the functionality and this small comment.

omarahmed1111 · 2025-02-28T11:23:47Z

@intel/llvm-gatekeepers Please merge, Thanks!

omarahmed1111 force-pushed the enable-level-zero-v2 branch from 32fded7 to 9bc55cd Compare January 16, 2025 10:31

omarahmed1111 temporarily deployed to WindowsCILock January 16, 2025 10:31 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock January 16, 2025 11:02 — with GitHub Actions Inactive

omarahmed1111 force-pushed the enable-level-zero-v2 branch from 9bc55cd to dd18cba Compare January 16, 2025 14:49

omarahmed1111 added 2 commits January 16, 2025 14:50

Enable level_zero_v2 on sycl

123099e

Update UR tag

cbbdcec

omarahmed1111 force-pushed the enable-level-zero-v2 branch from dd18cba to cbbdcec Compare January 16, 2025 14:51

omarahmed1111 temporarily deployed to WindowsCILock January 16, 2025 14:52 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock January 16, 2025 15:33 — with GitHub Actions Inactive

kbenzie mentioned this pull request Jan 17, 2025

Enable the registry of level_zero_v2 oneapi-src/unified-runtime#2571

Merged

omarahmed1111 marked this pull request as ready for review January 17, 2025 15:29

omarahmed1111 requested review from a team as code owners January 17, 2025 15:29

omarahmed1111 requested a review from againull January 17, 2025 15:29

sarnex reviewed Jan 17, 2025

View reviewed changes

omarahmed1111 added 2 commits January 29, 2025 11:06

Change to only build one adapter

436c0e6

Merge branch 'sycl' into enable-level-zero-v2

328b88f

omarahmed1111 temporarily deployed to WindowsCILock January 29, 2025 11:09 — with GitHub Actions Inactive

omarahmed1111 requested a review from pbalcer January 29, 2025 11:10

pbalcer reviewed Jan 29, 2025

View reviewed changes

sycl/cmake/modules/FetchUnifiedRuntime.cmake Outdated Show resolved Hide resolved

omarahmed1111 had a problem deploying to WindowsCILock January 29, 2025 12:04 — with GitHub Actions Failure

Remove the l0 envvar assign

a80036d

omarahmed1111 temporarily deployed to WindowsCILock January 29, 2025 15:49 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock January 29, 2025 16:53 — with GitHub Actions Inactive

kbenzie temporarily deployed to WindowsCILock February 14, 2025 11:41 — with GitHub Actions Inactive

omarahmed1111 requested a review from sarnex February 17, 2025 11:36

pbalcer mentioned this pull request Feb 20, 2025

[UR] option to build LLVM with UR LevelZero v2 adapter #17089

Closed

omarahmed1111 added 2 commits February 20, 2025 12:21

Merge branch 'sycl' into enable-level-zero-v2

ed555b4

Remove UR tag update

04a4d78

omarahmed1111 temporarily deployed to WindowsCILock February 20, 2025 12:25 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock February 20, 2025 13:08 — with GitHub Actions Inactive

omarahmed1111 changed the title ~~Enable level zero v2 in DPC++~~ [SYCL] Enable level zero v2 in DPC++ Feb 21, 2025

aelovikov-intel reviewed Feb 25, 2025

View reviewed changes

omarahmed1111 had a problem deploying to WindowsCILock February 26, 2025 11:24 — with GitHub Actions Error

Add a comment for the rationale of level_zero_v2

c4355e8

omarahmed1111 force-pushed the enable-level-zero-v2 branch from 3293119 to c4355e8 Compare February 26, 2025 12:33

omarahmed1111 temporarily deployed to WindowsCILock February 26, 2025 12:35 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock February 26, 2025 14:25 — with GitHub Actions Inactive

Merge branch 'sycl' into enable-level-zero-v2

9a1e811

omarahmed1111 requested a review from aelovikov-intel February 27, 2025 13:56

omarahmed1111 temporarily deployed to WindowsCILock February 27, 2025 13:57 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock February 27, 2025 14:41 — with GitHub Actions Inactive

aelovikov-intel approved these changes Feb 27, 2025

View reviewed changes

Add timeline plans for L0 v2

7bdb1e8

omarahmed1111 had a problem deploying to WindowsCILock February 27, 2025 17:38 — with GitHub Actions Error

omarahmed1111 temporarily deployed to WindowsCILock February 28, 2025 09:40 — with GitHub Actions Inactive

omarahmed1111 temporarily deployed to WindowsCILock February 28, 2025 10:09 — with GitHub Actions Inactive

dm-vodopyanov merged commit cc0b20f into intel:sycl Feb 28, 2025
29 of 30 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SYCL] Enable level zero v2 in DPC++ #16656

[SYCL] Enable level zero v2 in DPC++ #16656

omarahmed1111 commented Jan 16, 2025

pbalcer commented Jan 16, 2025 •

edited

Loading

omarahmed1111 commented Jan 16, 2025

jbrodman commented Jan 16, 2025

pbalcer commented Jan 17, 2025

sarnex Jan 17, 2025

pbalcer Jan 17, 2025 •

edited

Loading

aelovikov-intel Jan 17, 2025

sarnex Jan 17, 2025 •

edited

Loading

pbalcer Jan 17, 2025 •

edited

Loading

igchor commented Jan 29, 2025

omarahmed1111 commented Feb 21, 2025

pbalcer commented Feb 25, 2025

aelovikov-intel left a comment

igchor commented Feb 25, 2025

aelovikov-intel commented Feb 25, 2025

aelovikov-intel left a comment

omarahmed1111 commented Feb 27, 2025

omarahmed1111 commented Feb 28, 2025

[SYCL] Enable level zero v2 in DPC++ #16656

[SYCL] Enable level zero v2 in DPC++ #16656

Conversation

omarahmed1111 commented Jan 16, 2025

pbalcer commented Jan 16, 2025 • edited Loading

omarahmed1111 commented Jan 16, 2025

jbrodman commented Jan 16, 2025

pbalcer commented Jan 17, 2025

sarnex Jan 17, 2025

Choose a reason for hiding this comment

pbalcer Jan 17, 2025 • edited Loading

Choose a reason for hiding this comment

aelovikov-intel Jan 17, 2025

Choose a reason for hiding this comment

sarnex Jan 17, 2025 • edited Loading

Choose a reason for hiding this comment

pbalcer Jan 17, 2025 • edited Loading

Choose a reason for hiding this comment

igchor commented Jan 29, 2025

omarahmed1111 commented Feb 21, 2025

pbalcer commented Feb 25, 2025

aelovikov-intel left a comment

Choose a reason for hiding this comment

igchor commented Feb 25, 2025

aelovikov-intel commented Feb 25, 2025

aelovikov-intel left a comment

Choose a reason for hiding this comment

omarahmed1111 commented Feb 27, 2025

omarahmed1111 commented Feb 28, 2025

pbalcer commented Jan 16, 2025 •

edited

Loading

pbalcer Jan 17, 2025 •

edited

Loading

sarnex Jan 17, 2025 •

edited

Loading

pbalcer Jan 17, 2025 •

edited

Loading