Skip to content

Commit d99970b

Browse files
authoredFeb 8, 2025··
Change llm getting started tutorial to use to_edge_transform_and_lower (#8325)
1 parent 513e92d commit d99970b

File tree

1 file changed

+22
-24
lines changed

1 file changed

+22
-24
lines changed
 

‎docs/source/llm/getting-started.md

+22-24
Original file line numberDiff line numberDiff line change
@@ -434,18 +434,18 @@ to the backend(s) targeted at export. To support multiple devices, such as
434434
XNNPACK acceleration for Android and Core ML for iOS, export a separate PTE file
435435
for each backend.
436436

437-
To delegate to a backend at export time, ExecuTorch provides the `to_backend()`
438-
function in the `EdgeProgramManager` object, which takes a backend-specific
439-
partitioner object. The partitioner is responsible for finding parts of the
440-
computation graph that can be accelerated by the target backend,and
441-
`to_backend()` function will delegate matched part to given backend for
442-
acceleration and optimization. Any portions of the computation graph not
443-
delegated will be executed by the ExecuTorch operator implementations.
437+
To delegate a model to a specific backend during export, ExecuTorch uses the
438+
`to_edge_transform_and_lower()` function. This function takes the exported program
439+
from `torch.export` and a backend-specific partitioner object. The partitioner
440+
identifies parts of the computation graph that can be optimized by the target
441+
backend. Within `to_edge_transform_and_lower()`, the exported program is
442+
converted to an edge dialect program. The partitioner then delegates compatible
443+
graph sections to the backend for acceleration and optimization. Any graph parts
444+
not delegated are executed by ExecuTorch's default operator implementations.
444445

445446
To delegate the exported model to a specific backend, we need to import its
446447
partitioner as well as edge compile config from ExecuTorch codebase first, then
447-
call `to_backend` with an instance of partitioner on the `EdgeProgramManager`
448-
object `to_edge` function created.
448+
call `to_edge_transform_and_lower`.
449449

450450
Here's an example of how to delegate nanoGPT to XNNPACK (if you're deploying to an Android phone for instance):
451451

@@ -457,7 +457,7 @@ from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPar
457457

458458
# Model to be delegated to specific backend should use specific edge compile config
459459
from executorch.backends.xnnpack.utils.configs import get_xnnpack_edge_compile_config
460-
from executorch.exir import EdgeCompileConfig, to_edge
460+
from executorch.exir import EdgeCompileConfig, to_edge_transform_and_lower
461461

462462
import torch
463463
from torch.export import export
@@ -495,17 +495,14 @@ with torch.nn.attention.sdpa_kernel([SDPBackend.MATH]), torch.no_grad():
495495
# Convert the model into a runnable ExecuTorch program.
496496
# To be further lowered to Xnnpack backend, `traced_model` needs xnnpack-specific edge compile config
497497
edge_config = get_xnnpack_edge_compile_config()
498-
edge_manager = to_edge(traced_model, compile_config=edge_config)
499-
500-
# Delegate exported model to Xnnpack backend by invoking `to_backend` function with Xnnpack partitioner.
501-
edge_manager = edge_manager.to_backend(XnnpackPartitioner())
498+
# Converted to edge program and then delegate exported model to Xnnpack backend
499+
# by invoking `to` function with Xnnpack partitioner.
500+
edge_manager = to_edge_transform_and_lower(traced_model, partitioner = [XnnpackPartitioner()], compile_config = edge_config)
502501
et_program = edge_manager.to_executorch()
503502

504503
# Save the Xnnpack-delegated ExecuTorch program to a file.
505504
with open("nanogpt.pte", "wb") as file:
506505
file.write(et_program.buffer)
507-
508-
509506
```
510507

511508
Additionally, update CMakeLists.txt to build and link the XNNPACK backend to
@@ -651,8 +648,8 @@ DuplicateDynamicQuantChainPass()(m)
651648
traced_model = export(m, example_inputs)
652649
```
653650

654-
Additionally, add or update the `to_backend()` call to use `XnnpackPartitioner`. This instructs ExecuTorch to
655-
optimize the model for CPU execution via the XNNPACK backend.
651+
Additionally, add or update the `to_edge_transform_and_lower()` call to use `XnnpackPartitioner`. This
652+
instructs ExecuTorch to optimize the model for CPU execution via the XNNPACK backend.
656653

657654
```python
658655
from executorch.backends.xnnpack.partition.xnnpack_partitioner import (
@@ -661,8 +658,9 @@ from executorch.backends.xnnpack.partition.xnnpack_partitioner import (
661658
```
662659

663660
```python
664-
edge_manager = to_edge(traced_model, compile_config=edge_config)
665-
edge_manager = edge_manager.to_backend(XnnpackPartitioner()) # Lower to XNNPACK.
661+
edge_config = get_xnnpack_edge_compile_config()
662+
# Convert to edge dialect and lower to XNNPack.
663+
edge_manager = to_edge_transform_and_lower(traced_model, partitioner = [XnnpackPartitioner()], compile_config = edge_config)
666664
et_program = edge_manager.to_executorch()
667665
```
668666

@@ -682,20 +680,20 @@ target_link_libraries(
682680
For more information, see [Quantization in ExecuTorch](../quantization-overview.md).
683681

684682
## Profiling and Debugging
685-
After lowering a model by calling `to_backend()`, you may want to see what got delegated and what didn’t. ExecuTorch
683+
After lowering a model by calling `to_edge_transform_and_lower()`, you may want to see what got delegated and what didn’t. ExecuTorch
686684
provides utility methods to give insight on the delegation. You can use this information to gain visibility into
687685
the underlying computation and diagnose potential performance issues. Model authors can use this information to
688686
structure the model in a way that is compatible with the target backend.
689687

690688
### Visualizing the Delegation
691689

692-
The `get_delegation_info()` method provides a summary of what happened to the model after the `to_backend()` call:
690+
The `get_delegation_info()` method provides a summary of what happened to the model after the `to_edge_transform_and_lower()` call:
693691

694692
```python
695693
from executorch.devtools.backend_debug import get_delegation_info
696694
from tabulate import tabulate
697695

698-
# ... After call to to_backend(), but before to_executorch()
696+
# ... After call to to_edge_transform_and_lower(), but before to_executorch()
699697
graph_module = edge_manager.exported_program().graph_module
700698
delegation_info = get_delegation_info(graph_module)
701699
print(delegation_info.get_summary())
@@ -762,7 +760,7 @@ Through the ExecuTorch Developer Tools, users are able to profile model executio
762760
An ETRecord is an artifact generated at the time of export that contains model graphs and source-level metadata linking the ExecuTorch program to the original PyTorch model. You can view all profiling events without an ETRecord, though with an ETRecord, you will also be able to link each event to the types of operators being executed, module hierarchy, and stack traces of the original PyTorch source code. For more information, see [the ETRecord docs](../etrecord.md).
763761

764762

765-
In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_backend()` mutates the graph in-place.
763+
In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_edge_transform_and_lower()` mutates the graph in-place.
766764

767765
```
768766
# export_nanogpt.py

0 commit comments

Comments
 (0)
Please sign in to comment.