Skip to content

Commit 85ba2b7

Browse files
guangy10facebook-github-bot
guangy10
authored and
facebook-github-bot
committed
Restructuring demos (#531)
Summary: Pull Request resolved: #531 ----- ## Confusion arising from the current code structure: Reference to current code structure: https://github.com/pytorch/executorch/tree/main/examples The current code structure under `executorch/examples` mixes different levels of detail. For instance, the directories `backend/`, `quantization/`, and `export/` are intended for component-level demo code, such as showcasing quantization/export/delegation workflows or providing sample backend/quantizer/partitioner implementations. Assembling them to create a AOT path should not belong to any of it. However, the current structure combines recipes and individual components: - All contents of `backend/` are exclusively tailored for [xnnpack](https://github.com/pytorch/executorch/tree/main/examples/backend). - [export+delegate+composite](https://github.com/pytorch/executorch/blob/main/examples/export/export_and_delegate.py) and [bundled program example](https://github.com/pytorch/executorch/blob/main/examples/export/export_bundled_program.py) are placed in `export/` - Different runtimes are placed haphazardly., with items like [executor_runner](https://github.com/pytorch/executorch/tree/main/examples/executor_runner), [bundled_executor_runner](https://github.com/pytorch/executorch/tree/main/examples/bundled_executor_runner), and [xnn_executor_runner](https://github.com/pytorch/executorch/tree/main/examples/backend) - Backend-specific demos are scattered in the top-level directory, such as [arm's demo](https://github.com/pytorch/executorch/tree/main/examples/arm) and [partitioner/quantizer demo](https://github.com/pytorch/executorch/tree/main/examples/example_quantizer_and_delegate) Issues that are not directly tie to code restructuring but will need to be addressed as well. **Non-functional code/demo** (fix or hide): - [Unit tests](https://github.com/pytorch/executorch/tree/main/examples/export/test) under the `export/` directory are not runnable in the OSS environment. - The [example_quantizer_and_delegate](https://github.com/pytorch/executorch/tree/main/examples/example_quantizer_and_delegate) is not functioning and should be rectified or hidden. - The [arm's demo](https://github.com/pytorch/executorch/tree/main/examples/arm) is not runnable directly w/o additional guide. ----- ## Proposed code structure: organize demos in a way that is easier for users to understand how pieces are put together to create an e2e experience. We can go with this "vertical" structure if we could come up and agree on guidelines of whether a new demo should be considered as "adding a new demo" or "extending an existing demo". The purpose is to prevent randomly dumping demos in the top-level executorch/examples folder with boilerplate code. While this doesn't violate the rule of structuring demos vertically for user understanding, , but it literally has no structure at all, and it will quickly grow of out our maintenance. In my opinion, it would make sense to consider the following cases as **"adding a new demo"**: 1. Demo of a new 1st/3rd party backend, e.g. portable (default backend), xnnpack, arm, coreml, etc. 2. Demo of toolings, e.g. selective build, productivity sdk, etc. 3. Target specific apps, e.g. playground apps for Android/iOS/Embedded The rests will be considered as **"extending an existing demo"**, for example: - Demo of composibitlity, e.g. delegate and composite, whole/partial graph lowering, custom ops - Demo of specific components, e.g. quantization workflow, 2-stage export flow, example quantizer/partitioner, etc. After PTC we will spend more efforts on expanding and polishing the two major 1st-party demos (`examples/portable/` & `examples/xnnpack/`) we own in the long run. Maybe `examples/sdk` as well. So with the guidelines, the new top-level structure will looks like: ``` executorch/examples/ ├── README.md # top-level description for demos and folder structures ├── models/ ├── xnnpack/ # 1p e2e ├── portable/ # 1p e2e ├── arm/ # 3p backend e2e ├── qualcomm/ # 3p backend e2e ├── apple/ # 3p backend e2e ├── third-party/ # for the short-term, we can assume a centralized place for all third-party libs required by demos, e.g. sam, llama, etc. ├── selective_build/ ├── sdk/ ├── demo-apps/ └── utils/ # future ``` If we zoom in to `examples/portable/`, it will look like: ``` executorch/examples/portable/ ├── executor_runner/ ├── bundled_executor_runner/ ├── custom_ops/ ├── scripts/ │ ├── export_and_delegate.py │ ├── export_bundled_program.py │ └── export.py ├── test/ ├── __init__.py ├── utils.py └── README.md ``` If we zoom in to `examples/xnnpack/`, it will look like: ``` executorch/examples/xnnpack/ ├── quantization/ │ ├── example.py │ ├── TARGETS │ ├── test_quantize.sh │ └── utils.py ├── __init__.py ├── aot_compiler.py └── README.md ``` Reviewed By: mergennachin Differential Revision: D49714823 fbshipit-source-id: fd3c2339057e9a3d7dc1d69ec7ede1287f081d8f
1 parent dab7911 commit 85ba2b7

File tree

96 files changed

+412
-351
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

96 files changed

+412
-351
lines changed

‎.ci/scripts/gather_test_models.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
from typing import Any
1111

1212
from examples.models import MODEL_NAME_TO_MODEL
13-
from examples.recipes.xnnpack_optimization import MODEL_NAME_TO_OPTIONS
13+
from examples.xnnpack import MODEL_NAME_TO_OPTIONS
1414

1515
# NB: Skip buck2 on MacOS to cut down the number of combinations we
1616
# need to run there as the number of MacOS runner is limited. Buck2
@@ -80,8 +80,7 @@ def export_models_for_ci() -> None:
8080
}
8181
delegation_configs = {
8282
False,
83-
name in MODEL_NAME_TO_OPTIONS
84-
and MODEL_NAME_TO_OPTIONS[name].xnnpack_delegation,
83+
name in MODEL_NAME_TO_OPTIONS and MODEL_NAME_TO_OPTIONS[name].delegation,
8584
}
8685
for build_tool in BUILD_TOOLS.keys():
8786
if target_os not in BUILD_TOOLS[build_tool]:

‎.ci/scripts/test.sh

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -59,11 +59,11 @@ test_model() {
5959
cd ../../..
6060
fi
6161

62-
"${PYTHON_EXECUTABLE}" -m examples.export.export_example --model_name="${MODEL_NAME}"
62+
"${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export --model_name="${MODEL_NAME}"
6363

6464
# Run test model
6565
if [[ "${BUILD_TOOL}" == "buck2" ]]; then
66-
buck2 run //examples/executor_runner:executor_runner -- --model_path "./${MODEL_NAME}.pte"
66+
buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./${MODEL_NAME}.pte"
6767
elif [[ "${BUILD_TOOL}" == "cmake" ]]; then
6868
if [[ ! -f ${CMAKE_OUTPUT_DIR}/executor_runner ]]; then
6969
build_cmake_executor_runner
@@ -98,24 +98,24 @@ test_model_with_xnnpack() {
9898

9999
# Quantization-only
100100
if [[ ${WITH_QUANTIZATION} == true ]] && [[ ${WITH_DELEGATION} == false ]]; then
101-
bash examples/quantization/test_quantize.sh "${BUILD_TOOL}" "${MODEL_NAME}"
101+
bash examples/xnnpack/quantization/test_quantize.sh "${BUILD_TOOL}" "${MODEL_NAME}"
102102
exit 0
103103
fi
104104

105105
# Delegation
106106
if [[ ${WITH_QUANTIZATION} == true ]]; then
107107
SUFFIX="q8"
108-
"${PYTHON_EXECUTABLE}" -m examples.backend.xnnpack_examples --model_name="${MODEL_NAME}" --delegate --quantize
108+
"${PYTHON_EXECUTABLE}" -m examples.xnnpack.aot_compiler --model_name="${MODEL_NAME}" --delegate --quantize
109109
else
110110
SUFFIX="fp32"
111-
"${PYTHON_EXECUTABLE}" -m examples.backend.xnnpack_examples --model_name="${MODEL_NAME}" --delegate
111+
"${PYTHON_EXECUTABLE}" -m examples.xnnpack.aot_compiler --model_name="${MODEL_NAME}" --delegate
112112
fi
113113

114114
OUTPUT_MODEL_PATH="${MODEL_NAME}_xnnpack_${SUFFIX}.pte"
115115

116116
# Run test model
117117
if [[ "${BUILD_TOOL}" == "buck2" ]]; then
118-
buck2 run //examples/backend:xnn_executor_runner -- --model_path "${OUTPUT_MODEL_PATH}"
118+
buck2 run //examples/xnnpack:xnn_executor_runner -- --model_path "${OUTPUT_MODEL_PATH}"
119119
elif [[ "${BUILD_TOOL}" == "cmake" ]]; then
120120
if [[ ! -f ${CMAKE_OUTPUT_DIR}/backends/xnnpack/xnn_executor_runner ]]; then
121121
build_cmake_xnn_executor_runner
@@ -129,15 +129,15 @@ test_model_with_xnnpack() {
129129

130130
test_demo_backend_delegation() {
131131
echo "Testing demo backend delegation on AddMul"
132-
"${PYTHON_EXECUTABLE}" -m examples.export.export_and_delegate --option "composite"
133-
"${PYTHON_EXECUTABLE}" -m examples.export.export_and_delegate --option "partition"
134-
"${PYTHON_EXECUTABLE}" -m examples.export.export_and_delegate --option "whole"
132+
"${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export_and_delegate --option "composite"
133+
"${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export_and_delegate --option "partition"
134+
"${PYTHON_EXECUTABLE}" -m examples.portable.scripts.export_and_delegate --option "whole"
135135

136136
# Run test model
137137
if [[ "${BUILD_TOOL}" == "buck2" ]]; then
138-
buck2 run //examples/executor_runner:executor_runner -- --model_path "./composite_model.pte"
139-
buck2 run //examples/executor_runner:executor_runner -- --model_path "./partition_lowered_model.pte"
140-
buck2 run //examples/executor_runner:executor_runner -- --model_path "./whole.pte"
138+
buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./composite_model.pte"
139+
buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./partition_lowered_model.pte"
140+
buck2 run //examples/portable/executor_runner:executor_runner -- --model_path "./whole.pte"
141141
elif [[ "${BUILD_TOOL}" == "cmake" ]]; then
142142
if [[ ! -f ${CMAKE_OUTPUT_DIR}/executor_runner ]]; then
143143
build_cmake_executor_runner

‎.ci/scripts/utils.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ install_flatc_from_source() {
7070

7171
build_executorch_runner_buck2() {
7272
# Build executorch runtime with retry as this step is flaky on macos CI
73-
retry buck2 build //examples/executor_runner:executor_runner
73+
retry buck2 build //examples/portable/executor_runner:executor_runner
7474
}
7575

7676
build_executorch_runner_cmake() {

‎.github/workflows/pull.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ jobs:
5858
MODEL_NAME=${{ matrix.model }}
5959
BUILD_TOOL=${{ matrix.build-tool }}
6060
XNNPACK_QUANTIZATION=${{ matrix.xnnpack_quantization }}
61-
XNNPACK_DELEGATION=${{ matrix.xnnpack_delegation }}
61+
XNNPACK_DELEGATION=${{ matrix.delegation }}
6262
DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }}
6363
6464
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh "${BUILD_TOOL}"
@@ -87,7 +87,7 @@ jobs:
8787
BUILD_TOOL=${{ matrix.build-tool }}
8888
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh "${BUILD_TOOL}"
8989
# Test custom ops
90-
PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
90+
PYTHON_EXECUTABLE=python bash examples/portable/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
9191
9292
test-selective-build-linux:
9393
name: test-selective-build-linux

‎.github/workflows/trunk.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ jobs:
5555
MODEL_NAME=${{ matrix.model }}
5656
BUILD_TOOL=${{ matrix.build-tool }}
5757
XNNPACK_QUANTIZATION=${{ matrix.xnnpack_quantization }}
58-
XNNPACK_DELEGATION=${{ matrix.xnnpack_delegation }}
58+
XNNPACK_DELEGATION=${{ matrix.delegation }}
5959
DEMO_BACKEND_DELEGATION=${{ matrix.demo_backend_delegation }}
6060
6161
# Setup MacOS dependencies as there is no Docker support on MacOS atm
@@ -85,7 +85,7 @@ jobs:
8585
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-macos.sh "${BUILD_TOOL}"
8686
8787
# Build and test custom ops
88-
PYTHON_EXECUTABLE=python bash examples/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
88+
PYTHON_EXECUTABLE=python bash examples/portable/custom_ops/test_custom_ops.sh "${BUILD_TOOL}"
8989
popd
9090
9191
test-selective-build-macos:

‎CMakeLists.txt

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,8 @@ set(CMAKE_CXX_FLAGS_DEBUG "-O0 -g")
9393

9494
# Option to register custom operator `my_ops::mul3` or `my_ops::mul4` or no
9595
# custom ops at all. Custom ops are defined in
96-
# `examples/custom_ops/custom_ops_1.py` and
97-
# `examples/custom_ops/custom_ops_2.cpp`.
96+
# `examples/portable/custom_ops/custom_ops_1.py` and
97+
# `examples/portable/custom_ops/custom_ops_2.cpp`.
9898
option(
9999
REGISTER_EXAMPLE_CUSTOM_OP
100100
"Register whether custom op 1 (my_ops::mul3) or custom op 2 (my_ops::mul4) \
@@ -296,7 +296,7 @@ if(EXECUTORCH_BUILD_EXECUTOR_RUNNER)
296296

297297
# Generate custom_ops_lib based on REGISTER_EXAMPLE_CUSTOM_OP
298298
if(REGISTER_EXAMPLE_CUSTOM_OP EQUAL 1 OR REGISTER_EXAMPLE_CUSTOM_OP EQUAL 2)
299-
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/examples/custom_ops)
299+
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/examples/portable/custom_ops)
300300
list(APPEND _executor_runner_libs custom_ops_lib)
301301
endif()
302302

‎examples/example_quantizer_and_delegate/README.md renamed to ‎backends/example/README.md

File renamed without changes.

‎examples/example_quantizer_and_delegate/TARGETS renamed to ‎backends/example/TARGETS

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ python_library(
88
],
99
deps = [
1010
"//caffe2:torch",
11-
"//executorch/examples/example_quantizer_and_delegate/example_operators:example_operators_lib",
11+
"//executorch/backends/example/example_operators:example_operators_lib",
1212
],
1313
)
1414

@@ -18,7 +18,7 @@ python_library(
1818
"example_backend.py",
1919
],
2020
deps = [
21-
"//executorch/examples/example_quantizer_and_delegate/example_backend_delegate_passes:lib",
21+
"//executorch/backends/example/example_backend_delegate_passes:lib",
2222
"//executorch/exir/backend:backend_details",
2323
"//executorch/exir/backend:compile_spec_schema",
2424
],
@@ -32,7 +32,7 @@ python_library(
3232
deps = [
3333
":example_backend",
3434
"//caffe2:torch",
35-
"//executorch/examples/example_quantizer_and_delegate/example_operators:example_operators_lib",
35+
"//executorch/backends/example/example_operators:example_operators_lib",
3636
"//executorch/exir:graph_module",
3737
"//executorch/exir/backend:partitioner",
3838
"//executorch/exir/backend/canonical_partitioners:canonical_partitioner_lib",

‎examples/example_quantizer_and_delegate/diagrams/delegate.png renamed to ‎backends/example/diagrams/delegate.png

File renamed without changes.

‎examples/example_quantizer_and_delegate/diagrams/memory_permute.png renamed to ‎backends/example/diagrams/memory_permute.png

File renamed without changes.

0 commit comments

Comments
 (0)