Skip to content

Commit 683869d

Browse files
authored
Experiment with private rooted Pixel 3 devices (#10192)
I'm creating a new experimental workflow to run benchmarks on private rooted Pixel 3 devices. We have only 2 devices there though, so I think let's reduce the number of model we want to cover for now. ### Testing https://github.com/pytorch/executorch/actions/runs/14465442714
1 parent 900b42c commit 683869d

File tree

2 files changed

+63
-0
lines changed

2 files changed

+63
-0
lines changed

.ci/scripts/gather_benchmark_configs.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
"samsung_galaxy_s22": "arn:aws:devicefarm:us-west-2:308535385114:devicepool:02a2cf0f-6d9b-45ee-ba1a-a086587469e6/e59f866a-30aa-4aa1-87b7-4510e5820dfa",
2424
"samsung_galaxy_s24": "arn:aws:devicefarm:us-west-2:308535385114:devicepool:02a2cf0f-6d9b-45ee-ba1a-a086587469e6/98f8788c-2e25-4a3c-8bb2-0d1e8897c0db",
2525
"google_pixel_8_pro": "arn:aws:devicefarm:us-west-2:308535385114:devicepool:02a2cf0f-6d9b-45ee-ba1a-a086587469e6/d65096ab-900b-4521-be8b-a3619b69236a",
26+
"google_pixel_3_private_rooted": "arn:aws:devicefarm:us-west-2:308535385114:devicepool:02a2cf0f-6d9b-45ee-ba1a-a086587469e6/98d23ca8-ea9e-4fb7-b725-d402017b198d",
2627
}
2728

2829
# Predefined benchmark configurations
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
name: android-perf (private devices)
2+
3+
on:
4+
schedule:
5+
- cron: 0 0,4,8,12,16,20 * * *
6+
pull_request:
7+
paths:
8+
- .github/workflows/android-perf-private-device-experiment.yml
9+
push:
10+
branches:
11+
- main
12+
paths:
13+
- .github/workflows/android-perf-private-device-experiment.yml
14+
# Note: GitHub has an upper limit of 10 inputs
15+
workflow_dispatch:
16+
inputs:
17+
models:
18+
description: Models to be benchmarked
19+
required: false
20+
type: string
21+
default: mv3,meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8,meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8
22+
devices:
23+
description: Target devices to run benchmark
24+
required: false
25+
type: string
26+
default: google_pixel_3_private_rooted
27+
benchmark_configs:
28+
description: The list of configs used the benchmark
29+
required: false
30+
type: string
31+
workflow_call:
32+
inputs:
33+
models:
34+
description: Models to be benchmarked
35+
required: false
36+
type: string
37+
default: mv3,meta-llama/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8,meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8
38+
devices:
39+
description: Target devices to run benchmark
40+
required: false
41+
type: string
42+
default: google_pixel_3_private_rooted
43+
benchmark_configs:
44+
description: The list of configs used the benchmark
45+
required: false
46+
type: string
47+
48+
concurrency:
49+
group: android-perf-private-devices-${{ github.event.pull_request.number || github.ref_name }}-${{ github.ref_type == 'branch' && github.sha }}-${{ github.event_name == 'workflow_dispatch' }}-${{ github.event_name == 'schedule' }}
50+
cancel-in-progress: true
51+
52+
jobs:
53+
android:
54+
uses: ./.github/workflows/android-perf.yml
55+
secrets: inherit
56+
permissions:
57+
id-token: write
58+
contents: read
59+
with:
60+
models: ${{ inputs.models }}
61+
devices: google_pixel_3_private_rooted
62+
benchmark_configs: ${{ inputs.benchmark_configs }}

0 commit comments

Comments
 (0)