-
Notifications
You must be signed in to change notification settings - Fork 12.9k
[CI Energy Waste]: Optimize CI workflow by consolidating redundant jobs #62011
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Merged lint, knip, and format into quality-checks job - Merged typecheck, misc, and self-check into build-and-validate job - Reduced from 12 to 8 jobs while preserving all functionality - Eliminated 12 redundant setup steps (checkout + node setup + npm ci) Jobs consolidated: - quality-checks: lint + knip + format (saved 6 steps) - build-and-validate: typecheck + misc + self-check (saved 6 steps) Total savings: 12 step executions per CI run
This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise. |
1 similar comment
This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise. |
The point of having these jobs separated was that so PRs with multiple differing failures get each issue reported separately. Otherwise, PRs with multiple problems will have things fixed, then pushed, rerun everything, then another thing breaks, then that's fixed, and so on, as opposed to all problems being given up front. It's a lot less taxing for people who don't know to (or forget to) run all of the random checks in the repo, and for us to keep re-approving workflows or re-reviewing PRs. |
- Added continue-on-error to quality-checks and build-and-validate jobs - All checks now run even if one fails, providing complete feedback - Added failure reporting with specific commands to run locally This addresses the concern about separate failure reporting while keeping the optimization benefits of job consolidation.
Thank for your review. |
I just pushed the fix I meant |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR reduces CI energy waste by consolidating redundant jobs into fewer, more efficient workflows.
- Merged lint, knip, and format jobs into a single
quality-checks
job - Combined typecheck, misc, and self-check into a unified
build-and-validate
job - Kept a single baselines job and removed duplicate setup steps
Comments suppressed due to low confidence (2)
.github/workflows/ci.yml:104
- The consolidated
quality-checks
job is missing theactions/checkout
andactions/setup-node
(and optionalnpm ci
) steps. You need these initial steps to ensure the repository is checked out and Node is set up before running lint, knip, and dprint.
quality-checks:
.github/workflows/ci.yml:167
- The
build-and-validate
job starts withnpm ci
but does not include anactions/checkout
oractions/setup-node
step. Without checking out the repo and setting up Node, the build commands will fail.
build-and-validate:
Possibly, but as you saw, it's tedious/error-prone to get right since you have to restate the list of continue-on-error'd steps. All-in-all, these particular changes seem a little bit like small beans compared to the incidental removal of extra macOS jobs done in #61978, for example, and we're likely going to be able to reduce the (actually impactful) matrix size likely soon by dropping old versions of Node. I'll note that the diff is a lot larger than it needs to be since the |
We're not accepting unapproved changes, especially infrastructure-related, at this time. Please review CONTRIBUTING.md for guidance. Thanks! |
Change context
The CI workflow consumed more than 514 days of compute time (≈740,000 minutes) across all runners in the past year alone. Many jobs included redundant steps such as checkout, setup-node, and npm ci that repeated across 12 separate jobs. For example,
npm ci
alone was run up to 13 times. These redundant setup steps add unnecessary overhead and can each take up to 1–1.5 minutes, inflating overall runtime. For instance, only one simple checkout in one job can take up to 20s of runtime:Change details
Changes Made
Results
Redundant steps eliminated
4x actions/checkout
4x actions/setup-node
4x npm ci
Additional context
We are a team of researchers from University of Zurich (https://www.ifi.uzh.ch/en/zest.html) currently working on automating energy optimizations in GitHub Actions workflows. This optimization maintains full functionality while reducing computational overhead and energy consumption.
[email protected]