Skip to content

[CI Energy Waste]: Optimize CI workflow by consolidating redundant jobs #62011

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

souhailaS
Copy link

Change context

The CI workflow consumed more than 514 days of compute time (≈740,000 minutes) across all runners in the past year alone. Many jobs included redundant steps such as checkout, setup-node, and npm ci that repeated across 12 separate jobs. For example, npm ci alone was run up to 13 times. These redundant setup steps add unnecessary overhead and can each take up to 1–1.5 minutes, inflating overall runtime. For instance, only one simple checkout in one job can take up to 20s of runtime:

Screenshot 2025-07-07 at 19 26 33

Change details

Changes Made

  • Merged lint + knip + format into quality-checks
  • Merged typecheck + misc + self-check into build-and-validate

Results

  • 12 jobs → 8 jobs (33% reduction)
  • 12 fewer setup steps per CI run
  • Zero functionality lost

Redundant steps eliminated

4x actions/checkout
4x actions/setup-node
4x npm ci

Additional context

We are a team of researchers from University of Zurich (https://www.ifi.uzh.ch/en/zest.html) currently working on automating energy optimizations in GitHub Actions workflows. This optimization maintains full functionality while reducing computational overhead and energy consumption.

[email protected]

- Merged lint, knip, and format into quality-checks job
- Merged typecheck, misc, and self-check into build-and-validate job
- Reduced from 12 to 8 jobs while preserving all functionality
- Eliminated 12 redundant setup steps (checkout + node setup + npm ci)

Jobs consolidated:
- quality-checks: lint + knip + format (saved 6 steps)
- build-and-validate: typecheck + misc + self-check (saved 6 steps)

Total savings: 12 step executions per CI run
@Copilot Copilot AI review requested due to automatic review settings July 7, 2025 17:40
@github-project-automation github-project-automation bot moved this to Not started in PR Backlog Jul 7, 2025
@typescript-bot typescript-bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Jul 7, 2025
@typescript-bot
Copy link
Collaborator

This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.

1 similar comment
@typescript-bot
Copy link
Collaborator

This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.

Copilot

This comment was marked as outdated.

@jakebailey
Copy link
Member

This optimization maintains full functionality while reducing computational overhead and energy consumption.

The point of having these jobs separated was that so PRs with multiple differing failures get each issue reported separately. Otherwise, PRs with multiple problems will have things fixed, then pushed, rerun everything, then another thing breaks, then that's fixed, and so on, as opposed to all problems being given up front. It's a lot less taxing for people who don't know to (or forget to) run all of the random checks in the repo, and for us to keep re-approving workflows or re-reviewing PRs.

- Added continue-on-error to quality-checks and build-and-validate jobs
- All checks now run even if one fails, providing complete feedback
- Added failure reporting with specific commands to run locally

This addresses the concern about separate failure reporting while keeping
the optimization benefits of job consolidation.
@souhailaS
Copy link
Author

souhailaS commented Jul 7, 2025

Thank for your review.
I see, makes sense. Wouldn't this be fixed by a continue-on-error: true? and some nice logging in a reporting step with the details of which step has failed.
Adding a reporting step would be cheaper than a re-checkout

@souhailaS
Copy link
Author

I just pushed the fix I meant

@souhailaS souhailaS requested a review from Copilot July 7, 2025 18:06
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR reduces CI energy waste by consolidating redundant jobs into fewer, more efficient workflows.

  • Merged lint, knip, and format jobs into a single quality-checks job
  • Combined typecheck, misc, and self-check into a unified build-and-validate job
  • Kept a single baselines job and removed duplicate setup steps
Comments suppressed due to low confidence (2)

.github/workflows/ci.yml:104

  • The consolidated quality-checks job is missing the actions/checkout and actions/setup-node (and optional npm ci) steps. You need these initial steps to ensure the repository is checked out and Node is set up before running lint, knip, and dprint.
  quality-checks:

.github/workflows/ci.yml:167

  • The build-and-validate job starts with npm ci but does not include an actions/checkout or actions/setup-node step. Without checking out the repo and setting up Node, the build commands will fail.
  build-and-validate:

@jakebailey
Copy link
Member

Possibly, but as you saw, it's tedious/error-prone to get right since you have to restate the list of continue-on-error'd steps.

All-in-all, these particular changes seem a little bit like small beans compared to the incidental removal of extra macOS jobs done in #61978, for example, and we're likely going to be able to reduce the (actually impactful) matrix size likely soon by dropping old versions of Node.

I'll note that the diff is a lot larger than it needs to be since the baselines job has shifted position.

@RyanCavanaugh
Copy link
Member

We're not accepting unapproved changes, especially infrastructure-related, at this time. Please review CONTRIBUTING.md for guidance. Thanks!

@github-project-automation github-project-automation bot moved this from Not started to Done in PR Backlog Jul 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants