-
-
Notifications
You must be signed in to change notification settings - Fork 29
feat: optimize macOS build times with caching and parallel jobs #148
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add sccache-action for compilation caching (up to 50% faster on cache hits) - Cache Node.js source archives to avoid re-downloading - Increase arm64 MAKE_JOB_COUNT from 2 to 3 (matches 3 vCPU on macos-14) - Keep x64 MAKE_JOB_COUNT at 4 (matches 4 vCPU on macos-15-intel) Co-authored-by: robertsLando <[email protected]>
- Add yarn cache to setup-node to speed up dependency installation - Create detailed BUILD_OPTIMIZATION.md documenting all changes - Document expected improvements and future optimization opportunities Co-authored-by: robertsLando <[email protected]>
robertsLando
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot fix
.github/workflows/build-macos.yml
Outdated
| # Cache Node.js source archives | ||
| - name: Cache Node.js source archive | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: ~/.pkg-cache/node | ||
| key: node-source-${{ matrix.target-node }}-${{ hashFiles('patches/patches.json') }} | ||
| restore-keys: | | ||
| node-source-${{ matrix.target-node }}- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove caching of nodejs source archive
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed Node.js source archive caching. Moved to future optimization opportunities section in the documentation for further investigation. (e95671d)
| // Consider ThinLTO for faster builds | ||
| // args.push('--enable-lto'); // Full LTO | ||
| // Or set via CFLAGS/CXXFLAGS: -flto=thin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@faulpeltz any clue if this change could be legit or not?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thin LTO could help (it basically a faster version of LTO)
but it only works with clang AFAIK
also this is link-time only so not sure how much time this saves
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and what about the sccache then? Could that help? I'm not a C++ expert and I don't like to blindly accept copilot suggestions, I trust you more
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
well sccache works by using (compiler env, arch, platform, args, flags, filenames, hashes, ..) as a cache key, if any of that changes it wont do anything, it would only work if rebuilding somehow with the exact same setup, if you change any flag, file content, compiler version, etc. the cache misses
I think it creates a post-action with some stats so its something to try out, but I'm skeptical for our use case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the detailed response! Any other suggestions about how we could speed up macos builds then?
Node.js source archive caching has been moved to future optimization opportunities section as it needs more investigation to ensure it doesn't interfere with the build process. Co-authored-by: robertsLando <[email protected]>
macOS builds take 2h (node20), 4h (node22), and 6h (node24) - the main bottleneck for binary releases. This implements compilation caching and optimizes resource utilization.
Changes
Compilation caching with sccache
mozilla-actions/sccache-actionCC: "sccache clang",CXX: "sccache clang++"Optimize parallel compilation
MAKE_JOB_COUNTfrom 2→3 (matches 3 vCPU on macos-14)Yarn dependency caching
cache: 'yarn'insetup-nodeactionExpected Impact
Documentation
Created
docs/BUILD_OPTIMIZATION.mddocumenting:Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.