Skip to content

Commit 9620cce

Browse files
[CI] Reworked the CI to Make it Faster
The CI for VTR was doing far more work than it needed to, which was leading to long CI run times of around 1.5 hours on average. Overall, the CI used 12.5 hours of compute and used more than 20 GitHub runners at a time which hurt concurrency of runs. The PR reduces the compute by only building VTR once for general builds and using that build throughout when needed. This reduces the CI compute down to 7.5 hours. In this process, I also added dependency chains to try and schedule the runs efficiently to keep the number of active GitHub runners below 10 (which is our current maximum). I also fixed a bug with the way we have been using CCache. We were using the same cache for all builds, which works fine for some projects; but for ours that causes tons of cache misses when a gcc-11 build cache is used for a Clang-18 cache for example. This PR makes each build's cache unique, which enables better cache hit rates. I have found that when the cache hit rate is perfect (i.e. the build is unchanged from the last run), the CI uses less than 3 hours of compute and the test portion only takes 20 minutes. When this happens, building the container actually becomes the tall-pole since it often takes a little over 30 minutes.
1 parent 4bb340a commit 9620cce

File tree

13 files changed

+271
-161
lines changed

13 files changed

+271
-161
lines changed

.github/scripts/unittest.sh

Lines changed: 0 additions & 15 deletions
This file was deleted.

.github/workflows/nightly_test_manual.yml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,3 +135,37 @@ jobs:
135135
name: nightly_tests_golden
136136
path: |
137137
vtr_flow/**/vtr_reg_nightly*/**/golden_results.txt
138+
139+
Coverity:
140+
name: 'Coverity Scan'
141+
runs-on: ubuntu-24.04
142+
steps:
143+
144+
- uses: actions/setup-python@v5
145+
with:
146+
python-version: 3.12.3
147+
- uses: actions/checkout@v4
148+
with:
149+
submodules: 'true'
150+
151+
- name: Get number of CPU cores
152+
uses: SimenB/github-actions-cpu-cores@v2
153+
id: cpu-cores
154+
155+
- name: Install dependencies
156+
run: ./.github/scripts/install_dependencies.sh
157+
158+
- uses: hendrikmuhs/[email protected]
159+
with:
160+
key: ${{ github.job }}
161+
162+
- name: Test
163+
env:
164+
CMAKE_PARAMS: '-DVTR_ASSERT_LEVEL=3 -DWITH_BLIFEXPLORER=on'
165+
NUM_PROC: ${{ steps.cpu-cores.outputs.count }}
166+
_COVERITY_URL: 'https://scan.coverity.com/download/linux64'
167+
_COVERITY_MD5: 'd0d7d7df9d6609e578f85096a755fb8f'
168+
run: |
169+
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
170+
./.github/scripts/build.sh
171+

0 commit comments

Comments
 (0)