Improve the CI time

[tigercosmos]

Currently, we have too many GitHub action items, which results in a long building and testing time.
We need to refactor the current CI files and make the CI faster.

[seladb]

how can we improve it? Maybe we can list the action items so anyone can work on it

[tigercosmos]

@seladb let's brainstorm some ideas to improve this. cc @Dimi1010 @egecetin @clementperon

[clementperon]

I didn't look at the pipeline but using a cache + ccache could avoid to rebuild some files.
Else limiting the number of OS and run a nightly pipeline with all the OS / configurations

[egecetin]

My suggestion is that if the cache is not enough, we can distribute the workflow to several providers to prevent reaching compute limits. For example,

• Cirrus: only run doxy
• Azure: only windows runners
• Github: linux runners
• and so on ...

It should make it harder to maintain a bit, but we can handle it if we have to

[tigercosmos]

@egecetin I think it could be the last step. The issue is the current CI script is not efficient enough. Just like that we need to improve the algorithm before trying to use multithread(?)

[egecetin]

@tigercosmos Yes of course. If possible, I agree we should improve the pipeline first. To be honest I did not check all workflows but most of the runs between 5-8 mins (approx 2-3 mins from build so it is like 30%-50% of run is build), right? So, I'm not sure we can dramatically improve CI runtime (even with caching) unless we limit the runner OS/version/config or improve parallelism. I think for "multiplatform" definition the code should be tested as much as possible. So, I don't know what we can do.

[egecetin]

Something that comes to my mind, testing consumes approx 1 minute. If I'm not mistaken current test suite does not support multithreading (@seladb correct me if I'm wrong pls). So maybe using a test library supporting multithreading might lead better testing times? (eg. integrating GoogleTest + ctest to run tests parallel to efficiently saturate CPU)

or maybe we can just remove "Check Installation" step which can halve the workflow time but not sure about this.

[tigercosmos]

@egecetin true, I think the parallelism of test cases may be the key. Probably we should focus on it in the next step.

[tigercosmos]

@seladb what do you think?

[seladb]

The "Check installation" step re-builds the entire library, although it was built just a few steps earlier. Maybe we can use cache to prevent that?

Regarding the tests - they indeed take between 1 min (on Linux) and 4 mins (on Windows), so parallelism may help. But replacing the whole testing framework will take a very long time and I'm not sure it's worth the relatively small runtime gain...