Parallelise ./configure script

[nisbet-hubbard]

Description

We compile our PHP regularly. Would it be possible for the ./configure script to be parallelised using the ideas of https://github.com/tavianator/parconf?

[iluuu1994]

@petk is working on a cmake build system. I'm not sure what the status of it is, or whether it is even better in that regard. It would be best for Peter to answer this question. 🙂 I would support this for sure, with a large number of cores and ccache, ./configure is often the slowest part of a full rebuild.

[NattyNarwhal]

CMake and meson aren't parallel either. Although in theory they have more information where to insert i.e. barriers. This project seems like a cool proof of concept though.

I will say that the configure step tends to be pretty nasty on platforms like IBM i which don't have POSIX performance characteristics with expensive fork. Unfortunately, I also have a little bit of a complicated relationship with autotools for compatibility stuff...

[NattyNarwhal]

Throwing this out as a joke except maybe not if it's actually a good idea, the solution might be to port our Win32 configure to Unix, and run it with i.e. Node. Threaded configure? That might even be a solution for the WSH/JScript deprecation stuff that I think is happening too...

[petk]

Interesting discussion indeed. One of the issues in these build systems is that many of these checks are related to one another. For example, in Autotools when headers are being checked, they are added one after the other to the collection of included headers and they are then used when compiling other tests. Or when libraries are checked they are all added to the global LIBS variable, which other checks might use. Similar with flags.

Isolating all these checks is the way to go probably so that each check is like a standalone unit without any side effects and doesn't need other checks.

Making Autotools parallel is most likely very difficult to achieve. Most people just gave up on making configure phase parallel and accept the long configuration time. But if there is some hack or a macro where the checks could be run as a separate processes, then it would be a nice addition. There were some discussions about this also on the Autoconf mailing lists.

And, nope, CMake isn't running configuration phase in parallel either. Perhaps one day.

What can be also done to speed up the configure phase is to use some sort of cache for known checks on some platform. In Autotools, setting those ac_cv_* and php_cv_* variables. In CMake, cache is also being added for Windows target, as there is mostly everything known in advance. Otherwise, configuration phase on Windows in CMake can take 20x longer than on *nix platforms if such cached approaches aren't used. I was stunned when I saw how slow configuration phase is on Windows compared to running configure in msys/mingw/cygwin environment on the same Windows machine and similar. It has something to do with the MSVC and there aren't any fixes out there yet.

Yes, I completely agree that having non-parallel configuration phase is not something nice when there are multiple CPU cores available...