Open
Description
Describe the problem
When using make & makefiles a build followed immediately by another build causes nothing to be compiled or linked because make honors timestamps when building the dependency graph. The arduino-cli does not seem to do this. If i do a compile and then immediately another compile it rebuild the binary the second time. This is unnecessary and consumes time & processing power. Arduido-cli should behave like make.
To reproduce
Do a compile and the immediately another compile.
Expected behavior
The second compile should exit without doing anything.
Arduino CLI version
arduino-cli Version: 0.29.0 Commit: 76251df Date: 2022-11-17T09:21:37Z
Operating system
Windows
Operating system version
Windows 11
Additional context
Additional reports
- Sketch re-compiled when unnecessary #1996 (comment)
- Sketch re-compiled when unnecessary #1996 (comment)
- Avoid unnecessary compiling when clicking "Upload" button #2015
- Retry upload arduino-ide#270
- Avoid recompiling when uploading multiple times without making changes arduino-ide#1789
- "Upload" also does a build/verify arduino-ide#2103
- Avoid recompiling if the code has not changed arduino-ide#2140
- Sketch gets recompiled for every verification and upload Arduino#2335
- Switch in IDE to skip compiling Arduino#6625
- feature request: upload without compile Arduino#8149
- https://forum.arduino.cc/t/recompiling-on-upload-without-code-change/897394
- https://forum.arduino.cc/t/arduino-ide-compiles-every-time-and-is-slow/1044606
- https://forum.arduino.cc/t/ide-2-0-2-why-recompile-every-time/1064264
- https://forum.arduino.cc/t/upload-without-compile/1175639
- https://forum.arduino.cc/t/still-no-upload-without-recompile/1300242
- https://forum.arduino.cc/t/resolving-libraries-every-time-too-long/1322487
Issue checklist
- I searched for previous reports in the issue trackerI verified the problem still occurs when using the nightly buildMy report contains all necessary details
Activity
scottchiefbaker commentedon Feb 22, 2023
umbynos commentedon May 29, 2023
Actually the compile process does compile different "parts" of a sketch:
The first two already use a caching mechanism. The sketch currently does not have this behavior. Unfortunately, it's not that easy because the library detection can't be cached.
[-]Compiles when unnecessary[/-][+]Sketch re-compiled when unnecessary[/+]weswitt commentedon May 29, 2023
cmaglie commentedon Jun 6, 2023
Just to give some context: the slow part is the automatic library detection, the difference between a Makefile and a sketch is that with a Makefile the used libraries are given (it's the user's duty to write the paths manually or with the aid of other tools like cmake/automake). With a sketch, the libraries are automatically discovered.
Moving to a more pragmatic example, this is how a clean compile looks like:
and this is the same compile after immediately re-running it:
The second run is resuing almost everything from the cache except on:
Detecting libraries used...phase. This is the slowest. It's also not easy to fully cache and/or parallelize, because it actually requires runninggcc -Eon the sketch to check if there are missing includes (the installed libraries may have changed in the meantime). The same process must be done on all the compilation units in the included libraries: this may become very long if there are hundreds of files.About problem 2: this could be surely improved even if the gain is very little compared to the overall compilation time. I opened #2204 to track this one.
About problem 1: there is an interesting tentative here #1735 but it's flaky, BTW it gave some ideas to work on to actually get to a correct solution, I made a preparatory PR here #1766 to clean up the library resolution code in preparation of a bigger refactoring.
weswitt commentedon Jun 6, 2023
apos commentedon Jun 10, 2023
2 remaining items