Invalidate core cache if external dependency was modified

[Bodmer]

Describe the request

Add an option to prevent caching or to clear the cache.

Describe the current behavior

Some libraries and board package use a build_opt.h file in the sketch to configure the board core or indeed a library.

See for example here: https://github.com/stm32duino/wiki/wiki/Customize-build-options-using-build_opt.h

The problem is that swapping between sketches that use slightly different build options will fail to use the new options.

The crude work-around is to select a different processor, compile (even if this generates errors) so the cache is deleted/changed, then swap back to the correct processor and compile again. Clearly this is a rather clumsy way of forcing a cache clear.

If there is a neater solution then please post it. Otherwise please update the IDE with an option to use/not use caching. Clearly this option can default to the current arrangement.

Demo

$ arduino-cli version

arduino-cli.exe  Version: git-snapshot Commit: b82a519e Date: 2023-10-21T11:43:36Z

$ export ARDUINO_DIRECTORIES_DATA="/tmp/arduino-cli-directories/data"  # Use a throwaway directories.data for the demo

$ arduino-cli core install arduino:[email protected]  # Install an arbitrary platform for use in the demo

[...]

$ echo 'compiler.cpp.extra_flags="-I{build.source.path}"' > "${ARDUINO_DIRECTORIES_DATA}/packages/arduino/hardware/avr/1.8.6/platform.local.txt"  # Add the sketch folder to the compiler's search path

$ printf '#include <SomeSketchHeaderFile.h>\n#ifdef MACRO_FROM_SKETCH_HEADER_FILE\n#error "MACRO_FROM_SKETCH_HEADER_FILE is defined"\n#endif' > "${ARDUINO_DIRECTORIES_DATA}/packages/arduino/hardware/avr/1.8.6/cores/arduino/DependsOnSomeSketchHeaderFile.cpp"  # Add source file to the core that depends on a header file from the sketch

$ SKETCH_PATH="/tmp/FooSketch"

$ arduino-cli sketch new "$SKETCH_PATH"

$ touch "${SKETCH_PATH}/SomeSketchHeaderFile.h"

$ arduino-cli compile --fqbn arduino:avr:uno --verbose "$SKETCH_PATH" # Create core cache

[...]
Compiling core...
[...]
Archiving built core (caching) in: C:\Users\per\AppData\Local\Temp\arduino\cores\arduino_avr_uno_5958a1d0f1df880da5aa74c2f1a07415\core.a
[...]

$ echo "#define MACRO_FROM_SKETCH_HEADER_FILE" > "${SKETCH_PATH}/SomeSketchHeaderFile.h"

$ arduino-cli compile --fqbn arduino:avr:uno --verbose "$SKETCH_PATH"

[...]
Compiling core...
Using precompiled core: C:\Users\per\AppData\Local\Temp\arduino\cores\arduino_avr_uno_5958a1d0f1df880da5aa74c2f1a07415\core.a
[...]
Sketch uses 444 bytes (1%) of program storage space. Maximum is 32256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.

🙁 The cached core was used even though a dependency had changed.

The expected result is that the cache would be invalidated, which in this simplified demo would result in the compilation failing:

$ arduino-cli compile --clean --fqbn arduino:avr:uno --verbose "$SKETCH_PATH"

[...]
Compiling core...
[...]
C:\Users\per\AppData\Local\Temp\arduino-cli-directories\data\packages\arduino\hardware\avr\1.8.6\cores\arduino\DependsOnSomeSketchHeaderFile.cpp:3:2: error: #error "MACRO_FROM_SKETCH_HEADER_FILE is defined"
 #error "MACRO_FROM_SKETCH_HEADER_FILE is defined"
  ^~~~~
[...]

Arduino CLI version

Original report

0.27.1

Last verified with

b82a519e

Operating system

Windows

Operating system version

10/11

Additional context

See:

https://github.com/stm32duino/wiki/wiki/Customize-build-options-using-build_opt.h

Note that the suggested approach of closing the IDE and opening again no longer works.

Related

• https://github.com/arduino/arduino-ide/issues/419
• https://forum.arduino.cc/t/cache-clear-option/1040106

Issue checklist

• [X] I searched for previous requests in the issue tracker
• [X] I verified the feature was still missing when using the nightly build
• [X] My request contains all necessary details