Review all target options between build systems

[networkfusion]

trafficstars

Details about Problem

The build system has become a mess, and multiple targets are out of sync with the CI.

Description

Compare and review target build options across all all build systems and targets to ensure parity.

Build systems include:

• Azure-CI yaml (azure-pipelines.yml and azure-pipelines-nightly.yml)
• Build PS (build.ps1)
• cmake-variants.json (included for each target (folder) and .vscode/cmake-variants.TEMPLATE.json)
• VS settings (CMakeSettings.SAMPLE.json and CMakeSettings.json)
• Dev container cmake-variants-DEVCONTAINER.json
• community targets (that may use some of the above)...

All of these are independent and can become/are out of sync

[networkfusion]

I cannot help feel that we can do better here... Lets perhaps open a dissussion about what should be the source of truth, and/or what build systems we want to continue supporting out of the box.

e.g. lets make the variants file for each target the source and use ConvertFrom-Json https://devblogs.microsoft.com/scripting/working-with-json-data-in-powershell/ to parse into the CI? or... the other way around...

In an ideal world, we "should" be able to parse it in directly, and since Cmake-Tools VSCode extension is now under the direct contol of MS, might be worth linking up to make it compatible with Azure CI?!

[josesimoes]

@networkfusion this is hard to accomplish that's why it is the mess that you see there. This is serving:

1. local build with toolchain in VS Code
2. local build with dev container in VS Code.
3. local build with Visual Studio (full version)
4. Azure Pipeline build

As those pickup build definitions from different locations/file, it's hard to keep those in sync in a an automated fashion.

• If the cmake-variants on each target folder are the base line, you have to perform an extra parsing to feed those in the pipeline, because several of those don't belong into the pipelines var BuildOptions.
• If the Azure Pipelines are the base line and you try to automate the generation of the cmake-variants, you would be missing several configs, which are different from target to target.
• For STM32 targets you haver one cmake-config per target. In ESP32 there are mixed stored in a single file and some are grouped/associated by the series.
• Even if you keep the cmake-templates in sync with the pipelines, you would be missing the "port" of those changes to the general cmake-variants in each developer instance because that depends on manually running the PS1 to update it. That as the potential to make people mad as it would overwrite whatever they've changed to tweak their local builds.

TBO, I don't think we need a "source of truth" because there isn't one... There is the pipeline build with it's configuration, it servers the purpose of building the targets we offer "ready to flash" and there are the local builds which are available to have developers building their own stuff. I think that, at most we can improve the docs explaining the cmake variants and what to look at when someone what's to do a local build mirroring the pipeline.

[networkfusion]

@josesimoes , I am not disagreeing, but do thing there are improvements to be made (working on them).

For example, the cmake-variants-DEVCONTAINER.json is basically a superset of all other cmake-variants.json files, and would (possibly) be the best source of truth (with only minimal adjustments required through a PS script).

Would it not be better for the CI to call the build.PS1 file, or use the cmake-variants.json file, rather than defining its own (possibily out of sync parameter options)?

As a start... I am trying to create a POC using https://devblogs.microsoft.com/cppblog/cmake-presets-integration-in-visual-studio-and-visual-studio-code/ , this would hopefully (and eventually) solve all 3 problems. But... I am still struggling to get it working fully.

My current progress is CMakePresets.json.txt and once it works can (once I work out how) be expanded using CMakeUserPresets.json (in line with https://docs.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-170) and even usable in the CI (e.g. https://github.com/meltybk/samples/blob/6f305dff21a5c313ce28028b1aeb9f1002b45e2b/azure-pipelines.yml). This might not be the full answer, but will certainly be a big step forwards if it does work, since it would be compatible with VS2019/22 VSCode and the CI...

[networkfusion]

Here is a working POC (for VSCode).... CMakePresets.json.txt

At least, as a backup for further improvements....

[Ellerbach]

Just one thing: keep in mind the path in the dev container variant is not the same as in Windows. It has to be specified. Not sure how this impact the work but definitely something to keep in mind.

[networkfusion]

CMakePresets2.json.txt

Here is the next version (for a single target (F769I)) that works with both VS2019 (and probably 22) and VSCode. I have also added what should work with a dev container (although not yet able to test). At this point, it would be helpful for an expert in this area to help improve it... (taking into account hte dev container properly, the current options and multiple target boards)

[networkfusion]

I (think) this can be closed now... Are we in agreement @josesimoes ?

[josesimoes]

I'm good with our build system. There's always room for improvement of course, but I believe it's performing well.

[networkfusion]

Closing as improved. Will open a new issue when it is deemed necessary.