arduino-cli Generated compilation database compiler argument missing extension breaks compilation on Windows.

Generated compilation database compiler argument missing extension breaks compilation on Windows.

Open Falven opened this issue 3 years ago • 25 comments

Bug Report

The generated compilation database has a bug/is missing some compiler commands. If you are trying to use the compilation database as intellisense configuration as is done in pretty much any IDE, for example, clangd under VSCode, the generated compile_commands.json MUST explicity include the system header include directories otherwise clangd or other intellisense engines cannot find them. This is because Arduino uses different compilers in non-standard locations, with different system includes per compiler, as such the intellisense engines will not know where to find the system includes.

Current behavior

Here is an example file entry for the generated compilation database:

{
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0/CMSIS/Include/",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0/CMSIS/Device/ATMEL/",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino/api/deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino/api/deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "c:\\Users\\falve\\OneDrive\\Documents\\Arduino\\libraries\\oprintstream\\build\\sketch\\oprintstream.ino.cpp",
   "-o",
   "c:\\Users\\falve\\OneDrive\\Documents\\Arduino\\libraries\\oprintstream\\build\\sketch\\oprintstream.ino.cpp.o"
  ],
  "file": "c:\\Users\\falve\\OneDrive\\Documents\\Arduino\\libraries\\oprintstream\\build\\sketch\\oprintstream.ino.cpp"
 }

Expected behavior

Here is what the file entry from the compilation database should look like:

{
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0/CMSIS/Include/",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0/CMSIS/Device/ATMEL/",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino/api/deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino/api/deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "-isystem C:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1",
   "-isystem C:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\arm-none-eabi",
   "-isystem C:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\backward",
   "-isystem C:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include",
   "-isystem C:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include-fixed",
   "-isystem C:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include",
   "c:\\Users\\falve\\OneDrive\\Documents\\Arduino\\libraries\\oprintstream\\build\\sketch\\oprintstream.ino.cpp",
   "-o",
   "c:\\Users\\falve\\OneDrive\\Documents\\Arduino\\libraries\\oprintstream\\build\\sketch\\oprintstream.ino.cpp.o"
  ],
  "file": "c:\\Users\\falve\\OneDrive\\Documents\\Arduino\\libraries\\oprintstream\\build\\sketch\\oprintstream.ino.cpp"
 }

Environment

$ arduino-cli version
arduino-cli.exe alpha Version: 0.18.1 Commit: b3cf8e19 Date: 2021-04-13T13:08:30Z

Additional context

As an example, look at the compilation database, c_cpp_properties.json, that the Arduino vscode extension generates for it's intellisense configuration. They add on the system include directories.

{
    "version": 4,
    "configurations": [
        {
            "name": "Arduino",
            "compilerPath": "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++",
            "compilerArgs": [
                "-mcpu=cortex-m0plus",
                "-mthumb",
                "-w",
                "-std=gnu++11",
                "-ffunction-sections",
                "-fdata-sections",
                "-fno-threadsafe-statics",
                "-nostdlib",
                "--param",
                "-fno-rtti",
                "-fno-exceptions"
            ],
            "intelliSenseMode": "gcc-x64",
            "includePath": [
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
                "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1",
                "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\arm-none-eabi",
                "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\backward",
                "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include",
                "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include-fixed",
                "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include"
            ],
            "forcedInclude": [
                "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\Arduino.h"
            ],
            "cStandard": "c11",
            "cppStandard": "c++11",
            "defines": [
                "F_CPU=48000000L",
                 ...
        }
    ]
}

May 10 '21 14:05 Falven

Hm, one other solution to this (which I use), is to whitelist the Arduino compilers with clangd, e.g.:

clangd --query-driver="$HOME/.arduino15/packages/*/tools/**/*-gcc,$HOME/.arduino15/packages/*/tools/**/*-g++"

This allows clangd to query the compiler for its internal include paths directly. Maybe a bit hardcoded, but it works well in practice.

As for adding the include paths to the compilation database, I wonder if this is really common practice? The json format lacks a way to distinguish these includes from the actual include paths passed to the compiler, making the command listed in the database not the actual command that was ran. I guess that even if clangd manages to query the compiler, the worst case would be a duplicate system include path, which should be harmless, but still.

Also, I'm not entirely sure how we can do this portably and reliably: arduino-cli does not necessarily know how a compiler commandline looks exactly (it might not even be gcc...). There's a few challenges I can see:

What command to run to figure out the system includes (and maybe defines)? This probably requires a new recipe, since autogenerating this from the recipe for a given compilation might be fragile (I can imagine that options like -mcpu or so might infliuence the list, so you probably cannot just run the plain gcc ommand).
Should you run gcc a second time to generate includes (and maybe defines) again for every compiled file? You probably cannot assume that the list is the same for every compiled file (might differ between C and C++ and depending on the platform it could even be different for each file).
Where in the command listed in the compilation database should the extra include paths be inserted? I can imagine that ordering can be relevant, especially if there are explicit -isystem options in the recipe too.

So I wonder if we can do this reliably at all, or if we should just stick to listing the actual commands that were ran and rely on clangd to figure out the rest with --query-driver...

May 10 '21 14:05 matthijskooijman

I also wonder: You paste a c_cpp_properties.json file from vscode, but how is that file used? Does clangd understand it? Or is it input to vscode (seems like, from a quick web search)? Is your suggestion maybe that arduino-cli should generate a c_cpp_properties.json file in addition to the compile_commands.json maybe? If so, do you know if the format of that file is documented anywhere?

May 10 '21 14:05 matthijskooijman

Hm, one other solution to this (which I use), is to whitelist the Arduino compilers with clangd, e.g.:
clangd --query-driver="$HOME/.arduino15/packages/*/tools/**/*-gcc,$HOME/.arduino15/packages/*/tools/**/*-g++"
This allows clangd to query the compiler for its internal include paths directly. Maybe a bit hardcoded, but it works well in practice.

If this works, I agree this would be the optimal solution as the work is being done from the compiler. It's unfortunate that this isn't something easily configurable/you have to know about it.

And I realize that this may be off-topic at this point, but it could potentially help another person trying to use clangd with VSCode and Arduino. I am not able to get this to work under Windows 10 x64. I have my .vscode/settings.json:

"clangd.arguments": [
    "--query-driver=\"%LOCALAPPDATA%\\Arduino15\\packages\\*\\tools\\**\\*-gcc.exe,%LOCALAPPDATA%\\Arduino15\\packages\\*\\tools\\**\\*-g++.exe\""
],

And it does not work. I suspect it may have something to do with the generated compile_commands.json paths mixing forward and backslashes:

{
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++",
   ...

I also tried with the command you pasted:

"clangd.arguments": [
    "--query-driver=\"%LOCALAPPDATA%/.arduino15/packages/*/tools/**/*-gcc,$HOME/.arduino15/packages/*/tools/**/*-g++\""
],

As for adding the include paths to the compilation database, I wonder if this is really common practice? The json format lacks a way to distinguish these includes from the actual include paths passed to the compiler, making the command listed in the database not the actual command that was ran. I guess that even if clangd manages to query the compiler, the worst case would be a duplicate system include path, which should be harmless, but still.

That is true, I do think that this functionality is as you said part of the compiler. That would make more sense.

Also, I'm not entirely sure how we can do this portably and reliably: arduino-cli does not necessarily know how a compiler commandline looks exactly (it might not even be gcc...). There's a few challenges I can see:

What command to run to figure out the system includes (and maybe defines)? This probably requires a new recipe, since autogenerating this from the recipe for a given compilation might be fragile (I can imagine that options like -mcpu or so might infliuence the list, so you probably cannot just run the plain gcc ommand).

Should you run gcc a second time to generate includes (and maybe defines) again for every compiled file? You probably cannot assume that the list is the same for every compiled file (might differ between C and C++ and depending on the platform it could even be different for each file).

Where in the command listed in the compilation database should the extra include paths be inserted? I can imagine that ordering can be relevant, especially if there are explicit -isystem options in the recipe too.

So I wonder if we can do this reliably at all, or if we should just stick to listing the actual commands that were ran and rely on clangd to figure out the rest with --query-driver...

I also wonder: You paste a c_cpp_properties.json file from vscode, but how is that file used? Does clangd understand it? Or is it input to vscode (seems like, from a quick web search)? Is your suggestion maybe that arduino-cli should generate a c_cpp_properties.json file in addition to the compile_commands.json maybe? If so, do you know if the format of that file is documented anywhere?

The c_cpp_properties.json file is generated by the VSCode Arduino extension and is used by the VSCode C/C++ extension for intellisense and formatting. I believe the VSCode C/C++ extension uses fuzzy parsing which gives a lot of false results.

I personally much prefer the clangd extension for VS Code as I am used to it, it has arguably better, more modern intellisense, and also includes clang-format and clang-tidy. the clangd extension for VS Code does not interact with the c_cpp_properties.json file at all. It just uses the clangd CLI and the generated compile_commands.json.

The c_cpp_properties.json file is only generated for the VSCode C/C++ extension by the VSCode Arduino extension. Whether the CLI should generate that file I think would be a topic of discussion with the folks supporting the VSCode Arduino extension.

May 11 '21 17:05 Falven

I created a barebones VS Code project here to test the driver configuration. In this project you would see that clangd intellisense says Use of undeclared identifier 'std'clang(undeclared_var_use) because it does not know where to find <type_traits>.

And I can see in the clangd output that the --query-driver argument is being passed in:

I[14:00:01.840] argv[0]: c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe
I[14:00:01.840] argv[1]: --query-driver="%LOCALAPPDATA%\Arduino15\packages\*\tools\**\*-gcc.exe,%LOCALAPPDATA%\Arduino15\packages\*\tools\**\*-g++.exe"
I[14:00:01.872] Loaded compilation database from c:\Users\falve\OneDrive\Desktop\clangd-arduino\build/compile_commands.json
I[14:00:01.876] Loaded compilation database from c:\Users\falve\OneDrive\Desktop\clangd-arduino\build\compile_commands.json

I added --log=verbose and am getting some more informative output now: When pasting the full path to the executable, I get the following error message, confirming my suspicions about the format of the generated compile_commands.json path separators.

E[14:14:21.550] System include extraction: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++ does not exist.

Yep, when I correct the last few separators:

"--query-driver=C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-gcc.exe,C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++.exe"

clangd will output:

V[14:17:51.042] System include extraction: not allowed driver C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++

Now the question is: "Is the CLI the one generating these incorrect paths in the Windows compilation database or is it the VSCode Arduino extension after it calls the CLI?"

Oh @matthijskooijman! I think that you can either specify the driver in the compile_commands.json, which would take the manual configuration guesswork away from the developer, or you already are, but because the paths are messed up in Windows, it does not use that compiler as the driver. See the second to last comment from kadircet in this thread.

I have confirmed that calling the CLI directly generates these incorrect paths (It is not the Arduino extension doing this):

$ arduino-cli.exe compile --build-path "./build" --fqbn "arduino:samd:mkrwifi1010" --only-compilation-database --verbose
Using board 'mkrwifi1010' from platform in folder: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11
Using core 'arduino' from platform in folder: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11
Detecting libraries used...
"C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++"

So I believe this is actually a bug with the Arduino CLI rather than a feature request.

Edit 2:

Interestingly, after changing the generated compile_commands.json paths, I get the following error from clangd:

E[17:41:12.228] System include extraction: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++ does not exist.

May 11 '21 17:05 Falven

So to clear something up, @matthijskooijman, the --query-driver command only whitelists potential compilers, but the actual driver commands come from the compilers. So query-driver is useless unless you want to tell it which compiler to use specifically from the same toolchain.

I see now, when I remove the whitelist (which does not work on windows because of path mismatches from the arduino-cli-generated database) the following output:

V[17:58:40.955] Driver produced command: cc1 -cc1 -triple thumbv6m-none-unknown-eabi -fsyntax-only -disable-free -disable-llvm-verifier -discard-value-names -main-file-name clangd-arduino.ino -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -nostdsysteminc -target-cpu cortex-m0plus -target-feature +soft-float-abi -target-feature -crc -target-feature -crypto -target-feature -sha2 -target-feature -aes -target-feature -dotprod -target-feature -dsp -target-feature -mve -target-feature -mve.fp -target-feature -fullfp16 -target-feature -ras -target-feature -bf16 -target-feature -sb -target-feature -i8mm -target-feature -lob -target-feature -cdecp0 -target-feature -cdecp1 -target-feature -cdecp2 -target-feature -cdecp3 -target-feature -cdecp4 -target-feature -cdecp5 -target-feature -cdecp6 -target-feature -cdecp7 -target-feature -hwdiv-arm -target-feature -hwdiv -target-feature -fp16fml -target-feature +strict-align -target-abi aapcs -mfloat-abi soft -fallow-half-arguments-and-returns -fno-split-dwarf-inlining -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -ffunction-sections -fdata-sections -resource-dir c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0 -D F_CPU=48000000L -D ARDUINO=10607 -D ARDUINO_SAMD_MKRWIFI1010 -D ARDUINO_ARCH_SAMD -D USE_ARDUINO_MKR_PIN_LAYOUT -D __SAMD21G18A__ -D USB_VID=0x2341 -D USB_PID=0x8054 -D USBCON -D USB_MANUFACTURER="Arduino LLC" -D USB_PRODUCT="Arduino MKR WiFi 1010" -D USE_BQ24195L_PMIC -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0\CMSIS\Include\ -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0\CMSIS\Device\ATMEL\ -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated-avr-comp -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010 -internal-isystem C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/clang-runtimes\arm-none-eabi\include\c++\v1 -internal-isystem c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0\include -internal-isystem C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/clang-runtimes\arm-none-eabi\include -Os -w -std=gnu++11 -fdeprecated-macro -fdebug-compilation-dir c:\Users\falve\.vscode\extensions\vsciot-vscode.vscode-arduino-0.4.2\out\src -ferror-limit 19 -fno-rtti -fno-signed-char -fgnuc-version=4.2.1 -fno-threadsafe-statics -vectorize-loops -vectorize-slp -faddrsig -x c++-header c:\source\repos\clangd-arduino\clangd-arduino.ino

But nowhere in there are the C++ includes from the toolchain Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\arm-none-eabi\include\c++\7.2.1 hence my intellisense not working.

Which makes sense, because if you look at the c_cpp_properties.json file that the VS Code Arduino Extension generates for the C/C++ Extension, you can see that the casing for the C++ includes is different than the top 6:

"includePath": [
    "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
    "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
    "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
    "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
    "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
    "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
    "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1",
    "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\arm-none-eabi",
    "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\backward",
    "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include",
    "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include-fixed",
    "c:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include"
],

It is all lowercase, which leads me to believe that the extension is adding these on to the compiler driver output.

If I add those last include directories as shown below:

 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\cmsis\\4.5.0\\cmsis\\include\\",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\cmsis-atmel\\1.2.0\\cmsis\\device\\atmel\\",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\arm-none-eabi",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\backward",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include-fixed",
   "-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp.o"
  ],
  "file": "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp"
 },

Then intellisense is happy. Note: all of those includes should be -isystem; they are currently being included with -I in the generated arduino-cli compilation file, which can confuse some intellisense engines or code analyzers (such as clang-tidy) that rely on this metadata to know whether a header is a system header or not. And also the paths need to be fixed on windows...

May 11 '21 22:05 Falven

@Falven, ok so for the query driver option, the slashes are problematic? Did you try using only backward (or only forward) slashes in query-driver and manually modifying compile_commands.json to match? Does it work then? If so, then I guess the mixed slashes in compile_commands.json prevent clangd from working and should be fixed in arduino-cli.

I think that for executing a command, these mixed slashes are acceptable on Windows, so there has not been any need to fix them so far. The mixed slashes come from the fact that some paths are autofilled (so use \ on Windows) and some come from platform.txt (and typically use '/'). Note that fixing this for command arguments is probably not possible (arduino-cli does not know which arguments are paths and must be changed and which are not and must be left alone), but for the compiler command itself, it is safe to assume that is a path and it should be fixable.

So to clear something up, @matthijskooijman, the --query-driver command only whitelists potential compilers, but the actual driver commands come from the compilers. So query-driver is useless unless you want to tell it which compiler to use specifically from the same toolchain.

I do not understand what you mean here? AFAIU, the idea is that --query-driver should match the compiler command listed in compile_commands.json and then clangd will query the compiler from compile_commands.json for all needed options. It might be that it does this by asking the compiler for the driver used and then querying the driver for options maybe, but all this is handled by clangd and AFAIK you only need to whitelist the outer compiler.

May 12 '21 06:05 matthijskooijman

@matthijskooijman I know I gave a lot of information throughout my debugging, so I apologize for that. I think first we need to come to a common conclusion of what the --query-driver option does.

From the clangd thread I linked, @kadircet says: "--query-driver option on itself just specifies an allowed list of drivers to execute. The real driver paths are deduced from compile commands."

What I gathered from this, then, is the following: the --query-driver option is not necessary, and only works as a whitelist. If there are multiple compilers in one toolchain, you can tell clangd which one to use from the toolchain. As @kadircet mentioned, The real driver path is deduced from the compile command in the compile_commands.json. In our case, the following line:

...
"arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++",
...

What we can then deduce is that if you don't provide the --query-driver option, clangd will still query all provided compilers in the compile_commands.json for include paths. Which I believe is correct and is the behavior that I observed in the clangd output I pasted in comments above.

I think the mixed slashes are a potential issue for certain applications. I think they are the underlying issue with clangd, for example, if you are trying to use this --query-driver option with the clangd CLI, it will not consider:

...\7-2017q4\bin\arm-none-eabi-g++

and

...\7-2017q4/bin/arm-none-eabi-g++

to be the same compiler. As can be observed from my previous comments.

May 12 '21 12:05 Falven

First of, as for terminology, I just realized that the command from compile-commands.json is actually the driver command, which internally executes the compiler (I think I might have had them reversed in my earlier comment). And clangd needs to figure out the compiler command used internally, since that includes all include paths explicitly. Below, I'm using the correct terms.

What we can then deduce is that if you don't provide the --query-driver option, clangd will still query all provided compilers in the compile_commands.json for include paths.

I do not think this is true. I believe that the query-driver option is a security feature, since clangd needs to execute the driver to query it, so it only does that for some known-safe driver paths, plus explicitly whitelisted ones with query-driver. If you do not specify query-driver and you end up with a driver in compile_commands.json that is not (default) whitelisted, I suspect that clangd will just assume you're using some default driver (i.e. /usr/bin/gcc, or maybe clang or so) and query that, or just autogenerate a default compiler commandline and includepaths.

I think the mixed slashes are a potential issue for certain applications. I think they are the underlying issue with clangd, for example, if you are trying to use this --query-driver option with the clangd CLI, it will not consider:

I don't think this is entirely true: From your earlier comment, I see that if you specify all \ in --query-driver and mixed (back)slashes in compile_commands, then clangd complains "not allowed driver". This suggests that clangd does not do any slash normalization of commands, it just checks them against the whitelist verbatim. Seems like a sane approach.

But if you change the --query-driver option to match the mixed (back)slashes from compile_commands.json, then the whitelist passes, and (according to your post) clangd now complains with:

E[14:14:21.550] System include extraction: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++ does not exist.

So that suggests to me that clangd indeed does not like the mixed slashes, but the problem is not in the whitelist, but in the execution of the driver.

So, the proper fix is probably to produce a compile_commands.json with consistent slashes.

Which is why I suggested you test this fix manually, i.e. from my previous comment:

Did you try using only backward (or only forward) slashes in query-driver and manually modifying compile_commands.json to match? Does it work then? If so, then I guess the mixed slashes in compile_commands.json prevent clangd from working and should be fixed in arduino-cli.

May 12 '21 12:05 matthijskooijman

Paying attention to slashes, we can see that:

"--query-driver=C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-gcc.exe,C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++.exe"

outputs

V[14:17:51.042] System include extraction: not allowed driver C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++

and

"--query-driver=C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-gcc,C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4/bin/arm-none-eabi-g++"

outputs

E[08:27:22.735] System include extraction: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++ does not exist.

I think the last output is a good hint. I think the problem then is that the compiler doesn't have these system include paths to extract defined in it's "driver".

"-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1",
"-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\arm-none-eabi",
"-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include\\c++\\7.2.1\\backward",
"-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include",
"-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\lib\\gcc\\arm-none-eabi\\7.2.1\\include-fixed",
"-isystemc:\\users\\falve\\appdata\\local\\arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\arm-none-eabi\\include",

Edit: Maybe this will be a hint, but I was following this issue which seems to be similar. And the output from the suggested command is the following:

Using built-in specs.
COLLECT_GCC=C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++.exe
Target: arm-none-eabi
Configured with: /tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/src/gcc/configure --build=x86_64-linux-gnu --host=i686-w64-mingw32 --target=arm-none-eabi --prefix=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw --libexecdir=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/lib --infodir=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/share/doc/gcc-arm-none-eabi/info --mandir=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/share/doc/gcc-arm-none-eabi/man --htmldir=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/share/doc/gcc-arm-none-eabi/html --pdfdir=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-mingw-wildcard --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-headers=yes --with-newlib --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/arm-none-eabi --with-libiconv-prefix=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/build-mingw/host-libs/usr --with-gmp=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/build-mingw/host-libs/usr --with-mpfr=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/build-mingw/host-libs/usr --with-mpc=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/build-mingw/host-libs/usr --with-isl=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/build-mingw/host-libs/usr --with-libelf=/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/build-mingw/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for Arm Embedded Processors 7-2017-q4-major' --with-multilib-list=rmprofile
Thread model: single
gcc version 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204] (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 
COLLECT_GCC_OPTIONS='-E' '-v'
 c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/cc1plus.exe -E -quiet -v -iprefix c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/ -isysroot c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../arm-none-eabi -D__USES_INITFINI__ clangd-arduino.ino
ignoring duplicate directory "c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/lib/gcc/../../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1"
ignoring duplicate directory "c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/lib/gcc/../../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi"
ignoring duplicate directory "c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/lib/gcc/../../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward"
ignoring duplicate directory "c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/lib/gcc/../../lib/gcc/arm-none-eabi/7.2.1/include"
ignoring nonexistent directory "c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../arm-none-eabi/tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/install-mingw/lib/gcc/arm-none-eabi/7.2.1/../../../../include"
ignoring duplicate directory "c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/lib/gcc/../../lib/gcc/arm-none-eabi/7.2.1/include-fixed"
ignoring duplicate directory "c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/lib/gcc/../../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include"
ignoring nonexistent directory "c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../arm-none-eabi/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1
 c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi
 c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward
 c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include
 c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed
 c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include
End of search list.
COMPILER_PATH=c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/;c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/;c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/bin/
LIBRARY_PATH=c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/;c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/;c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/lib/;c:/users/falve/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../arm-none-eabi/lib/
COLLECT_GCC_OPTIONS='-E' '-v'

It looks like it is ignoring all of the c++ include directories that we are missing.

May 12 '21 12:05 Falven

I think the last output is a good hint. I think the problem then is that the compiler doesn't have these system include paths to extract defined in it's "driver".

I don't know what you mean by this exactly, but I think it is wrong. What this means is that clangd tries to run the driver to query it for include paths and fails, because it does not support forward slashes in the driver executable path.

It looks like it is ignoring all of the c++ include directories that we are missing.

Do you mean the "ignoring duplicate directory" messages? Those are normal and just indicate that it's going to output these directories just once instead of multiple times. As you can see, they are output after the #include <...> search starts here: line.

As I suggested before:

Did you try using only backward (or only forward) slashes in query-driver and manually modifying compile_commands.json to match? Does it work then?

This is the third time I've given the above suggestion, which I think is the way forward to a fix, but it seems you're ignoring it, so I'm slowly running out of patience here. If my suggestion is unclear, please let me know, but otherwise please try this.

May 12 '21 14:05 matthijskooijman

As I suggested before:

Did you try using only backward (or only forward) slashes in query-driver and manually modifying compile_commands.json to match? Does it work then?

This is the third time I've given the above suggestion, which I think is the way forward to a fix, but it seems you're ignoring it, so I'm slowly running out of patience here. If my suggestion is unclear, please let me know, but otherwise please try this.

@matthijskooijman It is not my intention to ignore anything, my apologies.

I have modified the generated compile_commands.json to use windows-style path separators:

[
 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp.o"
  ],
  "file": "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp"
 },
 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010\\variant.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\core\\variant.cpp.o"
  ],
  "file": "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010\\variant.cpp"
 }
]

I have set the query driver argument as such:

I[11:22:59.419] Working directory: c:\source\repos\clangd-arduino
I[11:22:59.419] argv[0]: c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe
I[11:22:59.419] argv[1]: --log=verbose
I[11:22:59.419] argv[2]: --query-driver=C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gcc,C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++

And the relevant output from clangd follows:

I[11:22:59.486] Loaded compilation database from c:\source\repos\clangd-arduino\build/compile_commands.json
V[11:22:59.486] Broadcasting compilation database from c:\source\repos\clangd-arduino
E[11:22:59.487] System include extraction: C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++ does not exist.
I[11:22:59.487] ASTWorker building file c:\source\repos\clangd-arduino\clangd-arduino.ino version 1 with command inferred from c:\source\repos\clangd-arduino\build\sketch\clangd-arduino.ino.cpp
[c:\Users\falve\.vscode\extensions\vsciot-vscode.vscode-arduino-0.4.2\out\src]
C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++ --target=arm-none-eabi --driver-mode=g++ -mcpu=cortex-m0plus -mthumb -c -g -Os -w -ffunction-sections -fdata-sections -fno-threadsafe-statics -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -DF_CPU=48000000L -DARDUINO=10607 -DARDUINO_SAMD_MKRWIFI1010 -DARDUINO_ARCH_SAMD -DUSE_ARDUINO_MKR_PIN_LAYOUT -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x8054 -DUSBCON -DUSB_MANUFACTURER="Arduino LLC" -DUSB_PRODUCT="Arduino MKR WiFi 1010" -DUSE_BQ24195L_PMIC -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0\CMSIS\Include\ -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0\CMSIS\Device\ATMEL\ -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated-avr-comp -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010 -x c++-header -std=gnu++11 c:\source\repos\clangd-arduino\clangd-arduino.ino -fsyntax-only -resource-dir=c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0
V[11:22:59.489] Driver produced command: cc1 -cc1 -triple thumbv6m-none-unknown-eabi -fsyntax-only -disable-free -disable-llvm-verifier -discard-value-names -main-file-name clangd-arduino.ino -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -nostdsysteminc -target-cpu cortex-m0plus -target-feature +soft-float-abi -target-feature -crc -target-feature -crypto -target-feature -sha2 -target-feature -aes -target-feature -dotprod -target-feature -dsp -target-feature -mve -target-feature -mve.fp -target-feature -fullfp16 -target-feature -ras -target-feature -bf16 -target-feature -sb -target-feature -i8mm -target-feature -lob -target-feature -cdecp0 -target-feature -cdecp1 -target-feature -cdecp2 -target-feature -cdecp3 -target-feature -cdecp4 -target-feature -cdecp5 -target-feature -cdecp6 -target-feature -cdecp7 -target-feature -hwdiv-arm -target-feature -hwdiv -target-feature -fp16fml -target-feature +strict-align -target-abi aapcs -mfloat-abi soft -fallow-half-arguments-and-returns -fno-split-dwarf-inlining -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -ffunction-sections -fdata-sections -resource-dir c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0 -D F_CPU=48000000L -D ARDUINO=10607 -D ARDUINO_SAMD_MKRWIFI1010 -D ARDUINO_ARCH_SAMD -D USE_ARDUINO_MKR_PIN_LAYOUT -D __SAMD21G18A__ -D USB_VID=0x2341 -D USB_PID=0x8054 -D USBCON -D USB_MANUFACTURER="Arduino LLC" -D USB_PRODUCT="Arduino MKR WiFi 1010" -D USE_BQ24195L_PMIC -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0\CMSIS\Include\ -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0\CMSIS\Device\ATMEL\ -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino\api\deprecated-avr-comp -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010 -internal-isystem C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/clang-runtimes\arm-none-eabi\include\c++\v1 -internal-isystem c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0\include -internal-isystem C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/clang-runtimes\arm-none-eabi\include -Os -w -std=gnu++11 -fdeprecated-macro -fdebug-compilation-dir c:\Users\falve\.vscode\extensions\vsciot-vscode.vscode-arduino-0.4.2\out\src -ferror-limit 19 -fno-rtti -fno-signed-char -fgnuc-version=4.2.1 -fno-threadsafe-statics -vectorize-loops -vectorize-slp -faddrsig -x c++-header c:\source\repos\clangd-arduino\clangd-arduino.ino

May 12 '21 15:05 Falven

Hm, that surprises me, so just fixing the slashes is not going to help here, it seems.

Also, it seems you're editing your github comments a lot, which sometimes makes it a bit hard to reply, since I often read messages from the e-mail notification, which shows the initial version. Maybe you could use the preview function a bit more, and if you do have significant new findings, just post a new comment? In the initial version of your last comment, the clangd error was different (does not exist rather than not allowed driver), what's up with that?

I wonder if maybe vs is messing up the query driver argument somehow or something else? Maybe you could try running clangd manually directly to rule that out?

May 12 '21 15:05 matthijskooijman

Hm, that surprises me, so just fixing the slashes is not going to help here, it seems.

Also, it seems you're editing your github comments a lot, which sometimes makes it a bit hard to reply, since I often read messages from the e-mail notification, which shows the initial version. Maybe you could use the preview function a bit more, and if you do have significant new findings, just post a new comment? In the initial version of your last comment, the clangd error was different (does not exist rather than not allowed driver), what's up with that?

I wonder if maybe vs is messing up the query driver argument somehow or something else? Maybe you could try running clangd manually directly to rule that out?

Yeah, sorry. I made a typo - I had added .exe to the --query-driver path by accident.

Let's try running it manually:

PS C:\source\repos\clangd-arduino> C:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe --log=verbose --query-driver="C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tooackages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gcc,C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++"
clangd is a language server that provides IDE-like features to editors.

It should be used via an editor plugin rather than invoked directly. For more information, see:
        https://clangd.llvm.org/
        https://microsoft.github.io/language-server-protocol/

clangd accepts flags on the commandline, and in the CLANGD_FLAGS environment variable.

I[11:28:22.410] clangd version 12.0.0 (https://github.com/kirillbobyrev/llvm-project 8a844a3a321ea56da21af4d48433ad493b1bcfae)
I[11:28:22.411] PID: 16544
I[11:28:22.411] Working directory: C:\source\repos\clangd-arduino
I[11:28:22.412] argv[0]: C:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe
I[11:28:22.412] argv[1]: --log=verbose
I[11:28:22.412] argv[2]: --query-driver=C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tooackages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gcc,C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++
V[11:28:22.423] User config file is C:\Users\falve\AppData\Local\clangd\config.yaml
I[11:28:22.424] Starting LSP over stdin/stdout

Now I have to figure out how to send it a file open command over stdout?

May 12 '21 15:05 Falven

Yeah, sorry. I made a typo - I had added .exe to the --query-driver path by accident.

Wait, so when you add .exe to the query-driver argument, it says "does not exist", while if you omit it, it says "not allowed"? That's weird, since I expect that it checks the whitelist first and then checks for existence of the file, so that would mean that "does not exist" means that the whitelist passed (so that would mean that internally it adds the .exe to the command from comipile_commands.json and checks that against the whitelist). Unless it strips the .exe from the whitelist and compares that, but that means that adding or removing the .exe form the whitelist should not affect the result and the error message. Or, unless the existence check is done before the whitelist, but then changing the whitelist should not change the result of the existence check (i.e. change a "does not exist" into a "not allowed").

I'm thoroughly confused and running out of ideas....

Now I have to figure out how to send it a file open command over stdout?

Ah, right, you'll have to send it a file to start doing something, that might not be so easy...

May 12 '21 16:05 matthijskooijman

Yeah, sorry. I made a typo - I had added .exe to the --query-driver path by accident.

Wait, so when you add .exe to the query-driver argument, it says "does not exist", while if you omit it, it says "not allowed"? That's weird, since I expect that it checks the whitelist first and then checks for existence of the file, so that would mean that "does not exist" means that the whitelist passed (so that would mean that internally it adds the .exe to the command from comipile_commands.json and checks that against the whitelist). Unless it strips the .exe from the whitelist and compares that, but that means that adding or removing the .exe form the whitelist should not affect the result and the error message. Or, unless the existence check is done before the whitelist, but then changing the whitelist should not change the result of the existence check (i.e. change a "does not exist" into a "not allowed").

I'm thoroughly confused and running out of ideas....

Now I have to figure out how to send it a file open command over stdout?

Ah, right, you'll have to send it a file to start doing something, that might not be so easy...

Yeah, I couldn't figure out how, but you can see that argv[2], the --query-driver parameter is identical to when it was run by VS Code so I don't think that is the issue. But I reopened this issue with clangd. Seeing as this is likely not an issue with the Arduino CLI. However, if it turns out to be an issue with the Arduino CLI I will re-post here with more findings. I do appreciate you going out of your way to try and help. Tell me, can you confirm that on your Linux/OS X system the C++ include paths are extracted properly to the compile_commands.json? Maybe post a simple compile_commands.json from the sample project I posted to confirm?

May 13 '21 12:05 Falven

Tell me, can you confirm that on your Linux/OS X system the C++ include paths are extracted properly to the compile_commands.json?

On Linux, using arduino-cli and clangd running inside Vim with the YouCompleteMe plugin, it indeed works. I tried using your clangd-arduino.ino sketch:

$ arduino-cli compile --only-compilation-database --fqbn arduino:samd:mkrzero
$ ln -s /tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/compile_commands.json 
$ vim clangd-arduino.ino

The compile_commands.json for the .ino file looks like this:

[
 {
  "directory": "/home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino",
  "arguments": [
   "/home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRZERO",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x804f",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKRZero\"",
   "-I/home/matthijs/.arduino15/packages/arduino/tools/CMSIS/4.5.0/CMSIS/Include/",
   "-I/home/matthijs/.arduino15/packages/arduino/tools/CMSIS-Atmel/1.2.0/CMSIS/Device/ATMEL/",
   "-I/home/matthijs/.arduino15/packages/arduino/hardware/samd/1.8.8/cores/arduino",
   "-I/home/matthijs/.arduino15/packages/arduino/hardware/samd/1.8.8/variants/mkrzero",
   "/tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/sketch/clangd-arduino.ino.cpp",
   "-o",
   "/tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/sketch/clangd-arduino.ino.cpp.o"
  ],
  "file": "/tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/sketch/clangd-arduino.ino.cpp"
 },
(... info about other files omitted)

clangd output is:

I[14:30:01.797] clangd version 10.0.0-4ubuntu1 
I[14:30:01.797] PID: 111697
I[14:30:01.797] Working directory: /home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino
I[14:30:01.797] argv[0]: /usr/bin/clangd-10
I[14:30:01.797] argv[1]: -cross-file-rename
I[14:30:01.797] argv[2]: --query-driver=/usr/bin/*-gcc,/usr/bin/*-g++,/usr/bin/gcc,/usr/bin/g++,/home/matthijs/docs/Electronics/Arduino/Arduino/build/linux/work/hardware/tools/avr/bin/avr-gcc,/home/matthijs/docs/Electronics/Arduino/Arduino/build/linux/work/hardware/tools/avr/bin/avr-g++,/home/matthijs/.arduino15/packages/*/tools/**/*-gcc,/home/matthijs/.arduino15/packages/*/tools/**/*-g++
I[14:30:01.797] argv[3]: -header-insertion-decorators=0
I[14:30:01.797] argv[4]: -limit-results=500
I[14:30:01.797] argv[5]: -log=verbose
I[14:30:01.797] Starting LSP over stdin/stdout
V[14:30:01.797] <<< {"id":1,"jsonrpc":"2.0","method":"initialize","params":{"capabilities":{"textDocument":{"codeAction":{"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}}},"completion":{"completionItem":{"documentationFormat":["plaintext","markdown"]},"completionItemKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]}},"hover":{"contentFormat":["plaintext","markdown"]},"signatureHelp":{"signatureInformation":{"documentationFormat":["plaintext","markdown"],"parameterInformation":{"labelOffsetSupport":false}}},"synchronization":{"didSave":true}},"workspace":{"applyEdit":true,"didChangeWatchedFiles":{"dynamicRegistration":true},"documentChanges":true,"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}}}},"initializationOptions":{},"processId":111654,"rootPath":"/home/matthijs/docs/Electronics/Arduino/Sketches","rootUri":"file:///home/matthijs/docs/Electronics/Arduino/Sketches"}}

I[14:30:01.797] <-- initialize(1)
I[14:30:01.798] --> reply:initialize(1) 0 ms
V[14:30:01.798] >>> {"id":1,"jsonrpc":"2.0","result":{"capabilities":{"codeActionProvider":{"codeActionKinds":["quickfix","refactor","info"]},"completionProvider":{"resolveProvider":false,"triggerCharacters":[".",">",":"]},"declarationProvider":true,"definitionProvider":true,"documentFormattingProvider":true,"documentHighlightProvider":true,"documentLinkProvider":{"resolveProvider":false},"documentOnTypeFormattingProvider":{"firstTriggerCharacter":"\n","moreTriggerCharacter":[]},"documentRangeFormattingProvider":true,"documentSymbolProvider":true,"executeCommandProvider":{"commands":["clangd.applyFix","clangd.applyTweak"]},"hoverProvider":true,"referencesProvider":true,"renameProvider":true,"selectionRangeProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"textDocumentSync":2,"typeHierarchyProvider":true,"workspaceSymbolProvider":true}}}

V[14:30:01.801] <<< {"jsonrpc":"2.0","method":"initialized","params":{}}

I[14:30:01.801] <-- initialized
I[14:30:01.801] unhandled notification initialized
V[14:30:01.801] <<< {"jsonrpc":"2.0","method":"workspace/didChangeConfiguration","params":{"settings":{}}}

I[14:30:01.801] <-- workspace/didChangeConfiguration
V[14:30:01.805] <<< {"jsonrpc":"2.0","method":"workspace/didChangeConfiguration","params":{"settings":{"compilationDatabaseChanges":{"/home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino/clangd-arduino.ino":{"compilationCommand":["clang-tool","-Wall","-Wextra","-Werror","-Wno-c99-designator","-fno-exceptions","-DARDUINO=10600","--target=arm-none-eabi","-mcpu=cortex-m0plus","-march=armv6-m","-mthumb","-mabi=aapcs","-mfloat-abi=soft","-fsingle-precision-constant","-x","c++","-std=gnu++11","-DARDUINO_MJS2020_PROTO2","-DARDUINO_ARCH_STM32L0","-DSTM32L072xx","-DUSB_TYPE=USB_TYPE_CDC","-isystem","/usr/arm-linux-gnueabi/include/c++/9","-isystem","/usr/arm-linux-gnueabi/include/c++/9/arm-linux-gnueabi/hf","-isystem","/usr/arm-linux-gnueabi/include","-I","","-I","hardware/meetjestad-git/stm32l0/cores/arduino/","-I","hardware/meetjestad-git/stm32l0/system/STM32L0xx/Include","-I","hardware/meetjestad-git/stm32l0/system/CMSIS/Device/ST/STM32L0xx/Include","-I","hardware/meetjestad-git/stm32l0/system/CMSIS/Include","-I","hardware/meetjestad-git/stm32l0/variants/MJS2020-PROTO2","-I","hardware/meetjestad-git/stm32l0/libraries/SPI/src","-I","hardware/meetjestad-git/stm32l0/libraries/Wire/src","-I","hardware/meetjestad-git/stm32l0/libraries/STM32L0/src","/home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino/clangd-arduino.ino"],"workingDirectory":"/home/matthijs/docs/Electronics/Arduino/Sketches"}}}}}

I[14:30:01.805] <-- workspace/didChangeConfiguration
I[14:30:01.808] Enqueueing 0 commands for indexing
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include-fixed
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include
I[14:30:01.821] System include extractor: succesfully executed /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/arm-none-eabi-g++, got includes: "/home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include-fixed, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include"
I[14:30:01.821] Enqueueing 1 commands for indexing

Note that if I drop the arduino paths from the query-driver option, I get the whitelist error as expected:

V[14:28:34.169] System include extraction: not whitelisted driver /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/arm-none-eabi-g++

May 13 '21 12:05 matthijskooijman

Tell me, can you confirm that on your Linux/OS X system the C++ include paths are extracted properly to the compile_commands.json?

On Linux, using arduino-cli and clangd running inside Vim with the YouCompleteMe plugin, it indeed works. I tried using your clangd-arduino.ino sketch:

$ arduino-cli compile --only-compilation-database --fqbn arduino:samd:mkrzero
$ ln -s /tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/compile_commands.json 
$ vim clangd-arduino.ino

The compile_commands.json for the .ino file looks like this:

[
 {
  "directory": "/home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino",
  "arguments": [
   "/home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/arm-none-eabi-g++",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRZERO",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x804f",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKRZero\"",
   "-I/home/matthijs/.arduino15/packages/arduino/tools/CMSIS/4.5.0/CMSIS/Include/",
   "-I/home/matthijs/.arduino15/packages/arduino/tools/CMSIS-Atmel/1.2.0/CMSIS/Device/ATMEL/",
   "-I/home/matthijs/.arduino15/packages/arduino/hardware/samd/1.8.8/cores/arduino",
   "-I/home/matthijs/.arduino15/packages/arduino/hardware/samd/1.8.8/variants/mkrzero",
   "/tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/sketch/clangd-arduino.ino.cpp",
   "-o",
   "/tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/sketch/clangd-arduino.ino.cpp.o"
  ],
  "file": "/tmp/arduino-sketch-361496944F32655D8A5911BF4E79224F/sketch/clangd-arduino.ino.cpp"
 },
(... info about other files omitted)

clangd output is:

I[14:30:01.797] clangd version 10.0.0-4ubuntu1 
I[14:30:01.797] PID: 111697
I[14:30:01.797] Working directory: /home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino
I[14:30:01.797] argv[0]: /usr/bin/clangd-10
I[14:30:01.797] argv[1]: -cross-file-rename
I[14:30:01.797] argv[2]: --query-driver=/usr/bin/*-gcc,/usr/bin/*-g++,/usr/bin/gcc,/usr/bin/g++,/home/matthijs/docs/Electronics/Arduino/Arduino/build/linux/work/hardware/tools/avr/bin/avr-gcc,/home/matthijs/docs/Electronics/Arduino/Arduino/build/linux/work/hardware/tools/avr/bin/avr-g++,/home/matthijs/.arduino15/packages/*/tools/**/*-gcc,/home/matthijs/.arduino15/packages/*/tools/**/*-g++
I[14:30:01.797] argv[3]: -header-insertion-decorators=0
I[14:30:01.797] argv[4]: -limit-results=500
I[14:30:01.797] argv[5]: -log=verbose
I[14:30:01.797] Starting LSP over stdin/stdout
V[14:30:01.797] <<< {"id":1,"jsonrpc":"2.0","method":"initialize","params":{"capabilities":{"textDocument":{"codeAction":{"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}}},"completion":{"completionItem":{"documentationFormat":["plaintext","markdown"]},"completionItemKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]}},"hover":{"contentFormat":["plaintext","markdown"]},"signatureHelp":{"signatureInformation":{"documentationFormat":["plaintext","markdown"],"parameterInformation":{"labelOffsetSupport":false}}},"synchronization":{"didSave":true}},"workspace":{"applyEdit":true,"didChangeWatchedFiles":{"dynamicRegistration":true},"documentChanges":true,"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}}}},"initializationOptions":{},"processId":111654,"rootPath":"/home/matthijs/docs/Electronics/Arduino/Sketches","rootUri":"file:///home/matthijs/docs/Electronics/Arduino/Sketches"}}

I[14:30:01.797] <-- initialize(1)
I[14:30:01.798] --> reply:initialize(1) 0 ms
V[14:30:01.798] >>> {"id":1,"jsonrpc":"2.0","result":{"capabilities":{"codeActionProvider":{"codeActionKinds":["quickfix","refactor","info"]},"completionProvider":{"resolveProvider":false,"triggerCharacters":[".",">",":"]},"declarationProvider":true,"definitionProvider":true,"documentFormattingProvider":true,"documentHighlightProvider":true,"documentLinkProvider":{"resolveProvider":false},"documentOnTypeFormattingProvider":{"firstTriggerCharacter":"\n","moreTriggerCharacter":[]},"documentRangeFormattingProvider":true,"documentSymbolProvider":true,"executeCommandProvider":{"commands":["clangd.applyFix","clangd.applyTweak"]},"hoverProvider":true,"referencesProvider":true,"renameProvider":true,"selectionRangeProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"textDocumentSync":2,"typeHierarchyProvider":true,"workspaceSymbolProvider":true}}}

V[14:30:01.801] <<< {"jsonrpc":"2.0","method":"initialized","params":{}}

I[14:30:01.801] <-- initialized
I[14:30:01.801] unhandled notification initialized
V[14:30:01.801] <<< {"jsonrpc":"2.0","method":"workspace/didChangeConfiguration","params":{"settings":{}}}

I[14:30:01.801] <-- workspace/didChangeConfiguration
V[14:30:01.805] <<< {"jsonrpc":"2.0","method":"workspace/didChangeConfiguration","params":{"settings":{"compilationDatabaseChanges":{"/home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino/clangd-arduino.ino":{"compilationCommand":["clang-tool","-Wall","-Wextra","-Werror","-Wno-c99-designator","-fno-exceptions","-DARDUINO=10600","--target=arm-none-eabi","-mcpu=cortex-m0plus","-march=armv6-m","-mthumb","-mabi=aapcs","-mfloat-abi=soft","-fsingle-precision-constant","-x","c++","-std=gnu++11","-DARDUINO_MJS2020_PROTO2","-DARDUINO_ARCH_STM32L0","-DSTM32L072xx","-DUSB_TYPE=USB_TYPE_CDC","-isystem","/usr/arm-linux-gnueabi/include/c++/9","-isystem","/usr/arm-linux-gnueabi/include/c++/9/arm-linux-gnueabi/hf","-isystem","/usr/arm-linux-gnueabi/include","-I","","-I","hardware/meetjestad-git/stm32l0/cores/arduino/","-I","hardware/meetjestad-git/stm32l0/system/STM32L0xx/Include","-I","hardware/meetjestad-git/stm32l0/system/CMSIS/Device/ST/STM32L0xx/Include","-I","hardware/meetjestad-git/stm32l0/system/CMSIS/Include","-I","hardware/meetjestad-git/stm32l0/variants/MJS2020-PROTO2","-I","hardware/meetjestad-git/stm32l0/libraries/SPI/src","-I","hardware/meetjestad-git/stm32l0/libraries/Wire/src","-I","hardware/meetjestad-git/stm32l0/libraries/STM32L0/src","/home/matthijs/docs/Electronics/Arduino/Sketches/Tests/clangd-arduino/clangd-arduino.ino"],"workingDirectory":"/home/matthijs/docs/Electronics/Arduino/Sketches"}}}}}

I[14:30:01.805] <-- workspace/didChangeConfiguration
I[14:30:01.808] Enqueueing 0 commands for indexing
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include-fixed
V[14:30:01.821] System include extraction: adding  /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include
I[14:30:01.821] System include extractor: succesfully executed /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/arm-none-eabi-g++, got includes: "/home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/include-fixed, /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include"
I[14:30:01.821] Enqueueing 1 commands for indexing

Note that if I drop the arduino paths from the query-driver option, I get the whitelist error as expected:

V[14:28:34.169] System include extraction: not whitelisted driver /home/matthijs/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/arm-none-eabi-g++

Thank you. What happens when you run:

falve@Fran-DESKTOP MSYS /c/source/repos/clangd-arduino
$ clang-tidy clangd-arduino.ino -p="build/"
1 error generated.
Error while processing C:\source\repos\clangd-arduino\clangd-arduino.ino.
C:\source\repos\clangd-arduino\clangd-arduino.ino:1:10: error: 'type_traits' file not found [clang-diagnostic-error]
#include <type_traits>
         ^
Found compiler error(s).

May 13 '21 12:05 Falven

I have made some progress, @matthijskooijman

If I fix all of the non-windows paths in the generated compile_commands.json, AND append the .exe extension to BOTH the compile_commands.json and --query-driver command, it works!

[
 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++.exe",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp.o"
  ],
  "file": "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp"
 },
 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++.exe",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010\\variant.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\core\\variant.cpp.o"
  ],
  "file": "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010\\variant.cpp"
 }
]

I[12:11:46.274] clangd version 12.0.0 (https://github.com/kirillbobyrev/llvm-project 8a844a3a321ea56da21af4d48433ad493b1bcfae)
I[12:11:46.275] PID: 21548
I[12:11:46.275] Working directory: c:\source\repos\clangd-arduino
I[12:11:46.275] argv[0]: c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe
I[12:11:46.275] argv[1]: --log=verbose
I[12:11:46.275] argv[2]: --query-driver=C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gcc.exe,C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++.exe
V[12:11:46.285] User config file is C:\Users\falve\AppData\Local\clangd\config.yaml
I[12:11:46.286] Starting LSP over stdin/stdout

...

I[12:11:46.685] Loaded compilation database from c:\source\repos\clangd-arduino\build/compile_commands.json
V[12:11:46.685] Broadcasting compilation database from c:\source\repos\clangd-arduino
I[12:11:46.689] Loaded compilation database from c:\source\repos\clangd-arduino\build\compile_commands.json
V[12:11:47.114] System include extraction: target extracted: "arm-none-eabi"
V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include

I[12:11:47.114] System includes extractor: successfully executed C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++.exe
	got includes: "c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include"
	got target: "arm-none-eabi"

@matthijskooijman seems we have an actionable fix. :)

From @kadircet

aha! i was just trying to point out the slash mismatch, but you beat me to it. we'll try to get this handled more graciously on clangd side as well (i.e. look for a file with os executable suffix if we can't find the driver at first try), but it won't be available before clangd-13 (which is due in 6 months).

So for a faster solution you might want to contact the tool generating the compile_commands.json to see why they don't keep the executable suffix. This is the first time we're facing this issue, so I guess most other generators in windows must be preserving the suffix.

May 13 '21 16:05 Falven

If I fix all of the non-windows paths in the generated compile_commands.json, AND append the .exe extension to BOTH the compile_commands.json and --query-driver command, it works!

Ah, cool, so the .exe is apparently essential.

That sounds like there's two changes to make:

Convert the compiler path (first element of arguments) to all backslashes and add .exe
Convert all (include) paths in the remaining elements of arguments to all backslashes

Of these, 1. is probably easy, but 2. is a bit harder, because arduino-cli has no idea which arguments are pathnames and which use literal slashes. For clangd it is probably ok to just convert all slashes, or maybe specialcase this to only options starting with -I or so. I'm pretty sure we should not be doing this for the actual commands being ran (too much chance for making the wrong changes), so this would mean the compile_commands.json no longer matches the exact commands, which I'm not very enthusiastic about.

IIUC you're saying that just fixing 1. is not enough and fixing 2. is also needed? I suspect that 1. is enough to fix the whitelist issue and have clangd call the driver, but maybe 2. is also needed to make it understand the additional include paths?

And I was indeed going to suggest fixing this at the clangd side, it would be good if it is more tolerant about omitting/adding .exe. And given that gcc also understands the mixed-slashes in its arguments, it would be nice if clangd would do the same. I'll provide a response to that issue as well in a minute.

May 13 '21 16:05 matthijskooijman

That sounds like there's two changes to make:

Convert the compiler path (first element of arguments) to all backslashes and add .exe

Convert all (include) paths in the remaining elements of arguments to all backslashes

Of these, 1. is probably easy, but 2. is a bit harder, because arduino-cli has no idea which arguments are pathnames and which use literal slashes. For clangd it is probably ok to just convert all slashes, or maybe specialcase this to only options starting with -I or so. I'm pretty sure we should not be doing this for the actual commands being ran (too much chance for making the wrong changes), so this would mean the compile_commands.json no longer matches the exact commands, which I'm not very enthusiastic about.

IIUC you're saying that just fixing 1. is not enough and fixing 2. is also needed? I suspect that 1. is enough to fix the whitelist issue and have clangd call the driver, but maybe 2. is also needed to make it understand the additional include paths?

And I was indeed going to suggest fixing this at the clangd side, it would be good if it is more tolerant about omitting/adding .exe. And given that gcc also understands the mixed-slashes in its arguments, it would be nice if clangd would do the same. I'll provide a response to that issue as well in a minute.

@matthijskooijman I'm sorry. I should have been more specific. I have just verified, you only need to fix:

Convert the compiler path (first element of arguments) to all backslashes and add .exe

That coupled with the windows-style path and extension in the --query-driver argument did the trick.

This will do both, whitelist the driver, and have clangd call it allowing it to extract the missing C++ include paths.

Note: This fix will work for my particular use case with clangd. I am unsure if the other mixed paths in compile_commands.json could cause issues with other intellisense engines/scenarios.

May 13 '21 16:05 Falven

Cool. Then one more question: If you just add the .exe to compile_commands.json, leaving the mixed slashes and then use the same mix of slashes in --query-driver, does it then also work? Or does clangd really requires only backslashes in the first element of "arguments"?

May 13 '21 16:05 matthijskooijman

Cool. Then one more question: If you just add the .exe to compile_commands.json, leaving the mixed slashes and then use the same mix of slashes in --query-driver, does it then also work? Or does clangd really requires only backslashes in the first element of "arguments"?

That worked as well.

I[13:02:31.760] clangd version 12.0.0 (https://github.com/kirillbobyrev/llvm-project 8a844a3a321ea56da21af4d48433ad493b1bcfae)
I[13:02:31.761] PID: 20500
I[13:02:31.761] Working directory: c:\source\repos\clangd-arduino
I[13:02:31.761] argv[0]: c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe
I[13:02:31.761] argv[1]: --log=verbose
I[13:02:31.761] argv[2]: --query-driver=C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-gcc.exe,C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++.exe
V[13:02:31.770] User config file is C:\Users\falve\AppData\Local\clangd\config.yaml
I[13:02:31.770] Starting LSP over stdin/stdout

...

V[13:02:31.957] System include extraction: target extracted: "arm-none-eabi"
V[13:02:31.957] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1

V[13:02:31.957] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi

V[13:02:31.957] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward

V[13:02:31.957] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include

V[13:02:31.957] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed

V[13:02:31.957] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include

I[13:02:31.957] System includes extractor: successfully executed C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++.exe
	got includes: "c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include"
	got target: "arm-none-eabi"
I[13:02:31.957] ASTWorker building file c:\source\repos\clangd-arduino\clangd-arduino.ino version 1 with command inferred from c:\source\repos\clangd-arduino\build\sketch\clangd-arduino.ino.cpp
[c:\Users\falve\.vscode\extensions\vsciot-vscode.vscode-arduino-0.4.2\out\src]
C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++.exe --target=arm-none-eabi --driver-mode=g++ -mcpu=cortex-m0plus -mthumb -c -g -Os -w -ffunction-sections -fdata-sections -fno-threadsafe-statics -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -DF_CPU=48000000L -DARDUINO=10607 -DARDUINO_SAMD_MKRWIFI1010 -DARDUINO_ARCH_SAMD -DUSE_ARDUINO_MKR_PIN_LAYOUT -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x8054 -DUSBCON -DUSB_MANUFACTURER="Arduino LLC" -DUSB_PRODUCT="Arduino MKR WiFi 1010" -DUSE_BQ24195L_PMIC -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0/CMSIS/Include/ -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0/CMSIS/Device/ATMEL/ -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated-avr-comp -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino -IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010 -x c++-header -std=gnu++11 c:\source\repos\clangd-arduino\clangd-arduino.ino -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1 -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include -fsyntax-only -resource-dir=c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0
V[13:02:31.960] Driver produced command: cc1 -cc1 -triple thumbv6m-none-unknown-eabi -fsyntax-only -disable-free -disable-llvm-verifier -discard-value-names -main-file-name clangd-arduino.ino -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -nostdsysteminc -target-cpu cortex-m0plus -target-feature +soft-float-abi -target-feature -crc -target-feature -crypto -target-feature -sha2 -target-feature -aes -target-feature -dotprod -target-feature -dsp -target-feature -mve -target-feature -mve.fp -target-feature -fullfp16 -target-feature -ras -target-feature -bf16 -target-feature -sb -target-feature -i8mm -target-feature -lob -target-feature -cdecp0 -target-feature -cdecp1 -target-feature -cdecp2 -target-feature -cdecp3 -target-feature -cdecp4 -target-feature -cdecp5 -target-feature -cdecp6 -target-feature -cdecp7 -target-feature -hwdiv-arm -target-feature -hwdiv -target-feature -fp16fml -target-feature +strict-align -target-abi aapcs -mfloat-abi soft -fallow-half-arguments-and-returns -fno-split-dwarf-inlining -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -ffunction-sections -fdata-sections -resource-dir c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0 -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1 -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed -isystem c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include -D F_CPU=48000000L -D ARDUINO=10607 -D ARDUINO_SAMD_MKRWIFI1010 -D ARDUINO_ARCH_SAMD -D USE_ARDUINO_MKR_PIN_LAYOUT -D __SAMD21G18A__ -D USB_VID=0x2341 -D USB_PID=0x8054 -D USBCON -D USB_MANUFACTURER="Arduino LLC" -D USB_PRODUCT="Arduino MKR WiFi 1010" -D USE_BQ24195L_PMIC -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0/CMSIS/Include/ -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0/CMSIS/Device/ATMEL/ -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated-avr-comp -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino -I C:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010 -internal-isystem C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/clang-runtimes\arm-none-eabi\include\c++\v1 -internal-isystem c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\lib\clang\12.0.0\include -internal-isystem C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/clang-runtimes\arm-none-eabi\include -Os -w -std=gnu++11 -fdeprecated-macro -fdebug-compilation-dir c:\Users\falve\.vscode\extensions\vsciot-vscode.vscode-arduino-0.4.2\out\src -ferror-limit 19 -fno-rtti -fno-signed-char -fgnuc-version=4.2.1 -fno-threadsafe-statics -vectorize-loops -vectorize-slp -faddrsig -x c++-header c:\source\repos\clangd-arduino\clangd-arduino.ino

So it seems the culprit is really only the missing exe file extensions and mismatched paths between the compile_commands.json and --query-driver.

May 13 '21 17:05 Falven

@matthijskooijman I know this is different, yet related; I am willing to open a separate issue if you would like, but are you planning on making the generated compile_commands.json -I includes into -isystem? I'm having problems telling clangd to ignore Arduino "system" includes #include <Arduino.h> warnings and was thinking this may be part of the problem. Currently researching.

Edit: See this stackoverflow about ignoring warnings from other include files suggesting to use -isystem.

May 17 '21 22:05 Falven

@matthijskooijman There is another problem that fixing these system includes presents. Edit: The answer is at the very bottom (If you want to skip my research).

When compiling a simple sketch, such as the following:

#include <Arduino.h>

#include <vector>

void setup() {}

void loop() {}

Before system includes were fixed on Windows, at least for generated compile_commands.json, compile commands looked like the following:

C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++.exe -mcpu=cortex-m0plus -mthumb -c -g -Os -w -std=gnu++11 -ffunction-sections -fdata-sections -fno-threadsafe-statics -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -MMD -DF_CPU=48000000L -DARDUINO=10607 -DARDUINO_SAMD_MKRWIFI1010 -DARDUINO_ARCH_SAMD -DUSE_ARDUINO_MKR_PIN_LAYOUT -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x8054 -DUSBCON -DUSB_MANUFACTURER="Arduino LLC" -DUSB_PRODUCT="Arduino MKR WiFi 1010" -DUSE_BQ24195L_PMIC "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0/CMSIS/Include/" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0/CMSIS/Device/ATMEL/" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated-avr-comp" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010" -x c++ "C:\source\repos\clangd-arduino\clangd-arduino.ino" -o "C:\source\repos\clangd-arduino\clangd-arduino.ino.o"

These, of course, succeed in creating object files.

However, now that the driver properly extracts system includes, compile commands would look like the following:

 C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++.exe -mcpu=cortex-m0plus -mthumb -c -g -Os -w -std=gnu++11 -ffunction-sections -fdata-sections -fno-threadsafe-statics -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -MMD -DF_CPU=48000000L -DARDUINO=10607 -DARDUINO_SAMD_MKRWIFI1010 -DARDUINO_ARCH_SAMD -DUSE_ARDUINO_MKR_PIN_LAYOUT -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x8054 -DUSBCON -DUSB_MANUFACTURER="Arduino LLC" -DUSB_PRODUCT="Arduino MKR WiFi 1010" -DUSE_BQ24195L_PMIC "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0/CMSIS/Include/" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0/CMSIS/Device/ATMEL/" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated-avr-comp" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010" "-isystemc:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1" "-isystemc:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi" "-isystemc:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward" "-isystemc:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include" "-isystemc:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed" "-isystemc:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include" -x c++ "C:\source\repos\clangd-arduino\clangd-arduino.ino" -o "C:\source\repos\clangd-arduino\clangd-arduino.ino.o"

This causes issues when including any header-only or template libraries, including the STL. In our example including <vector>, you will get:

In file included from c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\arm-none-eabi\include\c++\7.2.1\bits\stl_algobase.h:63:0,
                 from c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\arm-none-eabi\include\c++\7.2.1\vector:60,
                 from C:\source\repos\clangd-arduino\clangd-arduino.ino:3:
c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\arm-none-eabi\include\c++\7.2.1\ext\numeric_traits.h:54:3: error: template with C linkage
   template<typename _Value>
.. hundreds of other "error: template with C linkage" errors ...

It almost seems like it is being compiled as C code, even though -x c++ is specified...

NOTE: I understand the issues behind heap fragmentation and SRAM, <vector> here is just an easy example of an STL include. It is for example purposes. Saying you shouldn't use <vector> does not address the question or propose a viable solution.

Intellisense engines will use these commands to provide compiler output, so while people shouldn't be needing to execute these commands manually to create the object files, etc, the intellisense engines will add -fsyntax-only to the end of the command to perform syntax validation.

Any ideas what is going on here? Are the system includes somehow being included twice?

Search results seem to suggest it is something with the include order and #include <Arduino.h>, however I have explicitly included it at the top as you can see.

Edit: Figured out the issue! For some reason, the arm-none-eabi-g++ compiler implicitly applies C linkage to includes passed with the -isystem flag. So switching those to -I fixes the issue:

C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4/bin/arm-none-eabi-g++.exe -mcpu=cortex-m0plus -mthumb -c -g -Os -w -std=gnu++11 -ffunction-sections -fdata-sections -fno-threadsafe-statics -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -MMD -DF_CPU=48000000L -DARDUINO=10607 -DARDUINO_SAMD_MKRWIFI1010 -DARDUINO_ARCH_SAMD -DUSE_ARDUINO_MKR_PIN_LAYOUT -D__SAMD21G18A__ -DUSB_VID=0x2341 -DUSB_PID=0x8054 -DUSBCON -DUSB_MANUFACTURER="Arduino LLC" -DUSB_PRODUCT="Arduino MKR WiFi 1010" -DUSE_BQ24195L_PMIC "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS\4.5.0/CMSIS/Include/" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\CMSIS-Atmel\1.2.0/CMSIS/Device/ATMEL/" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino/api/deprecated-avr-comp" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\cores\arduino" "-IC:\Users\falve\AppData\Local\Arduino15\packages\arduino\hardware\samd\1.8.11\variants\mkrwifi1010" "-Ic:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1" "-Ic:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi" "-Ic:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward" "-Ic:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include" "-Ic:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed" "-Ic:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include" -xc++ "C:\source\repos\clangd-arduino\clangd-arduino.ino" -o "C:\source\repos\clangd-arduino\clangd-arduino.ino.o"

Apparently it is a bug with this version of the compiler from 2017. Will we ever see a fix/update? A proposed solution that does not involve updating the compiler would have me wrap all of my c++ includes in extern "C++" {} blocks which is a pretty ridiculous solution imo.

See here for an issue I raised to try to update the toolchain.

May 18 '21 20:05 Falven

I'm having the same issue in macOS with arduino-cli 0.33.1. I'm using Emacs (with the eglot LSP client) with clangd. I get this error:

clang [pp_file_not_found]: In included file: 'avr/pgmspace.h' file not found

/Users/gsgx/Library/Arduino15/packages/arduino/hardware/avr/1.8.6/cores/arduino/Arduino.h:28:10:

I fixed it by manually adding this to the compile_commands.json file:

"-isystem/Users/gsgx/Library/Arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/avr/include",

This is a long thread that I'm having some trouble following, so is the accepted solution to use the query-driver flag? Or would it make sense to just add the system library path to the compile arguments just for the convenience of generating a compile_commands.json that works out of the box?

Jul 05 '23 23:07 gsingh93

arduino-cli arduino-cli copied to clipboard

Generated compilation database compiler argument missing extension breaks compilation on Windows.

Bug Report

Current behavior

Expected behavior

Environment

Additional context

arduino-cli
arduino-cli copied to clipboard