Include insertion does not work if path in `-I` flag in compile_commands.json has different capitalization

[LuncyBloont]

IWYU cannot automatically add header file inclusions when relative paths to source files are used in the compile_commands.json file. Moreover, the completion of LSP has also become abnormal, and the macro definitions in some header files seem to be unable to be parsed normally.

in compile_commands.json

...
"command": "absolute/path/to/cl.exe /flags -Irelative/path/to/includedir relative/path/to/source.cpp",
"file": "relative/path/to/source.cpp",
...

Logs

Failed to generate include insertion edits for adding header (FileURI='file:///H:/absolute/path/to/test.hpp', IncludeHeader='file:///H:/absolute/path/to/test.hpp') into h:\relative\path\to\source.cpp: Header not on include path
...
Failed to generate include insertion edits for adding header (FileURI='file:///H:/absolute/path/to/test.hpp', IncludeHeader='file:///H:/absolute/path/to/test.hpp') into h:\relative\path\to\source.cpp: Header not on include path

System information

Output of clangd --version:

clangd version 17.0.6
Features: windows
Platform: x86_64-pc-windows-msvc

Editor/LSP plugin:

VSCode + llvm-vs-code-extensions.vscode-clangd

Operating system:

windows 11

[HighCommander4]

Is "directory" an absolute path?

[LuncyBloont]

Is "directory" an absolute path?

Yes, it is an absolute path.

[HighCommander4]

Ok, I believe that's valid according to the compilation database spec (the relative path in "file" is interpreted as being relative to "directory").

There may well be clangd bugs here, and if you're able to share a complete example / reproduction steps for further investigation, that's worth doing.

That said, I'll mention that the vast majority of compilation databases I've seen so far (both in my own usage and in other clangd issues) have used absolute paths in "file", and I suspect that's a much better tested configuration than relative paths in "file". If you're able to switch to using absolute paths, I would recommend that as a workaround.

[LuncyBloont]

Ok, I believe that's valid according to the compilation database spec (the relative path in "file" is interpreted as being relative to "directory").

There may well be clangd bugs here, and if you're able to share a complete example / reproduction steps for further investigation, that's worth doing.

That said, I'll mention that the vast majority of compilation databases I've seen so far (both in my own usage and in other clangd issues) have used absolute paths in "file", and I suspect that's a much better tested configuration than relative paths in "file". If you're able to switch to using absolute paths, I would recommend that as a workaround.

Sorry, I spent some time testing and found that this problem does not seem to be caused by the path (absolute or not). 😳

Through repeated editing and testing, I finally got these two database files (both using absolute path). With the first one, clangd's automatic inclusion can work normally, but with the second one, it does not:

good compile_commands.json:

[
  {
    "directory": "H:/test/build",
    "command": "C:\\PROGRA~1\\LLVM\\bin\\CLANG_~1.EXE -IH:/test/include -IC:/VulkanSDK/1.3.268.0/Include -O0 -g -Xclang -gcodeview -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -o CMakeFiles\\test.dir\\src\\main.cpp.obj -c H:\\test\\src\\main.cpp",
    "file": "H:\\test\\src\\main.cpp"
  }
]

bad one:

[
  {
    "directory": "H:/test/build",
    "command": "C:\\PROGRA~1\\LLVM\\bin\\CLANG_~1.EXE -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -IH:\\test\\include -fexceptions -fcxx-exceptions -IC:\\VulkanSDK\\1.3.268.0\\include -DNDEBUG -o build\\.objs\\test\\windows\\x64\\release\\src\\main.cpp.obj H:\\test\\src\\main.cpp",
    "file": "H:\\test\\src\\main.cpp"
  }
]

There are many differences between the commands in the two files. I haven't located the specific parameter that caused the problem. They come from different build tools.

[HighCommander4]

It's hard to guess what the relevant difference might be. (Direction of slashes in the -I flags perhaps? That's a complete guess though.)

If you can share clangd logs for the bad case, along with a more detailed description of what goes wrong, that might provide some clues.

[LuncyBloont]

It's hard to guess what the relevant difference might be. (Direction of slashes in the -I flags perhaps? That's a complete guess though.)

If you can share clangd logs for the bad case, along with a more detailed description of what goes wrong, that might provide some clues.

Maybe the direction of the slash in the "-I" flag is the reason? log:

I[23:05:39.393] <-- textDocument/didChange
I[23:05:39.393] --> textDocument/clangd.fileStatus
I[23:05:39.456] ASTWorker building file h:\test\src\main.cpp version 20 with command 
[H:/test/build]
"C:\\Program Files\\LLVM\\bin\\CLANG_~1.EXE" -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 "-IH:\\test\\include" -fexceptions -fcxx-exceptions "-IC:\\VulkanSDK\\1.3.268.0\\include" -DNDEBUG -o "build\\.objs\\test\\windows\\x64\\release\\src\\main.cpp.obj" "-resource-dir=C:\\Program Files\\LLVM\\lib\\clang\\17" -- "h:\\test\\src\\main.cpp"
I[23:05:39.461] --> textDocument/clangd.fileStatus
I[23:05:39.481] --> textDocument/publishDiagnostics
I[23:05:39.481] --> textDocument/inactiveRegions
I[23:05:39.481] --> textDocument/clangd.fileStatus
I[23:05:39.695] <-- textDocument/foldingRange(65)
I[23:05:39.696] --> reply:textDocument/foldingRange(65) 0 ms
I[23:05:39.840] <-- textDocument/didChange
I[23:05:39.840] --> textDocument/clangd.fileStatus
I[23:05:39.851] <-- textDocument/completion(66)
I[23:05:39.851] <-- textDocument/semanticTokens/full/delta(67)
I[23:05:39.853] ASTWorker building file h:\test\src\main.cpp version 21 with command 
[H:/test/build]
"C:\\Program Files\\LLVM\\bin\\CLANG_~1.EXE" -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 "-IH:\\test\\include" -fexceptions -fcxx-exceptions "-IC:\\VulkanSDK\\1.3.268.0\\include" -DNDEBUG -o "build\\.objs\\test\\windows\\x64\\release\\src\\main.cpp.obj" "-resource-dir=C:\\Program Files\\LLVM\\lib\\clang\\17" -- "h:\\test\\src\\main.cpp"
I[23:05:39.856] --> textDocument/clangd.fileStatus
I[23:05:39.879] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:39.879] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
... x2000+
I[23:05:40.467] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.467] Code complete: sema context Statement, query scopes [] (AnyScope=true), expected type <none>
I[23:05:40.468] Code complete: 0 results from Sema, 65 from Index, 0 matched, 0 from identifiers, 65 returned.
I[23:05:40.469] --> reply:textDocument/completion(70) 27 ms
I[23:05:40.501] ASTWorker building file h:\test\src\main.cpp version 24 with command 
[H:/test/build]
"C:\\Program Files\\LLVM\\bin\\CLANG_~1.EXE" -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 "-IH:\\test\\include" -fexceptions -fcxx-exceptions "-IC:\\VulkanSDK\\1.3.268.0\\include" -DNDEBUG -o "build\\.objs\\test\\windows\\x64\\release\\src\\main.cpp.obj" "-resource-dir=C:\\Program Files\\LLVM\\lib\\clang\\17" -- "h:\\test\\src\\main.cpp"
I[23:05:40.506] --> textDocument/clangd.fileStatus
I[23:05:40.519] <-- textDocument/semanticTokens/full/delta(71)
I[23:05:40.528] --> textDocument/publishDiagnostics
I[23:05:40.528] --> textDocument/inactiveRegions
I[23:05:40.528] --> reply:textDocument/semanticTokens/full/delta(71) 8 ms
I[23:05:40.528] --> textDocument/clangd.fileStatus
I[23:05:40.564] <-- textDocument/didChange
I[23:05:40.564] --> textDocument/clangd.fileStatus
I[23:05:40.565] <-- textDocument/completion(72)
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Failed to generate include insertion edits for adding header (FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h', IncludeHeader='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h') into h:\test\src\main.cpp: Header not on include path
I[23:05:40.589] Code complete: sema context Statement, query scopes [] (AnyScope=true), expected type <none>
I[23:05:40.590] Code complete: 0 results from Sema, 14 from Index, 0 matched, 0 from identifiers, 14 returned.
I[23:05:40.590] --> reply:textDocument/completion(72) 25 ms
I[23:05:40.616] ASTWorker building file h:\test\src\main.cpp version 25 with command 
[H:/test/build]
"C:\\Program Files\\LLVM\\bin\\CLANG_~1.EXE" -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 "-IH:\\test\\include" -fexceptions -fcxx-exceptions "-IC:\\VulkanSDK\\1.3.268.0\\include" -DNDEBUG -o "build\\.objs\\test\\windows\\x64\\release\\src\\main.cpp.obj" "-resource-dir=C:\\Program Files\\LLVM\\lib\\clang\\17" -- "h:\\test\\src\\main.cpp"
I[23:05:40.621] --> textDocument/clangd.fileStatus
I[23:05:40.645] --> textDocument/publishDiagnostics
I[23:05:40.645] --> textDocument/inactiveRegions
I[23:05:40.645] --> textDocument/clangd.fileStatus
I[23:05:40.875] <-- textDocument/foldingRange(73)
I[23:05:40.875] --> reply:textDocument/foldingRange(73) 0 ms
I[23:05:40.909] <-- textDocument/codeAction(74)
I[23:05:40.909] --> reply:textDocument/codeAction(74) 0 ms
I[23:05:40.909] --> textDocument/clangd.fileStatus
I[23:05:41.018] <-- textDocument/semanticTokens/full/delta(75)
I[23:05:41.018] --> reply:textDocument/semanticTokens/full/delta(75) 0 ms
I[23:05:41.018] --> textDocument/clangd.fileStatus
I[23:05:41.091] <-- textDocument/documentSymbol(76)
I[23:05:41.091] --> reply:textDocument/documentSymbol(76) 0 ms
I[23:05:41.091] --> textDocument/clangd.fileStatus
I[23:05:41.226] <-- textDocument/didChange
I[23:05:41.226] --> textDocument/clangd.fileStatus
I[23:05:41.279] ASTWorker building file h:\test\src\main.cpp version 26 with command 
[H:/test/build]
"C:\\Program Files\\LLVM\\bin\\CLANG_~1.EXE" -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 "-IH:\\test\\include" -fexceptions -fcxx-exceptions "-IC:\\VulkanSDK\\1.3.268.0\\include" -DNDEBUG -o "build\\.objs\\test\\windows\\x64\\release\\src\\main.cpp.obj" "-resource-dir=C:\\Program Files\\LLVM\\lib\\clang\\17" -- "h:\\test\\src\\main.cpp"
I[23:05:41.283] --> textDocument/clangd.fileStatus
I[23:05:41.302] --> textDocument/publishDiagnostics
I[23:05:41.302] --> textDocument/inactiveRegions
I[23:05:41.302] --> textDocument/clangd.fileStatus
I[23:05:41.528] <-- textDocument/foldingRange(77)
I[23:05:41.529] --> reply:textDocument/foldingRange(77) 0 ms
I[23:05:41.570] <-- textDocument/codeAction(78)
I[23:05:41.570] --> reply:textDocument/codeAction(78) 0 ms
I[23:05:41.570] --> textDocument/clangd.fileStatus
I[23:05:41.678] <-- textDocument/semanticTokens/full/delta(79)
I[23:05:41.679] --> reply:textDocument/semanticTokens/full/delta(79) 0 ms
I[23:05:41.679] --> textDocument/clangd.fileStatus
I[23:05:41.751] <-- textDocument/documentSymbol(80)
I[23:05:41.751] --> reply:textDocument/documentSymbol(80) 0 ms
I[23:05:41.751] --> textDocument/clangd.fileStatus
I[23:05:42.479] <-- textDocument/inlayHint(81)
I[23:05:42.479] --> reply:textDocument/inlayHint(81) 0 ms
I[23:05:42.479] --> textDocument/clangd.fileStatus
I[23:05:42.727] <-- textDocument/documentLink(82)
I[23:05:42.727] --> reply:textDocument/documentLink(82) 0 ms
I[23:05:42.727] --> textDocument/clangd.fileStatus

[HighCommander4]

Maybe the direction of the slash in the "-I" flag is the reason?

Another possibility is the capitalization: compare

"-IC:\\VulkanSDK\\1.3.268.0\\include"

and

FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h'

Note that the "I" in "Include" is capitalized in the latter.

[LuncyBloont]

Maybe the direction of the slash in the "-I" flag is the reason?

Another possibility is the capitalization: compare

"-IC:\\VulkanSDK\\1.3.268.0\\include"

and

FileURI='file:///C:/VulkanSDK/1.3.268.0/Include/vulkan/vulkan_core.h'

Note that the "I" in "Include" is capitalized in the latter.

I've tested that the header insertion works fine if I use "Include" instead of "include".

[HighCommander4]

I've tested that the header insertion works fine if I use "Include" instead of "include".

I think that makes this a case of https://github.com/clangd/clangd/issues/108 then.