Allow usage of the diagnostics generated for closed files

[childenProtos]

Hello,

I am not sure if this is the right place for this feature request/question. If not please point me in the right direction.

We are currently trying to use clangd to provide completions, diagnostics, etc. for embedded target code blocks within the room DSL [1]. Our current approach is to generate wrapper documents (.c) which we save to disk so that the clangd extension can work with them. We then use the request forwarding mechanism [2] in vscode to get diagnostics, completion items, and hovers.

For hovers this even works when the generated c file is not open in vscode. For diagnostics and completion this only works if the generated c file is open in another editor tab.

Lookin at the clangd trace while the document is not open it seems as if the generated file is read by clangd when it changes. Even the correct diagnostics are published. Unfortunatly the work seems to be discarded once vscode(?) generates the didClose event. I did try to catch the diagnostics from within our extension using the onDidChangeDiagnostics hook. But that is only called after the didClose event occured.

Is there any way to access the diagnostics for closed files or is there a way to enable project wide diagnostics? If not are there plans to support that in the future?

Alternatively: If you are open to extending the api exposed from the vscode extension to support the request forwarding approach for deep integration of c into other languages we might be able to help with that.

Our alternative would be to integrate clangd within out own language server. We are currently trying to avoid that due to the additional resources needed if a second clangd has to index the project.

The trace looks like this (I can provide verbose traces if they are of any use):

I[17:18:44.763] <-- textDocument/didOpen
I[17:18:44.764] <-- textDocument/documentSymbol(3)
I[17:18:44.764] <-- textDocument/semanticTokens/full(4)
I[17:18:44.765] ASTWorker building file [.....]\Dummy.room_ADummy.c version 1 with command 
[[....]/tempContentAssist]
"C:\\Program Files\\LLVM\\bin\\clang" --config ./compile_flags.txt -D__GNUC__ -Wno-unused-parameter -D__GNUC__ -Wno-unused-parameter "-resource-dir=[....]\\AppData\\Roaming\\Code\\User\\globalStorage\\llvm-vs-code-extensions.vscode-clangd\\install\\15.0.6\\clangd_15.0.6\\lib\\clang\\15.0.6" -- "[....]\\tempContentAssist\\Dummy.room_ADummy.c"
I[17:18:44.770] --> textDocument/clangd.fileStatus
I[17:18:45.509] --> workspace/semanticTokens/refresh(2)
I[17:18:45.509] --> textDocument/clangd.fileStatus
I[17:18:45.511] <-- reply(2)
I[17:18:45.528] --> textDocument/publishDiagnostics
I[17:18:45.529] --> reply:textDocument/documentSymbol(3) 765 ms
I[17:18:45.529] --> reply:textDocument/semanticTokens/full(4) 765 ms
I[17:18:45.529] --> textDocument/clangd.fileStatus
I[17:18:45.540] <-- textDocument/didClose
I[17:18:45.540] --> textDocument/publishDiagnostics

[1] https://www.eclipse.org/etrice/ [2] https://code.visualstudio.com/api/language-extensions/embedded-languages#request-forwarding

[HighCommander4]

Is there any way to access the diagnostics for closed files or is there a way to enable project wide diagnostics? If not are there plans to support that in the future?

I'm not aware of any plans but there is an open issue about it: https://github.com/clangd/vscode-clangd/issues/158

As for your use case more generally, I admit I don't fully understand it yet. Can you say more about how you are able to get hover information for a file that is closed?

[childenProtos]

Thanks for the pointer to the open issue.

For the hover part let me add some more context about our usecase:

An eTrice '.room' file may contain areas which are marked as so called "detail code" (wrapped in '''). Within those areas the user can write code which influences the behavior of the generated entities (see example below). For the vscode extension we currently only aim to support c as detail code language.

When a .room file is edited or opened we generate one or more .c files which contain function wrappers for our detail code sections + the mapping between locations in the .c files and the .room file.

If a location within a detail code section is hovered we execute the 'vscode.executeHoverProvider' command with the mapped location and the uri of the generated c file. When that command is executed after the clangd extension is activated I get the correct hover info and the following trace:

I[09:37:34.424] <-- textDocument/didOpen
I[09:37:34.424] <-- textDocument/hover(12)
I[09:37:34.424] <-- textDocument/semanticTokens/full(13)
I[09:37:34.425] ASTWorker building file [...]\build\tempContentAssist\Dummy.room_ADummy.c version 1 with command 
[[...]/build/tempContentAssist]
"C:\\Program Files\\LLVM\\bin\\clang" --config ./compile_flags.txt -D__GNUC__ -Wno-unused-parameter -D__GNUC__ -Wno-unused-parameter "-resource-dir=[...]\\AppData\\Roaming\\Code\\User\\globalStorage\\llvm-vs-code-extensions.vscode-clangd\\install\\15.0.6\\clangd_15.0.6\\lib\\clang\\15.0.6" -- [...]\\build\\tempContentAssist\\Dummy.room_ADummy.c"
I[09:37:34.432] --> textDocument/clangd.fileStatus
I[09:37:35.201] --> workspace/semanticTokens/refresh(6)
I[09:37:35.201] --> textDocument/clangd.fileStatus
I[09:37:35.203] <-- reply(6)
I[09:37:35.219] --> textDocument/publishDiagnostics
I[09:37:35.220] --> reply:textDocument/hover(12) 796 ms
I[09:37:35.221] --> reply:textDocument/semanticTokens/full(13) 796 ms
I[09:37:35.221] --> textDocument/clangd.fileStatus
I[09:37:35.236] <-- textDocument/didClose
I[09:37:35.236] --> textDocument/publishDiagnostics

For completions we are executing the 'vscode.executeCompletionItemProvider' command. If the file is open in another tab when we request the completions we get the completions we would expect. If the file is closed we also get a set of completions, but it is different and not as good as the one we get with the open file. My guess is that clangd is not done analyzing the file in the second case.

Ooutput with file open: I[09:45:16.496] Code complete: 105 results from Sema, 100 from Index, 2 matched, 0 from identifiers, 100 returned (incomplete). Output with file closed: I[09:39:46.826] Code complete: 0 results from Sema, 100 from Index, 0 matched, 33 from identifiers, 100 returned (incomplete).

I guess both might be solvable if the files would be kept open in clangd. I will have a closer look at the issue you linked.

ActorClass ADummy {
        Interface {
            Port dummy: etrice.api.timer.PTimer
            Port dummy2: PTest
        }
        Structure {
            usercode1 '''
                // C code here
                #include <stdint.h>
                #include "i2cTestTypes.h"

                typedef struct MyLocalType {
                    uint16_t memberOne;
                } MyLocalType;

            '''           
            usercode3 '''
                // C code here
                static MyLocalType userCodeLocalVar = {0};
            '''
            SAP timer: PTimer
            external Port dummy
            external Port dummy2
            Attribute testAttribute: uint8
            Attribute someAttribute: uint8
            Attribute someArrayAttribute[10]: uint8
        }

        Behavior {
            Operation doSomeDummyThing(param1: uint32): uint8 '''
                // C code here
                someAttribute = (param1 >> 24);

                return someAttribute; 
            '''
            StateMachine {
                State active
                State active2
                Transition init0: initial -> active {
                    action '''
                            // C code here
                           dummy.timeout();
                           dummy2.testMsgOut(5);

                           userCodeLocalVar.memberOne = 7;
                    '''
                }
                Transition tr1: active -> active2 {
                    triggers {
                        <testMsg: dummy2>
                    }
                    action '''
                       // C code here
                       if (transitionData == 12) {
                        dummy2.testMsgOut(12);
                        someAttribute = 7;
                       }
                    '''
                }

                Transition tr2: active2 -> active {
                    triggers {
                        <testMsg: dummy2>
                    }
                    action '''
                         // C code here
                        dummy.timeout();
                        

                    '''
                }
            }
        }
    }

[HighCommander4]

Ok, I think I understand better now: behind the scenes, vscode's "execute provider" functionality is opening the file over LSP with textDocument/didOpen, performing the operation (e.g. hover), and then closing the file with textDocument/didClose.

And this works well with hover because the response to hover blocks on clangd's AST build, but it doesn't work well for completion because the response to completion is sent without blocking on the AST (with results from the project's index only) if the AST is not ready.

(I'm not sure how diagnostics fit into this picture. Is there a "provider" for those the way there is for hover and completion? Diagnostics seems a bit different because they are not a response to a request, they are just pushed from the server to the client when available.)

Anyways, one thing I wonder is: does vscode expose the necessary APIs for you to trigger a textDocument/didOpen on the file in question directly (without physically opening the file in the editor)? If so, you could consider doing that for the various generated .c files in your open DSL file.