Sourcekit-LSP stops working after a cancel of completion

[angelozerr]

Swift version

Last

Platform

Windows 11

Editor

LSP4IJ (Free LSP support for IJ)

Description

Hi,

I contact you because I'm trying to consume Sourcekit-LSP with LSP4IJ a free LSP support for IntelliJ.

If you want to test it, please read https://github.com/redhat-developer/lsp4ij/blob/main/docs/user-defined-ls/sourcekit-lsp.md after installing LSP4IJ

All features are working pretty well (hover, highlight, semantic tokens, call hierarchy, etc) except completion. Completion is working pretty well if you type characters in the editor slowly, but as soon as you are typing quickly in the editor, the Sourcekit-LSP breaks all LSP features (no highlight, no hover, no completion, etc).

Please note that all language servers that I have tested like Go, Rust, etc are working pretty well. First I though it was a problem with didChange, but I send the same didChange notification than vscode.

After spending a lot of time to discover the usecase which causes the problem, I have noticed that this problem occurs when completion is cancelled.

As soon as I am typing quickly in the editor, it send cancelRequest for completion and after that the Sourcekit-LSP is down (it gives me some response but with cancel and not only for completion, for highlight, hover etc).

It have the impression that I have the similar problem than Emacs-LSP https://github.com/emacs-lsp/lsp-mode/issues/3028 but I find it is very strange to kill the Sourcekit-LSP when LSP client cancels the completion.

I have the impression that I don't send the proper triggerChars for completion after completion is cancelled. Perhaps I need to set triggerChars to 3 when completion is canceled but I have no problem with other LSP language servers?

I wanted just to share my problem.

Any help are welcome! Thanks!

Steps to Reproduce

No response

Logging

No response

[ahoppen]

Synced to Apple’s issue tracker as rdar://141508699

[ahoppen]

Thanks for the report, @angelozerr. Which version of SourceKit-LSP are you using? If you're using a Swift development snapshot, there was an issue with cancellation that sound similar and which I fixed recently (https://github.com/swiftlang/sourcekit-lsp/pull/1857).

If that's not it, could you run sourcekit-lsp diagnose and attach the diagnostics bundle to this issue? That should help us determine what's going wrong.

[angelozerr]

Which version of SourceKit-LSP are you using?

Swift version 6.0.2 (swift-6.0.2-RELEASE)
Target: x86_64-unknown-windows-msvc

First here my simple test. I have in my project just this file and I'm trying to open completion after name.:

var name = ""
name.

If I open completion slowly, it works great. If I write quickly some character after name. there is a cancelRequest (done by IntelliJ LSP client when completion is not relevant) and after that SourceKit-LSP is down.

Here the generated file with sourcekit-lsp diagnose:

sourcekit-lsp-32248.0.log

This diagnose is very strange because we have the impression that everything is working pretty well, but it is not true.

Here the LSP traces that I have in my IntelliJ LSP console:

lsp-traces.txt

If you see my LSP trace, completion is working upon the (8) completion request which report a cancel:

[Trace - 16:06:41] Sending request 'textDocument/completion - (8)'.
Params: {
  "context": {
    "triggerKind": 2,
    "triggerCharacter": "."
  },
  "textDocument": {
    "uri": "file:///C:/Users/azerr/IdeaProjects/untitled13/test.swift"
  },
  "position": {
    "line": 1,
    "character": 5
  }
}


[Trace - 16:06:41] Sending notification '$/cancelRequest'
Params: {
  "id": "8"
}

And after that if I reopen completion, there are none LSP response of textDocument/completion although in the diagnose there are some LSP response?

After debugging the LSP client, it seems that SourceKit-LSP doesn't write any response in the input stream. We can send request but there are no answer from the SourceKit-LSP language server although the SourceKit-LSP process is alive?

The diagnose displays a response of the 9 request:

---[org.swift.sourcekit-lsp:request-9] default 2024-12-16 14:00:28.4370 +0000
Received request "9": textDocument/completion
{
  "context" : {
    "triggerCharacter" : ".",
    "triggerKind" : 2
  },
  "position" : {
    "character" : 5,
    "line" : 1
  },
  "textDocument" : {
    "uri" : "file:///C:/Users/azerr/IdeaProjects/untitled13/test.swift"
  }
}
---[org.swift.sourcekit-lsp:request-9] default 2024-12-16 14:00:28.8400 +0000
Succeeded (took 402.39715576171875ms)
textDocument/completion
CompletionList
{
  "isIncomplete" : true,
  "items" : [
    {
      "deprecated" : false,
      "detail" : "Void",
      "documentation" : {
        "kind" : "markdown",
        "value" : "Appends the given character to the string."
      },
      "filterText" : "append(:)",
      "insertText" : "append(${1:Character})",
      "insertTextFormat" : 2,
      "kind" : 2,
      "label" : "append(c: Character)",
      "textEdit" : {
        "newText" : "append(${1:Character})",
        "range" : {
          "end" : {
            "character" : 5,
            "line" : 1
          },
          "start" : {
            "character" : 5,
            "line" : 1
          }
        }
      }
    },

and in my LSP trace you can see just:

[Trace - 16:06:41] Sending request 'textDocument/completion - (9)'.
Params: {
  "context": {
    "triggerKind": 1
  },
  "textDocument": {
    "uri": "file:///C:/Users/azerr/IdeaProjects/untitled13/test.swift"
  },
  "position": {
    "line": 1,
    "character": 6
  }
}

without response.

Please note that you can see a cancel of request 9:

[Trace - 16:06:41] Sending notification '$/cancelRequest'
Params: {
  "id": "9"
}

but this cancel occurs when completion is closed (I'm waiting for some times before closing it).

[ahoppen]

Oh, I suppose the cancellation of the code completion request somehow gets the code completion session that SourceKit-LSP maintains with sourcekitd in an invalid state. I can debug this in the new year.

[angelozerr]

Thanks so much @ahoppen !

[ahoppen]

Thanks for your detailed analysis, @angelozerr. Here’s my analysis:

• I looked into my initial suspicion of the completion session getting into an invalid state but that suspicion was wrong.
• It seems like the $/cancelRequest notifications are just missing from the SourceKit-LSP logs. I’m fixing that in https://github.com/swiftlang/sourcekit-lsp/pull/1904.
• I suspsect that the cancellation of request 8 didn’t get processed before we finished producing the results and thus we returned results and didn’t honor the cancellation. Since cancellation is best-effort this is LSP-compliant behavior.
• I am extremely surprised by the complete lack of responses logged in InelliJ’s LSP log. We had an issue like that a year ago (https://github.com/swiftlang/sourcekit-lsp/issues/982) but that was when a response exceeded a certain size and should not be order-dependent. Do you have access to a Linux or macOS machine? If so, I would be curious if this is a Windows-specific issue (which might indidate that we have more bugs with pipe-handling on Windows) or if it’s an issue specific to IntelliJ. With the information I have right now, I think this could also be a bug in IntelliJ, if it gets a response for a request that it has cancelled.
• https://github.com/emacs-lsp/lsp-mode/issues/3028 seems to be a different issue with offsets in the code completion session mismatching. I don’t remember which exact PR it was, but I changed that behavior recently-ish so that this kind of issue should not occur anymore.
• Could you try if the issue persists with a Swift 6.1 development snapshot? I know that I fixed a few bugs on Windows recently and maybe this was somehow covered by one of these.

[angelozerr]

Thanks for your detailed analysis, @angelozerr.

You are welcome!

For your information, It is working on Windows OS with vscode, so the problem comes from LSP4IJ but I don't know which fix I must to do since completion is working with all language servers that I have tested (Go, Rust, TypeScript, etc).

Could you try if the issue persists with a Swift 6.1 development snapshot?

Could you explain me please how to install this snapshot please.

[ahoppen]

Could you explain me please how to install this snapshot please.

You can download the snapshot from https://www.swift.org/install/windows/#development-snapshots. If I remember correctly, you might need to uninstall Swift beforehand because otherwise the two version conflict with each other.

Also, I’m adding https://github.com/swiftlang/sourcekit-lsp/pull/1905, which might help us determine if SourceKit-LSP is not producing output or if the client is ignoring the output.

[angelozerr]

When I go to https://www.swift.org/install/windows/#development-snapshots

The date is 21 december, is it correct?

[ahoppen]

Yes, that does appear to be the latest version. It should contain the fixes I made recently.

[bnbarham]

Unfortunately SwiftPM is broken in that release - I'm fixing that in https://github.com/swiftlang/swift/pull/78415 and https://github.com/swiftlang/swift-installer-scripts/pull/359. It might be simpler to build sourcekit-lsp and copy that into your existing toolchain instead (or wait until those fixes are in).

[angelozerr]

Unfortunately SwiftPM is broken in that release - I'm fixing that in https://github.com/swiftlang/swift/pull/78415 and https://github.com/swiftlang/swift-installer-scripts/pull/359. It might be simpler to build sourcekit-lsp and copy that into your existing toolchain instead (or wait until those fixes are in).

Indeed i have tried to install it and the the language server cannot be started. Please ping me when I will able to install the snapshot version. Thanks!

[ahoppen]

@angelozerr We have a new Windows toolchain snapshot now. Could you try:

• Downloading and installing a main Swift toolchain snapshot from https://www.swift.org/install/windows/#development-snapshots?
• Adding a .sourcekit-lsp/config.json configuration file into your home directory with the following contents (or something similar), which should mirror all of SourceKit-LSP’s communication into files in the specified directories (please double-check that the JSON doesn’t have syntax errors, I typed it on GitHub).

{
  "logging": {
    "inputMirrorDirectory": "C:\\Users\\<your user>\\Desktop\\inputMirror",
    "outputMirrorDirectory": "C:\\Users\\<your user>\\Desktop\\outputMirror"
  }
}

• Reproduce the issue and attach the files in the inputMirror and outputMirror directories as well as the IntelliJ LSP trace.

I hope that with that we’ll be able to tell where the messages are starting to get dropped.

[angelozerr]

@ahoppen I have changed my laptop and after installing last version of Swift toolchains, everything seems working pretty well now.

Thanks for having helped me to fix this issue!

[ahoppen]

That’s great to hear. Thanks for checking, @angelozerr!