Add support for cancellation and progress

[spoenemann]

LSP supports cancellation and progress reporting. We should consider this in the APIs for LSP services.

[sailingKieler]

Regarding cancellation:

https://user-images.githubusercontent.com/8513883/131699130-13dcbc0a-dbac-4aec-b50d-c485a85d70cd.mp4

I achieved this by changing https://github.com/langium/langium/blob/2f18b81256d1ca475eea9b8da955cb156517522a/packages/langium/src/lsp/language-server.ts#L129-L139

to

export function addDocumentHighlightsHandler(connection: Connection, services: LangiumServices): void {
    const documentHighlighter = services.lsp.DocumentHighlighter;
    connection.onDocumentHighlight((params: DocumentHighlightParams, canceled: CancellationToken): Promise<Location[]> => {
        return new Promise((resolve) => {
            console.log(`Initially canceled: ${canceled.isCancellationRequested}!`)
            setTimeout(() => {
                console.log(`After some time canceled: ${canceled.isCancellationRequested}!`)
                const document = paramsDocument(params, services);
                if (document) {
                    resolve(documentHighlighter.findHighlights(document, params));
                } else {
                    resolve([]);
                }
            }, 3000)
        })
    });
}

No further changes were necessary. Especially the hook offered by vs code extensions in the LanguageClientOptions to register a dedicated cancellationStrategy like shown below is not required, the default implementations seem to work as expected.

const cancelNotificationType = new NotificationType('$/cancelRequest')

// Options to control the language client
const clientOptions: LanguageClientOptions = {
    documentSelector: [{ scheme: 'file', language: 'my-language' }],
    synchronize: {
        // Notify the server about file changes to files contained in the workspace
        fileEvents: workspace.createFileSystemWatcher('**/*.[".mylang"]')
    },
    connectionOptions: {
        cancellationStrategy: {
            sender: {
                sendCancellation(conn : MessageConnection, id: CancellationId) : void {
                    conn.sendNotification(cancelNotificationType, { id });
                },
                cleanup(id: CancellationId) : void {}
            },
            receiver: {
                createCancellationTokenSource(id: CancellationId) {
                    return new CancellationTokenSource();
                }
            }
        }
    }
};

Note: In case customizations are made here, corresponding/symmetric customizations need to be done on the LS side as well.

This is possible via not just calling const connection = createConnection(ProposedFeatures.all) in the LS' main.ts, but calling

const cancelNotificationType = new NotificationType('$/cancelRequest')

// Create a connection to the client
const connection = createConnection(ProposedFeatures.all, {
    cancellationStrategy: {
        sender: {
            sendCancellation(conn : MessageConnection, id: CancellationId) : void {
                conn.sendNotification(cancelNotificationType, { id });
            },
            cleanup(id: CancellationId) : void {}
        },
        receiver: {
            createCancellationTokenSource(id: CancellationId) {
                return new CancellationTokenSource();
            }
        }
    }
});

In the use cases I observed, the client called cancellationStrategy.sender.sendCancellation(..., id), with id being the continuous number (id) of the corresponding preceding RequestMessage. For the documentHighlight use case such cancellation messages seem to be sent always before another request is sent even if the original request has been already answered by the LS (at least as far as I could observe). The cancellation requests seem to be also sent on document changes (character insertion/removal), I assume before the onDidChangeTextDocument msg, but I didn't verify.

On the receiver side, typically the LS side, one of the early steps during handling a request message (in https://github.com/microsoft/vscode-languageserver-node/blob/eba6a7308b21ab94bd412fbfa63e36964b6d82ad/jsonrpc/src/common/connection.ts#L644) is to create a cancellation token and keep it in a local memory, see https://github.com/microsoft/vscode-languageserver-node/blob/eba6a7308b21ab94bd412fbfa63e36964b6d82ad/jsonrpc/src/common/connection.ts#L699-L703

After corresponding handler processed the request, the cancellation token is basically always immediately dropped except for the case the request handler's result is a promise, see https://github.com/microsoft/vscode-languageserver-node/blob/eba6a7308b21ab94bd412fbfa63e36964b6d82ad/jsonrpc/src/common/connection.ts#L730-L751. In that case the dropping is executed as soon as the promise is resolved.

In case a cancellation request is sent to the LS, the LS looks for a corresponding cancellation token, and if none is present anymore, the request won't have any effect, see https://github.com/microsoft/vscode-languageserver-node/blob/eba6a7308b21ab94bd412fbfa63e36964b6d82ad/jsonrpc/src/common/connection.ts#L625-L631 (this impl is slightly different compared to my currently used version)

Long story short: As long as the LS, more precisely the request specific handler, eagerly processes the request, it will never see the cancellation token to change from non-canceled to canceled, as the processing cannot be interrupted, we have no (LS internal) concurrency, and thus no cancellation request can be processed until the handler finished it's work.

Consequently client-side cancellation will only be effective, if the request handler returns a promise and the resolution of the promise is delayed.

Not clear to me yet: Can we refactor the current handlers to be able to receive the cancellation and react on them, and if so should we? Need to dig deeper to get an impression, how re-parsing and the downstream activities like validation are currently integrated.

[spoenemann]

We could change all services to return something like ServiceResult:

export type ServiceResult<T> = T | Promise<T>

And pass the cancellation token as parameter, either directly or inside an extensible "context" object, which we could also reuse for progress, partial results etc.

How much we make use of that in the default implementations, for example for document highlights, is another question. It depends on the expected worst-case performance.

[sailingKieler]

My findings regarding progressReporting and partialResult:

The client has to provide corresponding tokens in the particular request messages, like shown in https://microsoft.github.io/language-server-protocol/specifications/specification-current/#partialResults I guess there is no rule on how clients construct such tokens.

Unfortunately, AFAICS vs code doesn't contain an implementation of this token provision and, more importantly, doesn't contain implementations accepting and collecting partial results of, e.g., completion items, symbol references, etc. See the current vs code implementations of sending requests for and processing responses of completions and symbol references:

• completion: https://github.com/microsoft/vscode-languageserver-node/blob/8ad6896d8910214b7a3c8fee8dd175077a743045/client/src/common/client.ts#L1636-L1650 with

  • asCompletionParams: https://github.com/microsoft/vscode-languageserver-node/blob/8ad6896d8910214b7a3c8fee8dd175077a743045/client/src/common/codeConverter.ts#L314-L323
  • asCompletionResult: https://github.com/microsoft/vscode-languageserver-node/blob/8ad6896d8910214b7a3c8fee8dd175077a743045/client/src/common/protocolConverter.ts#L429-L443

• references: https://github.com/microsoft/vscode-languageserver-node/blob/8ad6896d8910214b7a3c8fee8dd175077a743045/client/src/common/client.ts#L1838-L1854 with

  • asReferencesParams: https://github.com/microsoft/vscode-languageserver-node/blob/8ad6896d8910214b7a3c8fee8dd175077a743045/client/src/common/codeConverter.ts#L687-L693
  • asReferences: https://github.com/microsoft/vscode-languageserver-node/blob/8ad6896d8910214b7a3c8fee8dd175077a743045/client/src/common/protocolConverter.ts#L696-L704

Question: Is there already a client implementing the support of progress reporting & partial results? If not, we would have to implement them in our language server haphazardly :-/.

[spoenemann]

Cancellation is now supported (#238), so what's left to do here is progress reporting.