Document the first steps exactly on a tiny example

[tmssngr]

trafficstars

Please document all steps for getting started (https://github.com/eclipse/lsp4j/blob/main/documentation/README.md) as detailed as possible, so any Java experienced developer can get started easily, e.g.

• what jars are needed (download manually or using Maven)?
• from where to get the InputStream/OutputStream?
• where the myImpl (LanguageClientAware) comes from? Right now it looks like a documentation written to users who already know lsp4j which is not very helpful for those who don't know it. Thanks in advance.

[nixel2007]

All of these questions are about java ecosystem itself, and have no special meaning in scope of lsp4j.

[tmssngr]

You might be right, I just have 20 years of Java experience, but not with the Eclipse RCP (just SWT). And I'm just thinking about writing a language server for an ancient 8 bit processor, but don't know how to start. At least the "Getting started" topic does not help in this aspect.

[nixel2007]

About your questions.

You can grab jars from maven central. Example for gradle: implementation("org.eclipse.lsp4j", "org.eclipse.lsp4j", "0.12.0")

Streams source depends on your wiring method with client. Most of servers use stdout/stdin for communication. You can check other variants in LSP documentation (implementation notes).

To make your server you need to implement three interfaces: LanguageServer, TextDocumentService and WorkspaceService.

You don't need any implementation of LanguageClient at server side if your language client does not have any non-standard requests or notifications, so you can pass base interface to language server launcher.

I don't know exact requirements for implementation of language client, but I'm pretty sure that you should start with implementation of LanguageClient interface at client side.

But I have one question - do you really want to wire your devices with language server protocol? May be basic json-rpc would be enough? You can use json-rpc without all langserver stuff.

[tmssngr]

Sorry for the ambiguous information. I want to write a language server for assembly language for this ancient 8 bit processor. It should be used either with Eclipse or VSCode to develop software on a normal PC (that then will be uploaded manually to the device). So from my point of view it would be super helpful if the documentation would show how to create a very simplistic language server for use with Eclipse for a extremely simple language (e.g. a trivial text file where references are just searched literally). From what I understand so far, the language server (usually?) is an executable that is launched by the editor/IDE (client) and both communicate by the JSON based language server protocol which LSP4j should hide from the developer. I also don't know whether the language server actually needs to hold the whole file contents internally, whether it gets everything from the client or needs to access to the project's file system outside the client-server communication. It would be super if the "Getting Started" section would cover all these information, so a Java developer can easily use LSP4j to write tool support for their pet programming language.

BTW, if there already are language servers available using LSP4j, it also would help to tell the reader of the "Getting Started" how to get it running and to debug it.

[jonahgraham]

Hi @tmssngr - I do agree that LSP4J is light on documentation for getting started with languages servers. LSP4J is intended for people who know what LSP is already.

It is probably worth starting with the LSP documentation at https://microsoft.github.io/language-server-protocol/overviews/lsp/overview/ and VSCode's guide on creating a language server https://code.visualstudio.com/api/language-extensions/language-server-extension-guide

We really could have a link back to language servers using LSP4J. In the meantime, https://microsoft.github.io/language-server-protocol/implementors/servers/ contains many of the known LSP implementors and many of the Java ones listed use LSP4J.

[nixel2007]

BSL Language Server is written with lsp4j + spring as its basis.

https://github.com/1c-syntax/bsl-language-server

Feel free to ask any questions about ls impl, but you really should check official documentation about LSP itself. The only thing Language server must do is communicate via protocol. Most of other things is implementation specific.

[angelozerr]

I agree with @tmssngr. I think it should be really nice to provide a minimal sample of language server written with LSP4J (like vscode provides if I remember). I have that in my mind since a long time but I have not found time to do that.

More I think we should have have high level class which helps the write of the language server like:

• a TextDocument (like vscode provides) which maintains the String of the document after some didChange, and gives the capability to translate an int offset to LSP4J Position, compute the AST, process validation (with some delay) when didChange is occurs, etc
• kill correctly the language server
• provides the capability to create snippet on server side

We provide that in https://github.com/eclipse/lemminx/tree/master/org.eclipse.lemminx/src/main/java/org/eclipse/lemminx/commons and my idea is to provide a project "commons language server" to use it outside to LemMinx our XML language server. We use thoses classes on 2 another language server (MicroProfile LS and Quarkus Qute LS).

A sample should be good to show the best practivce of LSP4J, for instance when I see https://github.com/1c-syntax/bsl-language-server/blob/c2fafa28078378b7e3385e6726390f4d542e4a66/src/main/java/com/github/_1c_syntax/bsl/languageserver/BSLTextDocumentService.java#L118 it doesn't take care of client cancel. For having good performance, the cancel checker should be used.

@tmssngr if you are interested by the idea to have "commons language server", please tell me, I will see if we can provide it. The idea of "commons language server" is to provide the same feature than vscode like TextDocument, TextDocuments, etc.

[nixel2007]

to provide a project "commons language server"

did you see the LXTK project? https://github.com/lxtk-org/lxtk

Also we (bsl ls team) have internal project with genering of BSL LS core to separate "core ls" with plugin/addin functionality for concrete LSs. But we are not ready to share it, tbh

[angelozerr]

@nixel2007 perhaps Im wrong but it seems your project is linked to eclipse ui? I dont see some implementation of TextDocumentService? Can we use the language server in vscode?

[nixel2007]

@angelozerr is this question about lxtk? If yes, then answer is "no". It has generic part with basic structures over lsp4j and an eclipse bindings as separated subprojects.

[nixel2007]

Maybe @pisv could provide more info about lxtk? I am a watcher of this project, not a user. BSL Language Server is built over lsp4j itself, without lxtk

[pisv]

Just to clarify, LXTK (at least as it now stands) is a client-side framework built on top of LSP4J, i.e. it does not provide any means for creation of language servers (that's currently outside the scope of the project).

If I understand @nixel2007 correctly, the idea is that a similar framework could be created for the server-side.

If you ask me as an LSP4J committer and also from my experience with creating LXTK, I think that it would be better to create such a server-side framework as a separate project on top (but outside) of LSP4J.

LSP4J as it now stands has a clear scope as a lower-level framework with the intended symmetry of use in both language servers and clients.

But that's just my take on it, I know there are other opinions, as discussed previously in #182.

[nixel2007]

@pisv thanks for clarification!

[tmssngr]

I had the hope to avoid having to learn TypeScript, Gradle or Eclipse plugin development to write a language server (for Eclipse or VSCode) with LSP4j, because this is just for a tiny hobby project with limited time frame. Hence I've looked at LSP4j hoping to make use of my "normal" Java knowledge. I'm not sure whether a "commons language server" would help me (unless I misunderstand what it does). What makes it hard for me is that often there are so many new languages/tools involved that hide what is essentially needed and how things work together.

[tmssngr]

I think what would help me further would be the simplest possible code to get up and running (without the use of special build tools like Gradle, Maven, you name it) that I can debug. Usually languages need lexing/parsing, but for the simplest possible code it would IMHO be sufficient to treat the text as plain text and, for example, find usages of words, being able to jump to the first word. This is easy enough to understand for newbies without the need to introduce, e.g. ANTLR.

[pisv]

Hi @tmssngr - although not exactly what you ask - when searching for "lsp4j tutorial" I have found a number of useful links, including the following:

• Build Language Servers in Java
• Implementing a JSON-RPC protocol with Eclipse LSP4J

Hope this might help you get started on your project.

[tmssngr]

@pisv Thanks. I've already tried https://github.com/LucasBullen/LSP4J_Tutorial, but some essential parts are missing to get it compiling. Maybe it worked 5 years ago out of the box.

[pisv]

FWIW, I have updated the LSP4J Tutorial in my personal fork to make it compile and run with the current version of LSP4J: https://github.com/pisv/LSP4J_Tutorial. I think there is still room for improving the tutorial and will happily accept pull requests to that end.

[tmssngr]

Thanks. I need to confess, I don't use Eclipse but IDEA on a daily base and hence I'm already blocked with the very first instruction.

Open the target-platform.target file in the org.eclipsecon.languageserverplugin project and click on Set as Active Target Platform.

I don't know how to Set as Active Target Platform. Maybe assume, that the user is not an Eclipse expert.

[pisv]

@tmssngr You need to open this file with with the Target Editor (right-click on the file, choose Open With > Target Editor). Then, in the Definition tab, click on Set as Active Target Platform (the hyperlink in the top-right corner of the editor). I'll try to make it more clear in the tutorial.

[tmssngr]

Hm, there is no Target Editor option, neither in the context menu nor in the dialog which occurs when clicking Other.

[pisv]

Sorry, I did not notice that the original tutorial was vague about what Eclipse installation you need for exercises, and have not fixed it yet. Please ensure you have Eclipse Plug-in Development Environment installed, e.g. by downloading Eclipse IDE for Eclipse Committers.

[tmssngr]

Thanks. Now I was able to finish exercise 1. When debugging the completion the connection often gets lost - maybe a timeout issue?

[pisv]

@tmssngr Thanks for the feedback. I can reproduce the issue. It is sufficient to open the exer1.txt file with the Generic Editor and wait long enough for the following exception to be logged:

SEVERE: java.io.IOException: Pipe broken
org.eclipse.lsp4j.jsonrpc.JsonRpcException: java.io.IOException: Pipe broken
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:122)
	at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:113)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: Pipe broken
	at java.base/java.io.PipedInputStream.read(PipedInputStream.java:321)
	at java.base/java.io.FilterInputStream.read(FilterInputStream.java:83)
	at org.eclipsecon.languageserverplugin.AbstractConnectionProvider$1.read(AbstractConnectionProvider.java:49)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:79)
	... 6 more

This issue is caused by the AbstractConnectionProvider implementation being based on piped streams. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=565834 for more details.

Essentially, the AbstractConnectionProvider implementation needs to be updated in a way similar to https://github.com/eclipse/lsp4e/commit/004aae804f8f92fc0e1561224fe9f98455a6ff1b. Would you like to provide a PR? Or, I can try to fix it myself a bit later.