Solargraph with Docker

[vimaciel]

I'm using docker in my rails app. How can I work with Solargraph and Docker? Is there any configuration to do?

[castwide]

I don't know enough about docker to provide any specific advice. I can tell you that Solargraph needs to be able to resolve the gem paths in order to provide completion and documentation for them. It'll try to use your Gemfile to do so.

[vimaciel]

Thank you for answer my issue. Your project is awesome. Where do you set the path of ruby's library? Maybe we can work together to create this new feature.

[castwide]

Thanks!

Solargraph uses the runtime load paths to find gems. The extension starts the server process using your open folder as the current working directory, so if you have any files in the root that would affect the environment, such as a Gemfile or a .ruby-version file, it should use them automatically.

You might encounter a problem if your container has its own gem cache or makes any other changes to the runtime environment. I'm not sure if anything special needs to be done to start the process using the environment defined by the container. It might require an enhancement; for example, there's currently no mechanism for explicitly setting load paths. One workaround might be to install the same gems the container uses to your system Ruby environment, where Solargraph should be able to detect them.

I'm open to enhancements or features to improve Docker support if necessary. Any help you can provide is appreciated, whether it's bug reports, suggestions, or pull requests.

[jwkicklighter]

Would love this feature

[mantenie]

Me too. This is what I'm looking for since I use docker.

[markeissler]

All you should be doing to develop with Docker is to mount your project directory, as a volume, into the Docker container. You edit your files locally.

[mantenie]

Isn't it necessary to mount the bundle directory as a volume and link it into my develop folder as well? Or needs Solargraph just access to the Gemfile?

[stefansedich]

Docker support would be great, and more importantly support to work within a docker compose stack.

I am working on a reference project and enhancements to vscode-ruby to provide a nice debugging experience when using docker compose and might look at how this can spill over to this extension next.

Ideally I want to never have to install gems locally and have things like Solargraph work with the gems that have been installed inside the container (perhaps by starting the server inside a container and connecting to that instead), but even in the case of debugging I still need to install gems locally right now to provide the ability to step into gem sources without volume binds which I found in some environments was not ideal.

I am not sure there will be an easy one size fits all solution but if anyone wants to get together and chat about it and share some ideas it would be great as going forward solving the Docker story is going to be beneficial and definitely something I want to help with.

[RtypeStudios]

I've also been looking into docker with solargraph.

From what I can tell (and please correct me if i'm wrong):

Solargraph has a server (the gem) that can generates all of the information the solargraph vs extensions needs about your code. The communicates to vs by using event machine and exposing a connection over port 7658.

What I tried doing is running the solargaph server in a separate docker container pointing to the source code of my ruby app on my machine along side my other dockers (web, db)

version: '3.4'
services:

  db:
    build:
      context: ./db
      dockerfile: dockerfile
    environment:
        POSTGRES_DB: testproject_development
        POSTGRES_USER: postgres
        POSTGRES_PASSWORD: password
    ports:
      - "5432:5432"

  app:
    build:
      context: ./app
      dockerfile: dockerfile
    command: foreman start
    volumes:
      - ./app:/app
    ports:
      - "5000:5000"
    depends_on:
      - db

  solar:
    build:
      context: ./app
      dockerfile: dockerfile
    command: solargraph socket --host=0.0.0.0 --port=7658
    volumes:
      - ./app:/app
    ports:
      - "7658:7658"
    depends_on:
      - app
      - db

While this seems to start okay. I cant seem to communicate with solargraph on that port from my host. Within the solar container using wget I can a connection. But from the host nothing seems to reply on that port.

Has anyone tried this, or had any luck doing something similar?

[castwide]

This Dockerfile by Sourcegraph might be helpful: https://hub.docker.com/r/sourcegraph/codeintel-ruby/

[jwkicklighter]

@castwide Wish the Dockerfile was easy to find...

[RtypeStudios]

I have dug a bit further into this and spun up the VS code extension and stepped through what is going on.

From what I can tell my above docker setup should work file. But the VS extension itself would need to be changed to point to the port docker is exposing and to not start its own copy of solargraph.

In the solargraph-utils project there is code the extension depends on that handles socket provisioning and starting of solargraph (SocketProvider.js).

Below you can see it starts the solargraph gem on the local machine (command_1) with the port 0 option. Solargraph server then starts on the next available port. Once started the code parses out what port was allocated then returns that back to the solargraph extension.

        this.child = commands_1.solargraphCommand(['socket', '--port', '0'], this.configuration);
        let buffer = '';
        let that = this;
        this.child.stderr.on('data', (data) => {
            console.log(data.toString());
            if (!that.isListening()) {
                buffer += data.toString();
                var match = buffer.match(/PORT=([0-9]*)[\s]+PID=([0-9]*)/);
                if (match) {
                    that.listening = true;
                    that._port = parseInt(match[1]);
                    that._pid = parseInt(match[2]);
                    resolve();
                }
            }
        });

To be able to connect the VS extension to a running docker instance this code would need to be changed to allow choosing a port and to be told not to start a local instance of solargraph (otherwise it throws an error).

It is late here will explore further over the next couple days.

[jwkicklighter]

Great find! If that's the case, it seems like having an option to specify the port within VS Code's settings (and have it auto select like it currently does if that option is empty) shouldn't be that difficult.

[castwide]

The SocketProvider's primary purpose is to acquire an available port to avoid conflicts, e.g., if you're running multiple VS Code windows. If we add an option to specify the port, vscode-solargraph should be able to bypass the SocketProvider altogether.

There are a couple other issues to consider, but overall I think this is feasible.

[RtypeStudios]

Of course! I hadn't considered to possibility of multiple instances of VS. I thought it was just good house keeping on your part (Always making sure it uses a free port).

I will have a hack at the code. To see if I hard code things it will work. If so then I can work with Fred to come up with a patch if that suits everyone?

[castwide]

@RtypeStudios If you can work with me on a patch, I'm happy to merge it into the extension. You seem like a good subject matter expert where Docker is involved. Let me know if you have any problems in development.

[castwide]

The (slightly misnamed) stdio branch adds two new configuration options:

solargraph.transport: Either socket or external. If set to external, the extension will connect to an existing server instead of starting its own. Default is socket.

solargraph.externalServer: An object with host and port keys. Default is localhost:7658.

I expect to support a stdio transport as well, but it's on hold due to an unidentified bug that crashes the server process.

[RtypeStudios]

@castwide Thanks! Sorry, I haven't got back to you about all this yet. I've been pulled off onto an urgent project. Will try and take a look this weekend.

[castwide]

This feature is now in the master branch. The supported transports are socket, stdio, and external.

The configuration to connect to an external server should look something like this:

"solargraph.transport": "external",
"solargraph.externalServer": {
  "host": "localhost",
  "port": 7658
}

This feature is still open to change. I'm not even sure if it's the best solution for Docker integration, although it seems like it does the job for the most part.

One known issue: the server shuts down when the client disconnects. That probably shouldn't happen with external transports.

[RtypeStudios]

@castwide Thank you for adding that in!

I've just been giving it a try but haven't had much luck.

Just to make sure I'm doing the right thing these are the steps I took:

1. Cloned repo.
2. Switch to stdio or master (I tried both).
3. Pull latest.
4. run npm install.
5. Open the solargraph project with VS code.
6. Started up solar server in docker-compose using:

  solar:
    image: sourcegraph/codeintel-ruby
    volumes:
      - ./app:/app
    ports:
      - "7658:8080"
    depends_on:
      - app
      - db

7. Start debugging (I got an error could not find the task 'npm'. I ignored it).
8. New vscode window appears loading the extension.
9. Created a file in the root of the project called ".solargraph.yml" with the following:

"solargraph.transport": "external",
"solargraph.externalServer": {
  "host": "localhost",
  "port": 7658
}

10. Started editing ruby file. Tried to trigger auto completion.

I still get the cant contact server error. I added the following to the select client method to make sure my config is loaded. But it doesn't seem to be hit.

	var selectClient = function(): ServerOptions {
		var transport = vscode.workspace.getConfiguration('solargraph').transport;
		console.log('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');
		console.log(transport);
		console.log('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');

Any ideas what I'm doing wrong? Sorry debugging vs code extensions is new to me.

Thanks heaps for adding the feature, really appreciate it! I'm sure I'm just doing something daft.

[castwide]

@RtypeStudios 2 issues I see:

1. The solargraph.transport and solargraph.externalServer configurations belong in the VS Code settings (File -> Preferences -> Settings), not the .solargraph.yml file.

2. The external port for the solar service is 8080 instead of 7658 (the ports setting in your docker-compose configuration). Try changing it to "7658:7658", or changing the solargraph.externalServer port to 8080.

[castwide]

Version 0.18.0 is released with the solargraph.transport and solargraph.externalServer options.

[MilanJ88]

So I run bundle exec solargraph socket on my vm and

"solargraph.transport": "external",
"solargraph.externalServer": {
  "host": [IP of my vm],
  "port": 7658
}

in my vs code settings but I get Connection to server is erroring. Shutting down server. I confirmed that the port is open so i'm pretty much out of ideas on what is going wrong. How can I debug this?

[castwide]

@MilanJansenOGD Are there any more detailed error messages in the developer console (Help -> Toggle Developer Tools)?

[MilanJ88]

I've disabled all other extensions and tried again, nothing comes up in the dev tools console. If I run the check gem version command I get

notificationsAlerts.ts:38 Error: Running the contributed command:'solargraph.checkGemVersion' failed.
	at e._executeContributedCommand (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:460:256)
	at e.$executeContributedCommand (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:460:648)
	at t._doInvokeHandler (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:609:757)
	at t._invokeHandler (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:609:423)
	at t._receiveRequest (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:608:15)
	at t._receiveOneMessage (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:606:957)
	at c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:604:773
	at c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:103:886
	at e.fire (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:105:344)
	at a (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:156:881)
	at Socket._socketDataListener (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\node\extensionHostProcess.js:157:95)
	at emitOne (events.js:116:13)
	at Socket.emit (events.js:211:7)
	at addChunk (_stream_readable.js:263:12)
	at readableAddChunk (_stream_readable.js:250:11)
	at Socket.Readable.push (_stream_readable.js:208:10)
	at Pipe.onread (net.js:594:20)

Which I figure is just because either im not running solargraph locally on windows or that the connection isn't made.

[mvoropaiev]

@castwide first of all, thank you for your work! I was able to get to the point of satisfiable usage of solargraph in docker by doing the following:

• Add solargraph gem into Gemfile:

  group :development do
    gem 'solargraph'
  end
  

• Install all the gems into ./vendor/bundle to be able to access them from vscode (you can probably add additional mount point in Docker setting for another location and keep gems at /usr/local/bundle etc., but I haven't tried that yet, shouldn't also be a problem on Linux)
• Modify compose file configuration for solargraph to report "correct" paths (by default app is located at /app directory) back to vscode (just in case, my entrypoint script makes it bundle exec solargraph --socket...):

  solargraph:
    extends:
      file: docker-compose.common.yml
      service: app
    command: solargraph socket --host=0.0.0.0 --port=7658
    working_dir: $PWD
    environment:
      GEM_HOME: "$PWD/vendor/bundle"
    volumes:
      - .:$PWD:cached,rw
    ports:
     - 7658:7658
  

• Tell solargraph to use external connection:

  "solargraph.transport": "external",
  "solargraph.externalServer": {
      "host": "localhost",
      "port": 7658
  },
  

• Run bundle exec yard gems from within app container

After completing those steps both hovers and go to definition seems to work fine for both local app and installed gems. The only "issue" is that for gems installed with bundler it sometimes takes ~5 seconds to process go to definition, not really sure if that's expected behaviour or maybe something wrong with my setup. Thanks again!

UPDATE: After some testing, it appears that having gems directory mounted to docker on MacOS actually causing some slowdowns (slower application start etc), hopefully there is a way to solve it...

UPDATE2: I think the performance downgrade can be fixed (and even improved) with setting up a volume as NFS share (https://medium.com/@sean.handley/how-to-set-up-docker-for-mac-with-native-nfs-145151458adc),

[sanjibukai]

Hi @mvoropaiev Since I'm a complete noob, do you have a repo with this setup working? Just in order for me to find where to put exactly all that stuff.. Also how do you tell to install the gem in a particular folder? Thank you.

[mvoropaiev]

Hello @sanjibukai as for now I have stayed with running solargraph locally for vscode since even with nfs mount it is still kinda slow on macos (but without mounted gems directory it's actually faster than normal mount), maybe will try to play around with it more in some future as for installing gems in specific folder, I have just override a part of official ruby image (https://github.com/docker-library/ruby/blob/0ad99abb2b6d2df55d7b9b38440dacfe436fe0c3/2.5/alpine3.8/Dockerfile#L109) insde Dockerfile to something like this:

FROM ruby:2.5-alpine

# set up required group, user and directory
ARG WORKER_UID=1000
ARG WORKER_GID=1000
ARG APP_DIR=/usr/src/app
RUN set -ex \
    && delgroup "dialout" \
    && addgroup -g "$WORKER_GID" -S 'worker' \
    && adduser -u "$WORKER_UID" -G 'worker' -S -s '/bin/false' -h "$APP_DIR" 'worker' \
    && mkdir -p "$APP_DIR" \
    && chown -R worker:worker "$APP_DIR"
WORKDIR $APP_DIR

ENV GEM_HOME $APP_DIR/vendor/bundle
ENV BUNDLE_PATH="$GEM_HOME" \
	BUNDLE_SILENCE_ROOT_WARNING=1 \
	BUNDLE_APP_CONFIG="$GEM_HOME"
# path recommendation: https://github.com/bundler/bundler/pull/6469#issuecomment-383235438
ENV PATH $GEM_HOME/bin:$BUNDLE_PATH/gems/bin:$PATH
# adjust permissions of a few directories for running "gem install" as an arbitrary user
RUN mkdir -p "$GEM_HOME" && chmod 777 "$GEM_HOME"
# (BUNDLE_PATH = GEM_HOME, no need to mkdir/chown both)

USER "worker"

# install required gems
COPY --chown=worker:worker Gemfile .
COPY --chown=worker:worker Gemfile.lock .
RUN set -ex \
    && bundle install

# copy application code
COPY --chown=worker:worker . .

# other build steps...

this way, when building docker image normally, gems will end up in project directory inside /usr/src/app/vendor/bundle

and for local development i was using the following for docker-compose.yml

# for skipping bundle install and migrations: skip_init=1 docker-compose up app
version: '2.1'

services:
  app:
    build:
      context: .
      args:
        # id -u
        WORKER_UID: 501
        # id -g
        WORKER_GID: 20
        APP_DIR: $PWD
    working_dir: $PWD
    volumes:
      - .:$PWD
    environment:
      SKIP_INIT: ${skip_init}

# other configuration options...

this way it sets up project directory in container to be the same as your project directory locally and mounts it (as well as having correct file permissions due to worker user having the same ID as my local one)

one thing left is to make entrypoint script do a bundle install on launch

#!/bin/sh
set -e

case $RAILS_ENV in
development)
  bundle config --delete frozen
  [ -z "$SKIP_INIT" ] && bundle install
  ;;
test)
  bundle exec rake db:schema:load --trace
  ;;
staging | production)
  mkdir -p ./public/assets
  mv ./public/assets_tmp/* ./public/assets/
  ;;
*)
  printf 'Unknown RAILS_ENV value "%s"\n' "${RAILS_ENV:-<none>}"
  printf 'Valid values: development|test|staging|production\n'
  exit 1
  ;;
esac


[ -z "$SKIP_INIT" ] && bundle exec rake db:migrate --trace
exec "$@"

Hope that helps, sorry, don't have time to set up a sample project, but please let me know if you need some help!

[dogweather]

I use Docker Compose for my dev environment, and Solargraph running on the host machine with VS Code. I just do a bundle install on my host machine, and install solargraph via gem install.... Works fine.

[mvoropaiev]

@dogweather yeah, I think that's only way to go for now, at least on Mac OS, having two sets of gems is not that bad as poor performance 👍