docker run -it

[matti]

Any change to get that working?

[djenriquez]

@matti

Yes, this will work, you just need to override the entrypoint for the image:

docker run -it --rm --entrypoint /bin/bash djenriquez/sherpa

^ will get you into the container with bash. Or if the container is already running:

docker exec -it <CONTAINER ID> bash

[matti]

No, I meant when I'm using sherpa as DOCKER_HOST

my machine --> sherpa --> dockerd

[matti]

docker ps etc works, but running for example DOCKER_HOST=sherpasip:4550 docker run -it alpine sh won't work, it will just hang.

[djenriquez]

@matti how are you running sherpa?

The ability for sherpa to allow access depends on if you're running --allow or --deny mode, and the combination of configuration added. If you're running --deny mode, which is the default mode, you must explicitly define the HTTP methods allowed for the endpoints.

Can you post the run command you're using? Feel free to redact private information.

[matti]

^- above I said docker ps works..

Anyhow, here's a way to repro:

$ docker-machine start

$ docker-machine env
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/Users/mpa/.docker/machine/machines/default"
export DOCKER_MACHINE_NAME="default"
# Run this command to configure your shell:
# eval $(docker-machine env)

$ eval $(docker-machine env)
docker run -d \
> --name sherpa \
> -v /var/run/docker.sock:/tmp/docker.sock \
> -p 4550:4550 \
> djenriquez/sherpa --allow
Unable to find image 'djenriquez/sherpa:latest' locally
latest: Pulling from djenriquez/sherpa
43c265008fae: Pull complete
aa9fe9b2de70: Pull complete
de9a4fb4cf68: Pull complete
e616be8cf129: Pull complete
dc49c7f081ed: Pull complete
5c5f528fec5b: Pull complete
4b263b62ecad: Pull complete
8439c935f4ca: Pull complete
0fce8a835cac: Pull complete
99f16fdbd26c: Pull complete
Digest: sha256:c7f7ee08f50143979467857f0a9ff9dcc06da3a98ac09818e76304fa5c02a977
Status: Downloaded newer image for djenriquez/sherpa:latest
19cfb8c8dbc0f73a672be201be3ed24daa625e4e0b505343395f9d013fb73904

$ DOCKER_HOST=tcp://192.168.99.100:4550 docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES
19cfb8c8dbc0        djenriquez/sherpa   "./main --allow"    35 seconds ago      Up 34 seconds       0.0.0.0:4550->4550/tcp   sherpa

$ DOCKER_HOST=tcp://192.168.99.100:4550 docker run -it alpine sh
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
627beaf3eaaf: Pull complete
Digest: sha256:58e1a1bb75db1b5a24a462dd5e2915277ea06438c3f105138f97eb53149673c4
Status: Downloaded newer image for alpine:latest



lsslslsldfpl
HANGED
^C

$

[matti]

I'm assuming that this is due to docker protocol where the docker run output is attached to the HTTP body in non-standard HTTP way.

[djenriquez]

interesting, ok cool. it'll be later in the day when I can look at this, but i'll definitely check it out

[matti]

I think the issue is here: https://docs.docker.com/engine/api/v1.26/#operation/ContainerAttach

The logs start to stream and the stream is not handled, see:

POST /containers/16253994b7c4/attach?stream=1&stdout=1 HTTP/1.1
Upgrade: tcp
Connection: Upgrade
The Docker daemon will respond with a 101 UPGRADED response, and will similarly follow with the raw stream:

HTTP/1.1 101 UPGRADED
Content-Type: application/vnd.docker.raw-stream
Connection: Upgrade
Upgrade: tcp

[STREAM]
Stream format
When the TTY setting is disabled in POST /containers/create, the stream over the hijacked connected is multiplexed to separate out stdout and stderr. The stream consists of a series of frames, each containing a header and a payload.

The header contains the information which the stream writes (stdout or stderr). It also contains the size of the associated frame encoded in the last four bytes (uint32).

It is encoded on the first eight bytes like this:

header := [8]byte{STREAM_TYPE, 0, 0, 0, SIZE1, SIZE2, SIZE3, SIZE4}
STREAM_TYPE can be:

0: stdin (is written on stdout)
1: stdout
2: stderr
SIZE1, SIZE2, SIZE3, SIZE4 are the four bytes of the uint32 size encoded as big endian.

Following the header is the payload, which is the specified number of bytes of STREAM_TYPE.

The simplest way to implement this protocol is the following:

Read 8 bytes.
Choose stdout or stderr depending on the first byte.
Extract the frame size from the last four bytes.
Read the extracted size and output it on the correct output.
Goto 1.

Stream format when using a TTY
When the TTY setting is enabled in POST /containers/create, the stream is not multiplexed. The data exchanged over the hijacked connection is simply the raw data from the process PTY and client's stdin.

[djenriquez]

Hmm, I see. I read through the docs and quite frankly I'm not sure how to implement a fix for this. Sherpa supports all of the RESTful commands that the API offers, but attach uses hijacking that appears to require more app-level manipulation than what NGINX can offer. Not saying this is impossible, but it's not something I'd be able to investigate or work on in the near future.

That being said, if you or anyone wants to dig into the project and submit PRs, please feel free.