FR: Support granular API permissions on a per host basis via allowfrom

[nathang21]

Describe the feature request Naturally many services require access to the docket socket, and while socket proxy is an excellent solution, to achieve minimal permissions for each each service would require a seperate sidecar for each. Multiple hostname support in #15 is great, but they all have to share the same permissions. While socket-proxy is very lightweight, i'm not a huge fan of running this many sidecar containers in general.

In my case I have 8 services, and would need 8 sidecars, some of these are read-only, while others also write. If interested, here are the 8 I use now: ofelia, dozzle, watchtower, autoheal, beszel, boinc, dockwatch, portainer.

I realize supporting this would add complexity, but my ideal solution would be to have allow a separate configuration for each rest API Method on a per host (ip/hostname) basis. I suspect adjusting the configuration via environment variables or an configuration file would be required to make this maintainable.

I could imagine something like this:

allowfrom:
  - dozzle:
      allowGET=
  - ofelia:
      allowGET=
      allowPOST=
      allowDELETE=
...

Are you open to supporting this use case?

[wollomatic]

Hi @nathang21,

thanks for Your suggestion. Indeed, using one instance of socket-proxy for multiple services is a popular topic.

A yaml file would allow configuring this without breaking existing installations. Just give me a few days :)

Also, thanks for the list of the services you'd like to use. I only get to know how others use socket-proxy through feedback. So this is very important to me.

[nathang21]

Fantastic, thanks for the quick turnaround. Happy to help test when the time comes if needed.

If it helps, I'm also considering to add this soon, which has a native docker service discovery integration that requires the socket as well. https://github.com/gethomepage/homepage

[wollomatic]

Thanks for the additional information. BTW, I tested homepage by myself because of a question here: https://github.com/gethomepage/homepage/discussions/3266#discussioncomment-9239716

[nathang21]

Thanks for sharing the homepage requirements!

Just wanted to check back in, looking forward to harding my services with this.

[wollomatic]

Sorry for the delayed answer. I plan to get it done this month.

[Marshalldeluna]

Any chance this could still be added I would prefer to run just 1 instance of the socket-proxy

[wollomatic]

Well, it seems to be a very long month, but yes

[chrishoage]

Since this project by its very nature requires access to the raw socket exposed by Docker it could be interesting to instead use a similar approach to Traefik and use labels to control configuration.

One could imagine a label or set of labels applied to a service that could be read by socket-proxy.

These labels would inform socket-proxy as to what containers should be allowed which requests. By extension if a container does not have the required labels that container is denied access even if it is on the same internal docker network as socket-proxy.

Again relying on the information already available to socket-proxy through the docker socket it could restrict access via IP address gained through inspecting the container via the docker API. This could be further restricted through labels if necessary (but I could see this being unneeded complexity)

This approach could be a clean solution that wouldn't necessarily introduce any new configuration files and would also not require maintaining the configuration file for new services as they appear.

Of course a configuration file still may be prudent since these labels must be transformed into runtime configuration objects anyway.

[amanda-wee]

@wollomatic would you be open to @chrishoage's suggestion of using labels? I was intrigued by the idea and have a proof of concept running, though I'm still working on automatically updating the configuration when containers are started after the socket proxy has started by checking for the relevant Docker events. I expect that you'll want to retain -allowfrom with hostnames, but it effectively does replace that.

[amanda-wee]

Here's my proof of concept: https://github.com/amanda-wee/socket-proxy/tree/per-container-allowlists

I added a dependency on the Docker API SDK for Go. If necessary, it should be possible to rework the code to do without that dependency, but before that I would like some feedback about whether the labels idea is acceptable.

[edekeijzer]

Here's my proof of concept: https://github.com/amanda-wee/socket-proxy/tree/per-container-allowlists

I added a dependency on the Docker API SDK for Go. If necessary, it should be possible to rework the code to do without that dependency, but before that I would like some feedback about whether the labels idea is acceptable.

@amanda-wee I have built your fork and are trying to test it, but I do not get it running with -proxycontainername, it will tell me the container cannot be found. (it should point to its own container name to retrieve its network, right?)

[amanda-wee]

@edekeijzer yes, I have been testing with a name explicitly specified in compose.yaml as container_name: beszel-socket-proxy, and just tried with leaving it out and using the service name (socket-proxy, or dockerproxy in the examples) and it seems to work too, as did using the name printed when starting the container (beszel-socket-proxy-1 in my case as I was running it under the beszel project).

EDIT: oh, okay. I think I understand what's happening once I add in a bit more logging on the config initialisation: it looks like when the socket-proxy container starts, the Docker API might not have its data ready yet, and so it returns no matching container summaries, which in turn stops the container as I treat that as an error, and then my compose restarts it... and this time it does have the data ready, so it returns it correctly. I didn't notice because I had my focus on the network names rather than on the early error message. I've pushed a commit that adds retry logic to that part.

[edekeijzer]

@amanda-wee your conclusion seems correct, it didn't start most of the time but sometimes it did. Rebuilt with your latest commit and it starts correctly every time now. Anything you would like to have tested? I've put Watchtower behind it and will do Traefik later today.

[wollomatic]

@amanda-wee, thank you very much for your contribution — I really appreciate it! Also, thanks @edekeijzer for testing. I’m a bit under the weather with the flu right now, so I’ll get back to you in a few days.

[amanda-wee]

@edekeijzer thanks for testing. I guess the core is checking that the per-container allowlists come into effect correctly for the given containers, whether the socket proxy starts before or after these containers.

In my own testing with Beszel, I ran into the problem that Beszel attempted to access the Docker API to check Docker version before its allowlist came into effect, so that query got blocked. Unfortunately, it only ran once, so subsequent Docker API access attempts failed as they depended on the version info. I think this needed to be fixed in Beszel, and the maintainer agreed, but it's possible that this kind of assumption of success could be a drawback of the label processing. In Beszel's case, using the default allowlist to allow version info queries is a reasonable workaround, so it was not a dealbreaker.

@wollomatic no worries, I'm travelling until the weekend so I'm just waiting for you and interested socket-proxy users to provide feedback. I guess the important decisions from you would be whether you even want to implement the per-container allowlists in this way, and if so, if you want to keep the Docker Go SDK client and types dependency.

[amanda-wee]

@wollomatic I have created another branch from my per-container-allowlists branch for investigating integrating the parts of the Docker SDK for Go needed to get the functionality to work. See: https://github.com/amanda-wee/socket-proxy/tree/per-container-allowlists-integrate-sdk

This would eliminate the dependency, and is structured to reduce the difficulty of porting over future improvements and bug fixes to that SDK.

@edekeijzer did your testing uncover any problems that I should fix?

Thanks!

[edekeijzer]

@amanda-wee it's been running with Traefik, Watchtower and Gitea ACT Runner for two weeks now and I haven't seen any issues. What would be nice to have, is adding the generic permissions to the granular permissions, I have to add things like get for info, ping or version and allow.head \/.* to each container. Another nice to have would be a log for 'learning', so you'd either set the entire socket-proxy to learning, or add all permissions to a container and then show a deduplicated list of all endpoints that an IP address has used. (this is not specific to your label implementation, it would make implementing the proxy least-privileged much easier)

[amanda-wee]

it's been running with Traefik, Watchtower and Gitea ACT Runner for two weeks now and I haven't seen any issues.

Awesome. I haven't seen issues with Beszel either, also when testing with the integrated Docker client.

What would be nice to have, is adding the generic permissions to the granular permissions, I have to add things like get for info, ping or version and allow.head /.* to each container.

This should be a separate feature, methinks. I imagine there could be an allowed request "minimal preset" like what you mentioned that could be built on, and a "readonly preset" that would be more restrictive than -allowGET=.* (basically the idea of 11notes/docker-socket-proxy). Of course, there's also the argument that maybe wollomatic/socket-proxy's wiki just needs more examples.

Another nice to have would be a log for 'learning', so you'd either set the entire socket-proxy to learning, or add all permissions to a container and then show a deduplicated list of all endpoints that an IP address has used.

You can already do this by setting the log level to debug as successful requests will get logged at debug level with method, path, and IP address, just that they aren't deduplicated.

[amanda-wee]

I've made some optimisations to reduce the number of Docker API calls and data received from the Docker API, and merged the whole Docker SDK integration into my original per-container-allowlists branch to go back to the claim in the README of socket-proxy being "without any external dependencies".

@wollomatic It will be quite a big pull request in terms of number of files and the changes to config.go, but if you have no objections, I'll create a pull request and see how it goes from there.

[amanda-wee]

@wollomatic great, here's the pull request! https://github.com/wollomatic/socket-proxy/pull/69