Issues with proxy and internal networking as containers start up

[carlsonp]

I have what I think could be an opportunity to adjust how the hostnames are created and referenced within ShinyProxy.

We have a company proxy server that requires all outbound network traffic to the wider internet go through the proxy. We also have an external identity provider using OpenID Connect. I've been working to swap our authentication and access over to this new identity provider. I have modified the Docker image and added environment variables as usual for HTTP_PROXY, HTTPS_PROXY, and NO_PROXY. However, I found out that I had to also add them to the startup of ShinyProxy as they didn't seem to be picked up. I added the following:

CMD java -jar \
    -Dhttp.proxyHost=ourproxy.company.com -Dhttp.proxyPort=80 \
    -Dhttps.proxyHost=ourproxy.company.com -Dhttps.proxyPort=80 -Dhttps.proxySet=true \
    -Dhttp.proxySet=true \
    -Dhttp.nonProxyHosts='localhost|localnets|ourcompany.com|127.0.0.1' \
    /opt/shinyproxy/shinyproxy.jar

With this set, I was able to get OpenID Connect and Single-Sign-On (SSO) working.

However, I noticed that I was unable to get Shiny applications to properly start up. I get a message "Failed to start app..." in the UI. However, when I look at docker ps, I see the container start up just fine.

What I suspect is the hostnames for the containers are being incorrectly routed to our company proxy. I don't want the networking to the containers routing through the proxy but I also need some way of referencing them. Instead of the hostname for the container that gets spun up being called: 9f576925a7f8, maybe it should be given some kind of common ending like: 9f576925a7f8.local?

This way I could add *.local to NO_PROXY. Without this, I'm not sure how I can configure both SSO and get ShinyProxy working properly.

When I turn up logging I see messages like this:

2024-12-16T22:41:59.080Z DEBUG 7 --- [ProxyService-16] s.n.www.protocol.http.HttpURLConnection  : sun.net.www.MessageHeader@2fd0c3bf5 pairs: {null: HTTP/1.0 404 Not Found}{Content-Type: text/plain; charset=UTF8}{Server: BigIP}{Connection: close}{Content-Length: 66}
2024-12-16T22:41:59.482Z DEBUG 7 --- [ProxyService-16] s.n.www.protocol.http.HttpURLConnection  : sun.net.www.MessageHeader@541028535 pairs: {GET http://9f576925a7f8:3838/ HTTP/1.1: null}{User-Agent: Java/17.0.11}{Host: 9f576925a7f8:3838}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Proxy-Connection: keep-alive}

This again makes me suspect networking issues between ShinyProxy and the containers it is starting up.

We have a production server with the exact same setup but a different internal identity provider and that doesn't have the override for the CMD startup and that has no issues.

Does anyone have any further ideas or thoughts? Thanks.

ShinyProxy version: 3.1.1

[billy34]

You can control the naming of the spun container by defining resource-name attribute in the app spec definition. It uses SpEL expressions. Here for example I build the container name from it's app id and the first 8 digits of the container id. resource-name: "#{proxySpec.id}-#{proxy.id.substring(0,8)}"

Maybe look at this to append .local to the default naming (sp-container-#{proxy.id}-0) resource-name: "sp-container-#{proxy.id}-0.local"

[carlsonp]

I tried your suggestion but sadly, it didn't seem to work. The HttpConnection is still using just the ID when it makes the HTTP GET check.

2025-01-09T17:16:11.756Z DEBUG 7 --- [ProxyService-16] s.n.www.protocol.http.HttpURLConnection  : sun.net.www.MessageHeader@4e1df9c85 pairs: {GET http://8cc17fef08f0:3838/ HTTP/1.1: null}{User-Agent: Java/17.0.11}{Host: 8cc17fef08f0:3838}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Proxy-Connection: keep-alive}

Perhaps whatever is being passed into getTargets should be adjusted or add the resource name value in addition to the container ID to check?

I can see the changes I made using the new resource-name in the name of the container that it tries to start up.

I also tested running a ping command from within ShinyProxy. I can see the container try to start up and I can also successfully ping the container before it gets killed off during the startup process. This to me shows I have the no-proxy settings correct as it doesn't appear to be a networking issue between the containers.

[LEDfan]

Hi, currently there is no way in ShinyProxy for this to work, since this is related to docker networking, I don't think we can easily add .local here.

We could modify ShinyProxy to contact the containers using the IP address instead of the hostname, this might have some other benefits as well. You could test this yourselves by modifying this line https://github.com/openanalytics/containerproxy/blob/9f1c7964b8ab23ad19f0a4dade27e25ff9e122ff/src/main/java/eu/openanalytics/containerproxy/backend/docker/DockerEngineBackend.java#L211

to:

targetHostName = info.networkSettings().networks().get(info.hostConfig().networkMode()).ipAddress();

Next you have to add the IP range of your docker network to the no-proxy option. Please let me know if this works. I'll have a look at the impact of this change and then maybe implement it into the next version.

[LEDfan]

Hi

I looked into having ShinyProxy use the IP address of the container, but this also introduces some security issues. Therefore we cannot implement this. I also searched a lot to use some suffix for the DNS name, but I didn't find a way to implement this.

As a last resort, I can try to check whether it's possible for ShinyProxy to ignore the proxy settings while sending requests to the app. I'll keep this issue open until we have looked at this.

[carlsonp]

Thanks, we haven't been able to do any testing or work on this yet, sorry. It is preventing us from implementing SSO which is not the end of the world but would be helpful for users. Ignoring the proxy might work but we would need some way to not use it for local connectivity to spun up containers but then leverage the proxy when it connects to the external Open ID Connect Identity Provider.

[carlsonp]

What about using other tools or services to help this? Would oauth2-proxy for example be able to handle the single-sign-on (SSO) and connectivity to OIDC and then just pass further network traffic to Shiny Proxy? Can Shiny Proxy leverage X-Forwarded-Access-Token for example?

[carlsonp]

I was able to deploy oauth2-proxy and leverage the Shiny Proxy header based authentication with it to examine and use the header values from oauth2-proxy to validate group membership. It seems to be running so we at least have a work-around at the moment. Thanks again for the help and looking into this. Doing it natively in Shiny Proxy is preferred, however, the proxy adds a whole other layer of complexity that we often have to deal with. I'm not surprised all this required some extra work.

I'll leave this up to you if you want to close the issue or keep it open.

[LEDfan]

Hi I'm happy to hear you have found a workaround. I'll leave this ticket open, since it would be good to be able to disable the proxy settings for connecting to the apps. I created an internal ticket for it, and we'll have a look to implement this in a future release.

BTW if you wish it could be useful to share your oauth2-proxy configuration, I think this would be useful for other community members. But no worries if it's not possible.

[carlsonp]

Most of the setup is in setting up the oauth2-proxy service itself. We slightly customized the oauth2-proxy:latest-alpine image for our needs. In our docker-compose.yml we have the following:

oauth2-proxy:
    build:
      context: ./oauth2-proxy/
    image: youroauth2proxyimage:latest
    hostname: oauth2-proxy
    environment:
      OAUTH2_PROXY_REVERSE_PROXY: "true"
      OAUTH2_PROXY_PROVIDER: "entra-id"
      OAUTH2_PROXY_LOG_LEVEL: "debug"
      OAUTH2_PROXY_BANNER: "Shiny Proxy SSO"
      OAUTH2_PROXY_CUSTOM_SIGN_IN_LOGO: "image.png"
      OAUTH2_PROXY_FOOTER: "-" # - disables the footer which has the version information
      OAUTH2_PROXY_SET_AUTHORIZATION_HEADER: "true"
      OAUTH2_PROXY_CLIENT_ID: ${OAUTH2_PROXY_CLIENT_ID}
      OAUTH2_PROXY_AUTH_URL: "https://login.microsoftonline.com/<idhere>/oauth2/authorize"
      OAUTH2_PROXY_OIDC_ISSUER_URL: "https://login.microsoftonline.com/<idhere>/v2.0"
      OAUTH2_PROXY_COOKIE_NAME: "_oauth2_proxy"
      OAUTH2_PROXY_COOKIE_SECRET: ${OAUTH2_PROXY_COOKIE_SECRET}
      OAUTH2_PROXY_CLIENT_SECRET: ${OAUTH2_PROXY_CLIENT_SECRET}
      OAUTH2_PROXY_ENABLE_HTTPS: "true"
      OAUTH2_PROXY_TLS_CERT_FILE: "/etc/oauth2-proxy/tls.crt" # Path to the TLS certificate
      OAUTH2_PROXY_TLS_KEY_FILE: "/etc/oauth2-proxy/tls.key" # Path to the TLS key
      OAUTH2_PROXY_EMAIL_DOMAINS: "domain.com"
      # https://oauth2-proxy.github.io/oauth2-proxy/configuration/providers/ms_entra_id#example-configurations
      OAUTH2_PROXY_SCOPE: "openid"
      # The upstream URL of the application that the proxy is protecting
      OAUTH2_PROXY_UPSTREAMS: "http://shinyproxycontainer:8080/"
      OAUTH2_PROXY_HTTP_ADDRESS: "0.0.0.0:4180"
      OAUTH2_PROXY_PASS_USER_HEADERS: "true"
    ports:
      - 443:443
    healthcheck:
      test: curl -k --fail https://localhost/ping || exit 1
    restart: unless-stopped

All the oauth2-proxy settings can be set via environment variables which is very convenient. Secrets and other details get set in a .env file which then is read in when docker-compose runs. In our case, we used Azure EntraID for OAuth2 and OpenID Connect capabilities.

We also have Nginx setup to just port forward requests from HTTP to HTTPS.

On the application.yml shinyproxy config side we have the following:

  # https://www.shinyproxy.io/documentation/configuration/#custom-header-based-authentication
  # header based authentication
  authentication: customHeader
  custom-header:
    username-header-name: X-Forwarded-Email
    groups-header-name: X-Forwarded-Groups

Hope it's helpful for someone.

[LEDfan]

Thanks for sharing, this is definitely useful!