Sandbox service didn't listen on 8194 port for deploying on kubernetes

[namnh13]

Self Checks

• [X] This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Pleas do not modify this template :) and fill in all the required fields.

Dify version

v0.6.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I deployed sandbox on kubernetes but it listened on random port instead of 8194 port.

2024/04/12 22:06:07 nodejs.go:94: [INFO]nodejs runner environment initialized
2024/04/12 22:06:07 python.go:32: [INFO]initializing python runner environment...
2024/04/12 22:06:07 python.go:45: [INFO]python runner environment initialized
2024/04/12 22:06:07 server.go:18: [INFO]config init success
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

2024/04/12 22:06:07 cocrrent.go:31: [INFO]setting max requests to 50
2024/04/12 22:06:07 cocrrent.go:13: [INFO]setting max workers to 4
[GIN-debug] POST   /v1/sandbox/run           --> github.com/langgenius/dify-sandbox/internal/controller.RunSandboxController (6 handlers)
[GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value.
Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.
[GIN-debug] Listening and serving HTTP on :0

✔️ Expected Behavior

The sandbox listened on 8194 port.

❌ Actual Behavior

The sandbox listened on random port

[dosubot[bot]]

To resolve the issue where the sandbox service is listening on a random port instead of the specified 8194 port when deployed on Kubernetes, you need to explicitly define the port mapping in your docker-compose.yaml file. Here's the adjustment you should make:

  sandbox:
    image: langgenius/dify-sandbox:latest
    restart: always
    cap_add:
      - SYS_ADMIN
    environment:
      API_KEY: dify-sandbox
      GIN_MODE: release
      WORKER_TIMEOUT: 15
    ports:
      - "8194:8194"

By adding the ports section with - "8194:8194", you're directing Docker to map port 8194 of the host to port 8194 of the container, ensuring the sandbox service listens on the correct port. Make sure to apply this change to your docker-compose.yaml file before deploying. If you're converting Docker Compose to Kubernetes objects, remember to re-run the conversion tool after making this modification. This should help your sandbox service listen on the intended port.

Sources

• Sandbox service doesn't listen on 8194 port for deploying on kubernetes
• docker/docker-compose.yaml

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[namnh13]

@dosu-bot I didn't use docker-compose.yaml and I converted the docker-compose.yaml configs to Kubernetes configs but the sandbox listened on random port.

[dosubot[bot]]

To address the issue where your sandbox service is listening on a random port instead of the specified 8194 port when deployed on Kubernetes, you need to ensure that both your Kubernetes service and deployment configurations explicitly specify port 8194. Here's how you can do it:

For the Kubernetes service, define the port mapping like this:

apiVersion: v1
kind: Service
metadata:
  name: sandbox-service
spec:
  selector:
    app: sandbox
  ports:
    - protocol: TCP
      port: 8194
      targetPort: 8194

And for the deployment, ensure it exposes port 8194 in the container specification:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sandbox-deployment
spec:
  selector:
    matchLabels:
      app: sandbox
  template:
    metadata:
      labels:
        app: sandbox
    spec:
      containers:
      - name: sandbox
        image: langgenius/dify-sandbox:latest
        ports:
        - containerPort: 8194

This setup directs traffic from the Kubernetes service to the container's port 8194, ensuring the sandbox service listens on the intended port. Make sure your deployment has the matching label app: sandbox for the selector in your service configuration.

If you're still encountering issues, it might be worth checking for any updates or known issues with your current version of Dify, as newer versions might offer improved configuration options or fixes for port assignment issues.

Sources

• Sandbox service doesn't listen on 8194 port for deploying on kubernetes
• docker/docker-compose.middleware.yaml
• docker/docker-compose.yaml

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[khong07]

It doesn't work @dosu-bot in k8s, it's potential a bug. I use 0.6.3 version and having the same problem. Cannot make it worked in k8s even it works with docker compose. Please make sandbox's port service configured in enviroment variable.

[crazywoola]

You mean this one I guess

https://github.com/langgenius/dify/blob/5e16e7bf5312f86cc1fc6a7224568e520b087415/docker/docker-compose.middleware.yaml#L68

[liujin163]

kubectl exec -it <pod的名字> -n dify apt-get install net-tools netstat -tuln 查看实际暴露的端口，然后修改 k8s service的容器端口就可以用了。 建议最好暴露成环境变量

[namnh13]

@crazywoola Yes. I mean the port configuration of sandbox service. For development environment with docker compose the sandbox listens on 8194 but for the production which we converted docker compose template to kubernetes template and run on GKE, sandbox listened on random port. So Dify cannot make call to sandbox on our production.

[mikck]

I have the same problem

[115341chenzehao]

@crazywoola Yes. I mean the port configuration of sandbox service. For development environment with docker compose the sandbox listens on 8194 but for the production which we converted docker compose template to kubernetes template and run on GKE, sandbox listened on random port. So Dify cannot make call to sandbox on our production.

I'm facing a similar issue with my deployment on Rancher. Could you provide a solution for this?

[haoling]

My solution for kubernetes is "Headless service". The api container request sandbox access for ssrf_proxy, ssrf_proxy might setup sandbox container is originserver of "sandbox". But sandbox main process will listen random port, and kubernetes service need fixed port, its mismatch. My solution for this problem is "Headless service". Headless service has no ip address, return pod address for dns request.

example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sandbox
  namespace: dify
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sandbox
  template:
    metadata:
      labels:
        app: sandbox
    spec:
      containers:
        - name: sandbox
          image: langgenius/dify-sandbox:0.2.0
          imagePullPolicy: IfNotPresent
          envFrom:
            - configMapRef:
                name: sandbox-env
          stdin: true
          tty: true
---

kind: Service
apiVersion: v1
metadata:
  name: sandbox
  namespace: dify
spec:
  clusterIP: None
  selector:
    app: sandbox
---

When deployed these yamls, I seen these logs on ssrf_proxy:

2024-05-23T04:40:47.604054360Z     listening port: 3128
2024/05/23 13:40:45| Configuring Parent sandbox.dify.svc.cluster.local
2024/05/23 13:40:45 pinger| Initialising ICMP pinger ...
2024-05-23T04:40:47.604054360Z 2024/05/23 13:40:45 pinger| ICMP socket opened.
2024/05/23 13:40:45 pinger| ICMPv6 socket opened
2024/05/23 13:40:46| storeLateRelease: released 0 objects

And my workflow that includes code node was successful executed.

P.S. I am japanese, sorry for my not good english.

[RinaisSuper]

i have the same problem

[Yeuoly]

try using SANDBOX_PORT as your environment variable.

[jiachenpang]

  containers:
    - name: sandbox
      image: langgenius/dify-sandbox:0.2.0
      ports:
        - containerPort: 8194
          protocol: TCP
      env:
        - name: SANDBOX_PORT
          value: '8194'
        ......