gvisor icon indicating copy to clipboard operation
gvisor copied to clipboard

gvisor and network=host

Open kosmits-ai opened this issue 11 months ago • 6 comments

Description

Hello,

I'm using gVisor with Docker Compose and have encountered a network-related issue. From what I understand, when you create a user-defined bridge network, Docker sets up a DNS system for the containers. However, containers running with runsc can't access this DNS system due to the sandbox, and as a workaround, I have been using static IPs for communication between services.

The problem arises when I set the network of runsc to host, as my services can then reach external resources like github.com. Without setting network=host, my services can communicate with each other, but they cannot reach external resources like GitHub. Then i get fatal: unable to access 'https://github.com/<github_user>/<repo_name>/': Could not resolve host: github.com.This happens for every repo i try to access.

My question is: if I use network=host, do I lose all the security advantages that gVisor provides? Is there a safer workaround for allowing my containers to access external resources while maintaining gVisor's security benefits?

Thanks in advance for your help!

Steps to reproduce

No response

runsc version

runsc version release-20250113.0
spec: 1.1.0-rc.1

docker version (if using docker)

Docker version 27.5.0, build a187fa5

uname

No response

kubectl (if using Kubernetes)


repo state (if built from source)

No response

runsc debug logs (if available)


kosmits-ai avatar Jan 24 '25 14:01 kosmits-ai

  • if I use network=host, do I lose all the security advantages that gVisor provides?

the short answer is that it will be less secured on networking

please see https://gvisor.dev/docs/user_guide/networking/#network-passthrough, using network=host trades the security and isolation of netstack for the performance of native Linux networking.

  • Is there a safer workaround for allowing my containers to access external resources while maintaining gVisor's security benefits?

I can't reproduce the issue by

$ docker run --runtime=runsc-host --rm -it test git clone --depth 1 https://github.com/google/gvisor.git
Cloning into 'gvisor'...
remote: Enumerating objects: 4199, done.
remote: Counting objects: 100% (4199/4199), done.
remote: Compressing objects: 100% (3829/3829), done.
remote: Total 4199 (delta 958), reused 1739 (delta 289), pack-reused 0 (from 0)
Receiving objects: 100% (4199/4199), 15.96 MiB | 8.50 MiB/s, done.
Resolving deltas: 100% (958/958), done.


# runsc-host in /etc/docker/daemon.json

        "runsc-host": {
            "path": "/tmp/runsc/runsc",
            "runtimeArgs": [
                "--directfs=false",
		"--network=host"
            ]
        },

I would like to see more from your runsc config and docker config

milantracy avatar Jan 25 '25 01:01 milantracy

This is my docker compose.

services:
  mongo-service:
    image: kosmits/thecrimepoirot:mongo-service-1.0.1
    ports:
      - "5000:5000"
    env_file:
      - .env
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${MONGO_SERVICE_IP}
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  gitleaks:
    image: kosmits/thecrimepoirot:gitleaks-1.0.1
    ports:
      - "5001:5001"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${GITLEAKS_IP}
    depends_on:
      - mongo-service
    runtime: runsc
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  guarddog:
    image: kosmits/thecrimepoirot:guarddog-1.0.1
    ports:
      - "5002:5002"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${GUARDDOG_IP}
    depends_on:
      - mongo-service
    runtime: runsc
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  safety:
    image: kosmits/thecrimepoirot:safety-1.0.1
    ports:
      - "5003:5003"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${SAFETY_IP}
    depends_on:
      - mongo-service
    runtime: runsc
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  bearer:
    image: kosmits/thecrimepoirot:bearer-1.0.1
    ports:
      - "5004:5004"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${BEARER_IP}
    runtime: runsc
    depends_on:
      - mongo-service
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  calculate_percentile:
    image: kosmits/thecrimepoirot:calculate_percentile-1.0.1
    ports:
      - "5005:5005"
    env_file:
      - .env
    volumes:
      - ./report.csv:/app/report.csv
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${PERCENTILE_SERVICE_IP}
    depends_on:
      - mongo-service
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  api_gateway:
    image: kosmits/thecrimepoirot:api_gateway-1.0.1
    ports:
      - "5007:5007"
    env_file:
      - .env
    networks:
      crimepoirot_network:
        ipv4_address: ${API_GATEWAY_IP}
    runtime: runc
    depends_on:
      - gitleaks
      - guarddog
      - safety
      - bearer
      - calculate_percentile
    volumes:
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  frontend:
    image: kosmits/thecrimepoirot:frontend-1.0.2
    ports:
      - "8501:8501"
    env_file:
      - .env
    volumes:
      - ./images:/app/images
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${FRONTEND_IP}
    depends_on:
      - api_gateway
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

  update_db:
    image: kosmits/thecrimepoirot:update_db-1.0.1
    ports:
      - "5008:5008"
    env_file:
      - .env
    runtime: runc
    networks:
      crimepoirot_network:
        ipv4_address: ${UPDATE_DB_IP}
    depends_on:
      - mongo-service
      - gitleaks
      - guarddog
      - safety
      - bearer
      - api_gateway
    volumes:
      - ./report.csv:/app/report.csv
      - /home/vboxuser/Documents/RepoForTest:/app/RepoForTest
    dns:
      - 8.8.8.8  # Google DNS
      - 8.8.4.4  # Google DNS

networks:
  crimepoirot_network:
    external: true
    driver: bridge
    ipam:
      config:
        - subnet: ${CRIMEPOIROT_SUBNET}

When i use in docker config /etc/docker/daemon.json:

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc",
            "runtimeArgs": [
                "--network=host"
            ]
       }
    }
}

services communicate each other and can access github.com which is external.

When i use in docker config /etc/docker/daemon.json :

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc"
       }
    }
}

services communicate with each other but cant resolve github.com. [According to gvisor documentation] (https://gvisor.dev/docs/user_guide/faq/#docker-bridge) in the question: My container cannot resolve another container’s name when using Docker user defined bridge, I used static IPs for the communication of containers. So I need to find a way for gvisor to run without network=host for keeping my security in high levels but in the same time the services running in runsc must access github.com

kosmits-ai avatar Jan 25 '25 14:01 kosmits-ai

Also ,defining dns in compose doesnt seem to work for runsc containers.As far as I have understand when you create a define bridge network in docker it creates a DNS which is not accessible from runsc environment.So I thought that i could add DNS 8.8.8.8 in order to give DNS to this services but it did not work

kosmits-ai avatar Jan 25 '25 14:01 kosmits-ai

Any comments/ suggestions?

kosmits-ai avatar Jan 27 '25 20:01 kosmits-ai

how about bind mount /etc/resolv.conf in container? volumes: - /etc/resolv.conf:/etc/resolv.conf:ro

johnwmail avatar Feb 03 '25 07:02 johnwmail

how about bind mount /etc/resolv.conf in container? volumes: - /etc/resolv.conf:/etc/resolv.conf:ro

I've run into similar issues with DNS and Docker Compose. Bind mounting /etc/resolv.conf didn't fix the issue.

I believe this is the underlying issue: https://github.com/google/gvisor/issues/7469

evangineer avatar Mar 21 '25 11:03 evangineer