Launch container image with read-only filesystem

[abrownfi]

Q&A (please complete the following information)

• OS: Fedora 30, Centos 7, probably most others
• Browser: Any
• Version: Any
• Method of installation: docker image pull
• Swagger-UI version: 3.24.2
• Swagger/OpenAPI version: N/A

Content & configuration

Example Swagger/OpenAPI definition: N/A Swagger-UI configuration options: default

Describe the bug you're encountering

Cannot launch container image when filesystem is read-only. This problem was initially detected in a Kubernetes cluster where the default PodSecurityPolicy has readOnlyRootFilesystem set as true. Can be reproduced directly with docker.

To reproduce...

docker run --publish 8080:8080 --rm --read-only docker.io/swaggerapi/swagger-ui:v3.24.2

Expected behavior

Able to launch with a readonly filesystem.

Additional context or thoughts

[abrownfi]

The simple approach in this commit allows the application to launch. Being new to swagger-ui, it's not clear what side-effects to look for, but a rough check shows the functionality working.

The docker command is more complex to appease the nginx cache directories.

docker run \
	--env READONLY_FILESYSTEM="true" \
	--publish 8889:8080 \
	--rm \
	--tmpfs /var/cache/nginx/client_temp \
	--tmpfs /var/cache/nginx/proxy_temp \
	--tmpfs /var/cache/nginx/fastcgi_temp \
	--tmpfs /var/cache/nginx/uwsgi_temp \
	--tmpfs /var/cache/nginx/scgi_temp \
	--volume /tmp/nginx_tmp:/var/run \
	--read-only \
	swagger-ui:localbuild

[webron]

@abrownfi Please consider submitting a PR with the changes and we'll review them.

[robdesbois]

This issue has languished a bit, so I rebased the changes and filed a PR. Needs a workflow approval from a maintainer to continue, and a review.

[robdesbois]

How can I get a reviewer for this PR?

[char0n]

Hi @robdesbois,

I understand your requirement of having a an image running on read-only filesystem. Let's try to understand how the changes in your PR change the behavior of the image.

[robdesbois]

Hi @char0n, were you asking me to clarify something?

[bluebrown]

I am also interested in rooing this with readonly rootfs

[astarche]

There are a few ways to workaround this. I'll do examples in docker-compose with read_only: true. In kubernetes, tmpfs is replaced with emptyDir, static files are replaced with config maps, and depends_on containers are replaced with initContainers.

There are 3 types of files that are modified at runtime:

1. /var/cache/nginx and /var/run: "temp"/cache files
2. /etc/nginx/conf.d: generated nginx config
3. /usr/share/nginx/html: swagger-initializer.js and pre-compressed *.gz files.

(3) is the most challenging because the base image has files in /usr/share/nginx/html. We can't just mount an empty temp dir there.

I'm always using the official swaggerapi/swagger-ui docker image. There are more options with a custom Dockerfile, but I think they follow the same concepts.

Examples: docker-compose-read-only-swagger.zip

OPTION 1: docker-compose-mutable-content.yml

The straightforward fix is to create an init container that copies the original content to a shared tmp volume that the main container mounts as writable:

version: "3.8"
services:
  # Copy the original content to a shared volume that can be mounted as writable.
  copy-content-to-mutable-temp-volume:
    image: swaggerapi/swagger-ui
    read_only: true
    command: ["cp", "-r", "/usr/share/nginx/html", "/mutable-content"]
    volumes:
      - mutable-content:/mutable-content

  # Mounts content as mutable volume, create tmpfs for volumes populated at ini, run as normal.
  swagger-ui:
    depends_on:
      copy-content-to-mutable-temp-volume:
        condition: service_completed_successfully
    ports:
      - 8080:8080
    environment:
      URLS: "[{ url: \"https://my-api.my-company.com/service1/swagger/v1/service1.json\", name: \"Service 1\" },
              { url: \"https://my-api.my-company.com/service2/swagger/v1/service2.json\", name: \"Service 2\" }]"
    image: swaggerapi/swagger-ui
    restart: always
    read_only: true
    volumes:
      # Content must be mutable because /usr/share/nginx/html is modified at init time.
      - mutable-content:/usr/share/nginx/html
      - type: tmpfs
        target: /etc/nginx/conf.d
      - type: tmpfs
        target: /var/cache/nginx
      - type: tmpfs
        target: /var/run

volumes:
  mutable-content:

Note that in Kubernetes mutable-content can be an emptyDir volume that is shared between the init container and run-time container (when the init container is done, it is not "empty" anymore).

This is easy and effective. The "negative" is that it leaves several important directories writable at run-time, even though they are only changed during initialization.

OPTION 2: docker-compose-static-initializer.yml

A fussier fix is to pre-generate the swagger-initializer.js file and then short-circuit the run-time initialization. This is similar to @abrownfi 's approach:

version: "3.8"
services:
  # Force the server to use a static-swagger-initializer.js
  swagger-ui-static-initializer:
    ports:
      - 8080:8080
    # Run-time image does not need special environment variables (in our config) because
    # the URLs are included in the static-swagger-initializer.js. 
    environment: {}
    image: swaggerapi/swagger-ui
    restart: always
    read_only: true
    volumes:
      # Dynamic nginx config generated to tmpfs/emptyDir volume.
      - type: tmpfs
        target: /etc/nginx/conf.d
      # Caches use tmpfs/emptyDir volumes.
      - type: tmpfs
        target: /var/cache/nginx
      - type: tmpfs
        target: /var/run
      # "Static" swagger-initializer.js (generated by helm templates?) without the start/end
      # markers will short-circuit modifications in the configurator script (and print an error message).
      # NOTE: this solution will also fail to gzip the content, but is still functional.
      - ./static-swagger-initializer.js:/usr/share/nginx/html/swagger-initializer.js
      # ALTERNATIVE: specific init steps can be skipped by mounting a no-op script.
      # - ./no-op-init.sh:/docker-entrypoint.d/40-swagger-ui.sh
      # ALTERNATIVE: Patching a no-op configurator disables MOST edits to swagger-initializer.js while preserving nginx conf.
      # - ./no-op-configurator.js:/usr/share/nginx/configurator/index.js

• This has several non-fatal errors at runtime.
• This would be a good solution if the code explicitly supported it. But it would be hard to support all of the configuration options. Especially if it was extended to include a static nginx config.

OPTION 3: docker-compose-configurator.yml

This is similar to OPTION 1 but allows both of the generated/configurated directories to be mounted read-only by adding a second init container that runs configuration and copies to shared volumes:

version: "3.8"
# All services have read_only root filesystems and use shared volumes for writable content.
services:
  # Copy the original content to a shared volume that can be mounted as writable.
  copy-content:
    image: swaggerapi/swagger-ui
    read_only: true
    command: ["cp", "-r", "/usr/share/nginx/html", "/original-image-content"]
    volumes:
      - original-image-content:/original-image-content

  # Runs the nginx/swaggerui config steps and copies data to shared volumes.
  configurator:
    depends_on:
      copy-content:
        condition: service_completed_successfully
    image: swaggerapi/swagger-ui
    environment:
      URLS: "[{ url: \"https://my-api.my-company.com/service1/swagger/v1/service1.json\", name: \"Service 1\" },
              { url: \"https://my-api.my-company.com/service2/swagger/v1/service2.json\", name: \"Service 2\" }]"
    read_only: true
    entrypoint: ["/bin/sh", "-c"]
    # Run docker-entrypoint script to configure nginx and content.
    # It requires that the first argument be "nginx", so use "-t" to test the configuration and exit.
    # Then copy directories that are modified by docker-entrypoint.sh to shared volumes for the run-time image.
    command: ["/docker-entrypoint.sh nginx -t && cp -r /usr/share/nginx/html/* /configurated-content && cp -r /etc/nginx/conf.d/* /configurated-nginx"]
    volumes:
      - ./configurator.sh:/configurator.sh
      # Mount original-image-content over the image's content directory so that we can modify it in a read_only container.
      - original-image-content:/usr/share/nginx/html/
      - type: tmpfs
        target: /etc/nginx/conf.d
      - configurated-content:/configurated-content
      - configurated-nginx:/configurated-nginx
      # Caches used by "nginx -t" need writable tmpfs (or emptyDir) volumes.
      - type: tmpfs
        target: /var/run
      - type: tmpfs
        target: /var/cache/nginx

  # Mounts configurated content as "read-only", skips configurator steps, and runs nginx.
  swagger-ui:
    depends_on:
      configurator:
        condition: service_completed_successfully
    ports:
      - 8080:8080
    # Run-time image does not need special environment variables.
    environment: {}
    image: swaggerapi/swagger-ui
    restart: always
    read_only: true
    # Override the default "docker-entrypoint.sh" to skip configuration.
    entrypoint: ["/bin/sh", "-c"]
    command: ["nginx -g 'daemon off;'"]
    volumes:
      # NOTE: content and nginx config are read-only in the run-time image!
      - configurated-content:/usr/share/nginx/html/:ro
      - configurated-nginx:/etc/nginx/conf.d/:ro
      # Caches still need writable tmpfs (or emptyDir) volumes.
      - type: tmpfs
        target: /var/cache/nginx
      - type: tmpfs
        target: /var/run

volumes:
  original-image-content:
  configurated-content:
  configurated-nginx:

This maximizes "read-only" directories. But it's too complex and dependent on internals.

[CameronGo]

This is an important feature for anyone running in a regulated environment, so I'm eager to see if there's a viable way of supporting this without some of the complicated workarounds.

[imalik8088]

Solution for Kubernetes users:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: swagger-ui
spec:
  replicas: 1
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 101
        fsGroup: 101
      # InitContainer copies static files to shared volume for readOnlyRootFilesystem compatibility
      initContainers:
        - name: init
          image: swaggerapi/swagger-ui:latest
          command: [ "sh", "-c", "cp -r /usr/share/nginx/html/* /tmp/share/nginx/html/" ]
          volumeMounts:
            - name: nginx-html
              mountPath: /tmp/share/nginx/html/
      containers:
        - name: main
          terminationMessagePolicy: FallbackToLogsOnError
          securityContext:
            capabilities:
              drop:
                - ALL
            privileged: false
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 101
          image: swaggerapi/swagger-ui:latest
          envFrom:
            - configMapRef:
                name: swagger-ui
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /
              port: http
          # no explicit health endpoint in swagger-ui, check main web ui
          readinessProbe:
            httpGet:
              path: /
              port: http
          volumeMounts:
            - name: nginx-html
              mountPath: /usr/share/nginx/html
            - name: nginx-conf
              mountPath: /etc/nginx/conf.d
            - name: nginx-cache
              mountPath: /var/cache/nginx
            - name: nginx-run
              mountPath: /var/run
            - name: tmp
              mountPath: /tmp
          resources:
            requests:
              cpu: "1m"
              memory: "50Mi"
            limits:
              memory: "50Mi"
      volumes:
        - name: nginx-html
          emptyDir: {}
        - name: nginx-conf
          emptyDir: {}
        - name: nginx-cache
          emptyDir: {}
        - name: nginx-run
          emptyDir: {}
        - name: tmp
          emptyDir: {}