nomad icon indicating copy to clipboard operation
nomad copied to clipboard

Nomad connect functionality not working with SELinux enabled

Open apollo13 opened this issue 5 years ago • 11 comments

Nomad version

Nomad v0.10.4 (f750636ca68e17dcd2445c1ab9c5a34f9ac69345)

Operating system and Environment details

Fedora 31, with 18.09.8

Issue

The envoy health check in consul stays red and var/log/audit/audit.log contains denials:

type=AVC msg=audit(1583672022.178:2020): avc:  denied  { write } for  pid=70868 comm="envoy" name="consul_grpc.sock" dev="tmpfs" ino=676989 scontext=system_u:system_r:container_t:s0:c121,c146 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=sock_file permissive=0

After sentenforce 0 the health check turns green.

Reproduction steps

Run nomad agent -dev and consul agent -dev and deploy the job file from below

Job file (if appropriate)

job "example" {
	datacenters = ["dc1"]
	type = "service"
	update { max_parallel = 1 }
	group "http1" {
		network {
			mode = "bridge"
			port "http" { to=80 }
		}
                service {
                        port = "http"
                        name = "http1"

			connect {
				sidecar_service {}
			}
                }
		task "http1" {
			driver = "docker"
			config { image = "nginx" }
		}
	}
	group "http2" {
		network {
			mode = "bridge"
			port "http" { to=80 }
		}
		service {
			port = "http"
			name = "http2"

		      connect {
			sidecar_service {
			  proxy {
			    upstreams {
			      destination_name = "http1"
			      local_bind_port  = 8080 
			    }
			  }
			}
		      }

		}
		task "http2" {
			driver = "docker"
			config { image = "nginx" }
		}
	}
}

Consul logs have:

    2020-03-08T14:02:56.962+0100 [WARN]  agent: Check socket connection failed: check=service:_nomad-task-6d05e4c5-b5d8-2941-c6a4-dc9bb1e675c6-group-http2-http2-http-sidecar-proxy:1 error="dial tcp 127.0.0.1:30124: connect: connection refused"
    2020-03-08T14:02:56.963+0100 [WARN]  agent: Check is now critical: check=service:_nomad-task-6d05e4c5-b5d8-2941-c6a4-dc9bb1e675c6-group-http2-http2-http-sidecar-proxy:1
    2020-03-08T14:03:00.781+0100 [WARN]  agent: Check socket connection failed: check=service:_nomad-task-a6ae689e-6b3c-206d-5b58-0562248a595c-group-http1-http1-http-sidecar-proxy:1 error="dial tcp 127.0.0.1:20423: connect: connection refused"
    2020-03-08T14:03:00.781+0100 [WARN]  agent: Check is now critical: check=service:_nomad-task-a6ae689e-6b3c-206d-5b58-0562248a595c-group-http1-http1-http-sidecar-proxy:1

The other logs do not contain anything interesting sadly.

apollo13 avatar Mar 08 '20 13:03 apollo13