traefik Internal services incorrectly require a port to be defined

Bug

What did you do?

I migrated the following traefik from a bridge network to a macvlan one

What did you expect to see?

The dashboard should have been available under traefik.mydomain.com

What did you see instead?

Error message port is missing

Output of `traefik version`: (What version of Traefik are you using?)

Version:      2.4.0                                                                                                                                                                                                                                                 
Codename:     livarot                                                                                                                                                                                                                                               
Go version:   go1.15.6                                                                                                                                                                                                                                              
Built:        2021-01-19T17:26:51Z                                                                                                                                                                                                                                  
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

version: "2.0"
networks:
  macvlan:
    driver: macvlan
    driver_opts:
      parent: interface
    ipam:
      config:
      - subnet: 192.168.2.0/24
        ip_range: 192.168.2.0/24
        gateway: 192.168.2.254
services:
  traefik:
    image: traefik
    networks:
      - macvlan
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: always
    command:
      - --api
      - --providers.docker=true
    labels:
      - traefik.enable=true
      - traefik.http.routers.traefik.middlewares=auth
      - traefik.http.routers.traefik.entrypoints=websecure
      - traefik.http.routers.traefik.rule=Host(`traefik.mydomain.com`)
      - traefik.http.routers.traefik.service=api@internal

The problem is certainly on that line: https://github.com/traefik/traefik/blob/326be29568426e4ac8e2bf65fdae3906e8dd58fd/pkg/provider/docker/config.go#L269 This condition should not apply to internal services.

Adding - traefik.http.services.traefik.loadbalancer.server.port=443 to my config is a working workaround

Jan 29 '21 14:01 FuNK3Y

Hello @FuNK3Y,

Could you provide our Traefik log from the beginning and the associated configuration?

Jan 29 '21 16:01 jbdoumenjou

@jbdoumenjou Here is the complete log:

traefik_1  | time="2021-01-30T10:14:59Z" level=info msg="Configuration loaded from flags."
traefik_1  | time="2021-01-30T10:14:59Z" level=error msg="service \"traefik-traefik-debug\" error: port is missing" providerName=docker container=traefik-traefik-debug-7cbebd50f934c65ed54ddb5c2cae7d2bd1ad461dc68293442894cf5574a09d3c

Here is the debug one (not sure it helps there):

traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="Configuration loaded from flags."
traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="Traefik version 2.4.0 built on 2021-01-19T17:26:51Z"
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"http\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":15000000000}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"}}"
traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="Start TCP Server" entryPointName=http
traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="Starting provider *docker.Provider {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":15000000000}"
traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="Starting provider *traefik.Provider {}"
traefik_1  | time="2021-01-30T10:17:44Z" level=info msg="Starting provider *acme.ChallengeTLSALPN {\"Timeout\":2000000000}"
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"tls\":{}}" providerName=internal
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="Provider connection established with docker 19.03.13 (API 1.40)" providerName=docker
traefik_1  | time="2021-01-30T10:17:44Z" level=error msg="service \"traefik-traefik-debug\" error: port is missing" providerName=docker container=traefik-traefik-debug-b93e9fc3b97ea1a5789c508af3cd5e60b48e86f9a87fe58d8dbc5972f741359f
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="Configuration received from provider docker: {\"http\":{},\"tcp\":{},\"udp\":{}}" providerName=docker
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2021-01-30T10:17:44Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2021-01-30T10:17:45Z" level=debug msg="No default certificate, generating one"

Jan 30 '21 10:01 FuNK3Y

I'm also seeing this but with a standard bridge network.

networks:
  traefik:
    internal: true
    name: traefik_internal
    ipam:
      config:
        - subnet: 10.0.3.0/24

Jun 03 '21 15:06 scottwallacesh

It works when not defining a rule and just rely on the default rule or when making the network not internal:

services:
  traefik:
    labels:
      - "traefik.enable=true"
      #- "traefik.http.routers.traefik.rule=Host(`traefik.${DOMAIN_NAME}`)"
      - "traefik.http.routers.traefik.service=api@internal"

networks:
  proxy:
    name: proxy
    internal: true

works with the default rule.

But as soon as I overwrite the default rule with a label:

services:
  traefik:
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${DOMAIN_NAME}`)"
      - "traefik.http.routers.traefik.service=api@internal"

networks:
  proxy:
    name: proxy
    internal: true

I get:

time="2021-11-05T12:34:04+01:00" level=error msg="service \"traefik-proxy\" error: port is missing" container=traefik-proxy-e29d2960a744b1e66173b05865fe51a72569be1e754e5c353240c8c93705b15f providerName=docker

And:

services:
  traefik:
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${DOMAIN_NAME}`)"
      - "traefik.http.routers.traefik.service=api@internal"

networks:
  proxy:
    name: proxy
    #internal: true

works.

So to me it looks like to be a combination of an internal only network with a rule on the container.