ACL is not working properly

[smedelyan]

Description

I cannot push if allow push-only for a user. I also saw another case when "pull-only" suddenly allowed push for user (UP: example in the next comment), but example configuration below is for push-only

Config:

server:
  addr: ":5001"
  certificate: "/auth/cert.pem"
  key: "/auth/key.pem"

token:
  issuer: "Acme auth server"  # Must match issuer in the Registry config.
  expiration: 900

users:
  # Password is specified as a BCrypt hash. Use `htpasswd -nB USERNAME` to generate.
  "test":
    password: "$2y$05$mdG8K4U.5d4h7RYqVw7vWeN7F8qztzgn5CvzKQfFtOY8Fz7WD..9a" # 123

acl:
  - match: {account:"test"}
    actions: ["push"]
  # Access is denied by default.

Then:

$ docker login <url>
Username: test
Password:
WARNING! Your password will be stored unencrypted in /home/.../.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

$ docker push <url>/hello-world
The push refers to repository [<url>/hello-world]
e07ee1baac5f: Preparing
denied: requested access to the resource is denied

Logs:

$ docker-compose up docker-auth
Starting registry_docker-auth_1 ... done
Attaching to registry_docker-auth_1
docker-auth_1  | I1111 11:59:51.147314       1 main.go:213] docker_auth 20211004 build 20211004-192635/1.8.0@b7b9d6e0
docker-auth_1  | I1111 11:59:51.148561       1 main.go:61] Config from /config/auth_config.yml (1 users, 1 ACL static entries)
docker-auth_1  | I1111 11:59:51.148588       1 acl.go:109] Created ACL Authorizer with 1 entries
docker-auth_1  | I1111 11:59:51.148602       1 main.go:110] Cert file: /auth/cert.pem
docker-auth_1  | I1111 11:59:51.148609       1 main.go:111] Key file : /auth/key.pem
docker-auth_1  | I1111 11:59:51.149253       1 main.go:142] Serving on :5001
docker-auth_1  | I1111 12:00:06.272664       1 server.go:488] Auth request: {test:***@172.28.0.1:57484 []}
docker-auth_1  | I1111 12:00:06.277488       1 server.go:317] Authn static test -> true, map[], <nil>
docker-auth_1  | I1111 12:00:06.284409       1 server.go:429] New token for {test:***@172.28.0.1:57484 []} map[]: {"iss":"Acme auth server","sub":"test","aud":"Docker registry","exp":1636632906,"nbf":1636631996,"iat":1636632006,"jti":"2653821073426952576","access":[]}
docker-auth_1  | I1111 12:00:14.957211       1 server.go:488] Auth request: {test:***@172.28.0.1:57512 [{repository  hello-world [pull push]}]}
docker-auth_1  | I1111 12:00:14.963503       1 server.go:317] Authn static test -> true, map[], <nil>
docker-auth_1  | I1111 12:00:14.963559       1 acl.go:121] {test pull,push repository hello-world} matched {"Match":{},"Actions":["push"],"Comment":null} (Comment: (nil))
docker-auth_1  | I1111 12:00:14.963897       1 server.go:339] Authz static ACL {test pull,push repository hello-world} -> [push], %!s(<nil>)
docker-auth_1  | I1111 12:00:14.972476       1 server.go:429] New token for {test:***@172.28.0.1:57512 [{repository  hello-world [pull push]}]} map[]: {"iss":"Acme auth server","sub":"test","aud":"Docker registry","exp":1636632914,"nbf":1636632004,"iat":1636632014,"jti":"7158462588365033085","access":[{"type":"repository","name":"hello-world","actions":["push"]}]}

Registry logs:

$ docker logs -f registry_registry_1
time="2021-11-11T11:59:31.930641155Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.11.2 instance.id=05158f38-23d6-4780-a8c7-05035cf011e9 service=registry version=v2.7.1
time="2021-11-11T11:59:31.930714673Z" level=info msg="Starting upload purge in 42m0s" go.version=go1.11.2 instance.id=05158f38-23d6-4780-a8c7-05035cf011e9 service=registry version=v2.7.1
time="2021-11-11T11:59:31.930740319Z" level=info msg="redis not configured" go.version=go1.11.2 instance.id=05158f38-23d6-4780-a8c7-05035cf011e9 service=registry version=v2.7.1
time="2021-11-11T11:59:31.944043636Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.11.2 instance.id=05158f38-23d6-4780-a8c7-05035cf011e9 service=registry version=v2.7.1
time="2021-11-11T11:59:31.945053953Z" level=info msg="listening on [::]:5000" go.version=go1.11.2 instance.id=05158f38-23d6-4780-a8c7-05035cf011e9 service=registry version=v2.7.1
time="2021-11-11T12:00:05.505275156Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=ba47e826-f941-4ede-b915-2a09c7e2ff29 http.request.method=GET http.request.remoteaddr="172.28.0.1:56328" http.request.uri="/v2/" http.request.useragent="docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \(linux\))"
172.28.0.1 - - [11/Nov/2021:12:00:05 +0000] "GET /v2/ HTTP/1.0" 401 87 "" "docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \\(linux\\))"
172.28.0.1 - - [11/Nov/2021:12:00:07 +0000] "GET /v2/ HTTP/1.0" 200 2 "" "docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \\(linux\\))"
time="2021-11-11T12:00:07.009598351Z" level=info msg="authorized request" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=933b2b18-dbe1-45db-97ae-18ab29f48767 http.request.method=GET http.request.remoteaddr="172.28.0.1:56346" http.request.uri="/v2/" http.request.useragent="docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \(linux\))"
time="2021-11-11T12:00:07.009723766Z" level=info msg="response completed" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=933b2b18-dbe1-45db-97ae-18ab29f48767 http.request.method=GET http.request.remoteaddr="172.28.0.1:56346" http.request.uri="/v2/" http.request.useragent="docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \(linux\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.025691ms http.response.status=200 http.response.written=2
time="2021-11-11T12:00:14.227668609Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=313bbbb9-7d81-47c8-b594-31da0fab8c31 http.request.method=GET http.request.remoteaddr="172.28.0.1:56356" http.request.uri="/v2/" http.request.useragent="docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \(linux\))"
172.28.0.1 - - [11/Nov/2021:12:00:14 +0000] "GET /v2/ HTTP/1.0" 401 87 "" "docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \\(linux\\))"
172.28.0.1 - - [11/Nov/2021:12:00:15 +0000] "HEAD /v2/hello-world/blobs/sha256:2db29710123e3e53a794f2694094b9b4338aa9ee5c40b930cb8063a1be392c54 HTTP/1.0" 401 154 "" "docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \\(linux\\))"
time="2021-11-11T12:00:15.72710083Z" level=warning msg="error authorizing context: insufficient scope" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=f5642f07-2d63-4bde-8cad-d2f82877afad http.request.method=HEAD http.request.remoteaddr="172.28.0.1:56364" http.request.uri="/v2/hello-world/blobs/sha256:2db29710123e3e53a794f2694094b9b4338aa9ee5c40b930cb8063a1be392c54" http.request.useragent="docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \(linux\))" vars.digest="sha256:2db29710123e3e53a794f2694094b9b4338aa9ee5c40b930cb8063a1be392c54" vars.name=hello-world
time="2021-11-11T12:00:16.450703541Z" level=warning msg="error authorizing context: insufficient scope" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=c17cea66-31ef-435d-9e7a-81e1e350428e http.request.method=POST http.request.remoteaddr="172.28.0.1:56368" http.request.uri="/v2/hello-world/blobs/uploads/" http.request.useragent="docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \(linux\))" vars.name=hello-world
172.28.0.1 - - [11/Nov/2021:12:00:16 +0000] "POST /v2/hello-world/blobs/uploads/ HTTP/1.0" 401 224 "" "docker/19.03.13 go/go1.13.15 git-commit/4484c46d9d kernel/4.15.0-162-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.13 \\(linux\\))"

Network setup

Nginx proxies HTTPS requests to both registry and docker_auth

[smedelyan]

Unrestricted push (wrong rule match?)

Config:

server:
  addr: ":5001"
  certificate: "/auth/cert.pem"
  key: "/auth/key.pem"

token:
  issuer: "Acme auth server"  # Must match issuer in the Registry config.
  expiration: 900

users:
  # Password is specified as a BCrypt hash. Use `htpasswd -nB USERNAME` to generate.
  "test":
    password: "$2y$05$mdG8K4U.5d4h7RYqVw7vWeN7F8qztzgn5CvzKQfFtOY8Fz7WD..9a" # 123
  "admin":
    password: "$2y$05$mdG8K4U.5d4h7RYqVw7vWeN7F8qztzgn5CvzKQfFtOY8Fz7WD..9a" # 123

acl:
        - match: {account:"admin"}
          actions: ["*"]
          comment: "Comment for admin" # NOTICE THIS COMMENT
        - match: {account:"test"}
          actions: ["pull"]
          comment: "Comment for test"
  # Access is denied by default.

Logs.

rm@NC-PH-0365-19:~/onpremise/registry$ docker-compose up docker-auth
Creating registry_docker-auth_1 ... done
Attaching to registry_docker-auth_1
docker-auth_1  | I1111 12:19:44.810827       1 main.go:213] docker_auth 20211004 build 20211004-192635/1.8.0@b7b9d6e0
docker-auth_1  | I1111 12:19:44.811980       1 main.go:61] Config from /config/auth_config.yml (2 users, 2 ACL static entries)
docker-auth_1  | I1111 12:19:44.812004       1 acl.go:109] Created ACL Authorizer with 2 entries
docker-auth_1  | I1111 12:19:44.812020       1 main.go:110] Cert file: /auth/cert.pem
docker-auth_1  | I1111 12:19:44.812029       1 main.go:111] Key file : /auth/key.pem
docker-auth_1  | I1111 12:19:44.812520       1 main.go:142] Serving on :5001
docker-auth_1  | I1111 12:19:51.034653       1 server.go:488] Auth request: {test:***@192.168.64.1:53466 []}
docker-auth_1  | I1111 12:19:51.039595       1 server.go:317] Authn static test -> true, map[], <nil>
docker-auth_1  | I1111 12:19:51.048264       1 server.go:429] New token for {test:***@192.168.64.1:53466 []} map[]: {"iss":"Acme auth server","sub":"test","aud":"Docker registry","exp":1636634091,"nbf":1636633181,"iat":1636633191,"jti":"2371071496851593163","access":[]}
docker-auth_1  | I1111 12:20:05.503673       1 server.go:488] Auth request: {test:***@192.168.64.1:53542 [{repository  hello-world [pull push]}]}
docker-auth_1  | I1111 12:20:05.510018       1 server.go:317] Authn static test -> true, map[], <nil>
docker-auth_1  | I1111 12:20:05.510076       1 acl.go:121] {test pull,push repository hello-world} matched {"Match":{},"Actions":["*"],"Comment":"Comment for admin"} (Comment: Comment for admin)
docker-auth_1  | I1111 12:20:05.510297       1 server.go:339] Authz static ACL {test pull,push repository hello-world} -> [pull push], %!s(<nil>)
docker-auth_1  | I1111 12:20:05.520118       1 server.go:429] New token for {test:***@192.168.64.1:53542 [{repository  hello-world [pull push]}]} map[]: {"iss":"Acme auth server","sub":"test","aud":"Docker registry","exp":1636634105,"nbf":1636633195,"iat":1636633205,"jti":"1280997329469189576","access":[{"type":"repository","name":"hello-world","actions":["pull","push"]}]}

Notice the following line in logs:

docker-auth_1  | I1111 12:20:05.510076       1 acl.go:121] {test pull,push repository hello-world} matched {"Match":{},"Actions":["*"],"Comment":"Comment for admin"} (Comment: Comment for admin)

Looks like the wrong rule matched?

[tomaswarynyca]

I'm getting worried when I see this 😑

[smedelyan]

This issue looks critical to me, but looks like no one is maintaining this repository anymore?

[techknowlogick]

This issue looks critical to me, but looks like no one is maintaining this repository anymore?

I am, however I'm having trouble tracing this down.

[smedelyan]

Wow! Do you at least confirm you can reproduce?

[techknowlogick]

Wow! Do you at least confirm you can reproduce?

I haven't yet, so I may need to start a virtual machine to test in to ensure that nothing else is affecting my debugging.

[FlyveHest]

Was this ever resolved / looked into? I am considering using docker_auth, but this seems like something of a dealbreaker

[smedelyan]

@FlyveHest @tomaswarynyca I recently heard about https://github.com/goharbor/harbor . My colleagues have had some experience with it and they say that:

• ACL is working and can be configured quite accurately
• it is able not only to serve as Registry but also proxy the official registry as well
• it is able to serve as artifacts storage not only for Docker, has lots of other features which may or may not be needed in your case

I'm going to try it out sooner or later, and if I don't forget I will let you know here if I succeeded

[skwair]

For some reason, docker push seems to require both push and pull permissions from what I've tested, so the original issue might be normal.

I logged the content of accessItems here while running docker push some-registry/some-image: https://github.com/distribution/distribution/blob/26163d82560f4dda94bd7b87d587f94644c5af79/registry/auth/token/accesscontroller.go#L277-L283

and this is what I got:

[
  {
    "Type": "repository",
    "Class": "",
    "Name": "some-image",
    "Action": "pull"
  },
  {
    "Type": "repository",
    "Class": "",
    "Name": "some-image",
    "Action": "push"
  }
]

[s-weigand]

@skwair Thanks to your investigation I was able to fix the issue by giving the user push AND pull permissions.

  - match: {account: "ci_bot"}
    actions: ["push", "pull"]
    comment: "Push images, but not allowed to delete."

Seams like my naive assumption that allowing anonymous pulling would always propagate to the logged in user didn't hold up 😅

  - match: {account: "", type: "registry", name: "catalog"}
    actions: ["*"]
    comment: "Anonymous can query the catalog"
  - match: {account: ""}
    actions: ["pull"]
    comment: "Anonymous can pull everything"