Allow `ServerAuthorization` targeting multiple namespaces by namespace label or naming pattern

[ripta]

What problem are you trying to solve?

I'd like to be able to allow more flexible patterns in identities and/or service account specifications, both in the old-style ServerAuthorization and in the newly introduced MeshTLSAuthentication.

The reasoning is that we have labels on our Kubernetes namespaces that allow us to "group" certain namespaces together. These namespace labels are managed by a separate service (out of the scope of this request).

For example, in our dev cluster, we allow every engineer to stand up complete copies of apps, where we create namespace prefixes. The mobile app API might normally live in the mobile-api namespace. But if I were developing a new feature for the app, I might run my feature branch in the namespace dev-ripta01-mobile-api, while some other engineer—Alice—might have dev-alice01-mobile-api. The namespaces in this case might have a special label on them, say foo.dev/app: mobile-api.

Any number of developers might have many copies, and currently, the only solution is to exhaustively list all identities, like so:

apiVersion: policy.linkerd.io/v1beta1
kind: ServerAuthorization
# some fields hidden
spec:
  client:
    meshTLS:
      serviceAccounts:
        - namespace: mobile-api
          name: default
        - namespace: dev-alice01-mobile-api
          name: default
        - namespace: dev-ripta01-mobile-api
          name: default

or using wildcard support in identities:

spec:
  client:
    meshTLS:
      identities:
        - "*.mobile-api.serviceaccount.identity.linkerd.cluster.local"
        - "*.dev-alice01-mobile-api.serviceaccount.identity.linkerd.cluster.local"
        - "*.dev-ripta01-mobile-api.serviceaccount.identity.linkerd.cluster.local"

How should the problem be solved?

I'm not sure what the best solution is, but a common pattern I've seen in NetworkPolicy and ValidatingWebhookConfiguration is a field called namespaceSelector, which allows one to target namespaces by their label. So an example might look like:

apiVersion: policy.linkerd.io/v1beta1
kind: ServerAuthorization
# some fields hidden
spec:
  client:
    meshTLS:
      serviceAccounts:
        - name: mobile
          namespaceSelector:
            matchExpressions:
              - key: foo.dev/app
                operator: In
                values:
                  - mobile-api

which would grant access from any pods having the mobile service account located in namespaces with the label foo.dev/app=mobile-api.

Any alternatives you've considered?

I'm not sure if the namespaceSelector solution above has any scaling problems.

For our specific case, even though we mostly rely on namespace labels, most of our namespaces follow a naming convention. So an alternative might be to allow multiple wildcards in the identity, e.g.:

spec:
  client:
    meshTLS:
      identities:
        - '*.mobile-api.serviceaccount.identity.linkerd.cluster.local'
        - '*.*-mobile-api.serviceaccount.identity.linkerd.cluster.local'

I tested the above on an existing cluster, and it returned the error:

serverauthorizations.policy.linkerd.io "dev-grpc" was not valid:

• spec.client.meshTLS.identities: Invalid value: ".-mobile-api.serviceaccount.identity.linkerd.cluster.local": spec.client.meshTLS.identities in body should match '^(*|a-z0-9?)(.a-z0-9?)*$'

How would users interact with this feature?

They would specify the meshTLS parameters in their manifest, and hopefully it Just Works™.

I answered 'maybe' at this feature, mostly because I'm not experienced with a Rust codebase (though I am familiar with Go). And obviously, I'd like to discuss what folks think might be the best solution.

Would you like to work on this feature?

maybe

[kleimkuhler]

Thanks for the thoughts on this @ripta. I'd like to preface that ServerAuthorizations haven't been deprecated, but the more flexible resource — AuthorizationPolicies — were introduced in 2.12 as explained in the documentation.

A lot of their design is based off the Gateway API so we should consider that when possible.

I'd like to point out that in the Gateway API, Gateways can restrict the Routes that restrict them with namespaces.selector field; I think with some design we could consider how this would work with the Linkerd policy resources. Looking at the AuthorizationPolicy Examples it currently supports targeting an entire namespace

...
spec:
  targetRef:
    kind: Namespace
    name: booksapp
...

In order support to target Servers (or HttpRoutes) in multiple namespaces, maybe something like the namespaces.selector field would fit here. However an explicit targetRef that selects a single resource is part of the design so I'm unsure how likely we are to deviate from that.

#9319 has some similar discussion about this and I think if we are to address this we'd more likely more forward with a new CRD that introduces a grouping of Servers and allow authentication that way. I'd keep an eye on that issue for updates.

[ripta]

@kleimkuhler — Thanks for responding and for the clarification about the ServerAuthorizations not currently being deprecated.

I've heard of Gateway API, but hadn't read it fully, so thanks for the link! There's a lot to digest. I think the Gateway API design is something more abstract than what I'm expecting. I understand the linkerd side better, so I hope you'll forgive me for sticking to that.

The issue linked (9319) does have a similar discussion, but I believe I'm looking for the other side of the same coin: I'd like to allow a dynamic list of namespaces in MeshTLSAuthentication. The design on the other issue seems to only limit to Server / HTTPRoute. If the intent is to make a generalized solution, that's definitely ok too; I just want to make sure it's covered.

To take one of the examples from the linkerd docs, my understanding is I already can list multiple namespaces in identityRefs (which is just a list of targetRefs):

## Currently works in linkerd 2.12
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata:
  name: authors-get-authn
  namespace: booksapp
spec:
  identityRefs:
    - kind: ServiceAccount
      name: books
      namespace: dev-alice01-booksapp
    - kind: ServiceAccount
      name: books
      namespace: dev-ripta01-booksapp

And it works fine.

The problem is that I would need to know the list of namespaces ahead of time. I instead need to be able to authorize a set of service accounts from namespaces that is governed by the labels on the namespaces. So maybe something similar to ServerGroup being discussed in the other ticket, but for the required auth refs side of the policy?

## Proposed new custom resource
apiVersion: policy.linkerd.io/v1alpha1
kind: IdentityReferenceGroup
metadata:
  name: books-identity-group
  namespace: booksapp
spec:
  namespaces:
    from: Selector
    selector:
      matchLabels:
        foo.dev/app: booksapp

---
## Existing MeshTLSAuthentication resource that refers to the identity group above
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata:
  name: authors-get-authn
  namespace: booksapp
spec:
  identityRefs:
    - kind: IdentityReferenceGroup
      name: books-identity-group

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.