Add Labels and Annotations to Listener CRD
Summary
Allow users to specify custom labels and annotations directly in the Listener CRD spec that will be applied to the created Kubernetes Service.
Current Behavior
Labels/annotations can only be added via ConfigMaps with skupper.io/label-template: "true":
apiVersion: v1
kind: ConfigMap
metadata:
name: service-labels
labels:
skupper.io/label-template: "true"
monitoring.io/scrape: "true"
data:
kind: "Service"
Problems:
- Requires separate ConfigMap resource
- No per-Listener control
- Applies globally unless filtered by name
- Not intuitive
Proposed Solution
Add labels and annotations fields to ListenerSpec:
apiVersion: skupper.io/v2alpha1
kind: Listener
metadata:
name: backend
spec:
routingKey: backend
host: backend
port: 8080
labels:
monitoring.io/scrape: "true"
team: "platform"
annotations:
prometheus.io/port: "8080"
Benefits
- Follows standard Kubernetes patterns
- Per-Listener customization
- Simpler UX
- Better integration with service meshes (Istio, Linkerd)
- Easier Prometheus monitoring setup
Use Cases
Prometheus:
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "8080"
Service Mesh:
labels:
app: backend
version: v1
Tagging:
labels:
environment: production
team: platform
Backward Compatibility
ConfigMap-based labels should continue working for global policies.
Thank you for this great issue, @dhia-gharsallaoui. We are considering this - I agree that your proposed UX is an improvement and is more in line with what kubernetes users expect. I am reasonably confident we'll move forward with this. It is clearly an in-demand improvement.
I do think we need to consider how this behaves when combined with the exposePodsByName feature - where the Listener resolves to one Service (name = .spec.host) plus a Service per target pod across the network.
For purposes of historical context, this is how we recall arriving at the present status-quo:
When developing version 2.0 of Skupper and introducing these CRDs the choice to omit options like these from our API (and to introduce the label-template ConfigMap scheme as a work-around) was a principled decision.
- Platform agnostic: We use the same API resources to configure skupper outside of kubernetes. Labels+annotations on a listener are less relevant on linux system site. Admittedly, we've only half-way held this line - see Connector.spec.selector for an obvious exception.
- Fear of the "slippery slope" and past lessons learned in v1 where configuration ballooned wildly over its lifetime. Listener to Service(s) is about the cleanest relation we have. Our "busiest" resource, the Site, can resolve to one or two Deployments, a Service, a few ConfigMaps and a skupper.io/v2alpha1 RouterAccess - which itself can resolve to a few Services and one of several ingress-type-specific resources. Adding every reasonable configuration to the Site spec could bury the core concepts and make exploring the API more difficult.
One thing to consider here, would be to use the Listener.metadata.labels and annotations, instead of introducing new fields to the CRD, which would imply a new version.
If passing the Listener self labels and annotations is not a common or desirable behavior, this could be controlled through a flag as part of the Listener.spec.settings map.
@c-kruse Thanks for the detailed explanation about the origins. I see that your goal is to have a universal engine that behaves the same across Kubernetes and other environments, but I think each platform will still need some platform-specific handling.
I agree with @fgiorgetti idea propagating the labels and annotations from the origin CRD seems like a good and quick first step. I’m happy to help with that while we think about a more Kubernetes-idiomatic approach later on.
Maybe instead of propagating them directly, we could use a dedicated annotation like this, and let the controller parse and apply them:
skupper.io/svc-annotations: "prometheus.io/scrape=true,prometheus.io/port=8080"
@dhia-gharsallaoui there is a possible issue with this approach for propagating annotations, as more complex data could be passed, like json (that could break the value parsing). But we could think of something in this direction where a specific prefix could be used, without relying on a comma separated list.