Support for additional pod scheduling definition on IntegrationSource

[acolombier]

Problem

Currently, when using IntegrationSource (or IntegrationSink), it would be great to be possible to define Scheduling to help with the orchestrion of the pod integrations

Persona:

System Operator

Exit Criteria

A good measure would be the ability to restrict the scheduling of the integration pod on a specific node pool or affinity

Time Estimate (optional):

It feels pretty straight forward as the definiton should only be reflected in here. It however would requires updating the API, which may make it slightly more complex/time consuming

Additional context (optional) Here is a suggested API definition

apiVersion: sources.knative.dev/v1alpha1
kind: IntegrationSource
metadata:
  name: ny-aws-sqs-queue
spec:
  aws:
    sqs:
      arn: arn:aws:...
      region: eu-central-1
    auth:
      secret:
        ref:
          name: my-secret
  sink:
    ref:
      apiVersion: eventing.knative.dev/v1
      kind: Broker
      name: my-broker
  template:
    spec:
      nodeSelector:
        kubernetes.io/os: linux
      affinity:
        nodeAffinity:                                                                                                                                                                                                 
          requiredDuringSchedulingIgnoredDuringExecution:                                                                                                                                                             
            nodeSelectorTerms:                                                                                                                                                                                        
            - matchFields:                                                                                                                                                                                            
              - key: metadata.name                                                                                                                                                                                    
                operator: In                                                                                                                                                                                          
                values:                                                                                                                                                                                               
                - my-node.eu-central-1.compute.internal  
      tolerations:  
      - effect: NoExecute                                                                                                                                                                                             
        key: node.kubernetes.io/not-ready                                                                                                                                                                             
        operator: Exists                                                                                                                                                                                              
        tolerationSeconds: 300  

Note that we could leverage template.spec to also (or later on) support customisation of additional pod aspect, such as adding metadata or customising serviceAccount, which may help with IRSA support (#8437) Also note that ContainerSource also already support template definition so it would align both CRDs

[creydr]

/cc @matzew

[matzew]

@acolombier Thanks for the feedback. That sounds like a great addition

[Cali0707]

/triage accepted

[S-Mann]

Hi, I would like to work on this. I am using the pod spec from k8s.io/api/core/v1. I am wondering where do I make the changes to use the pod spec attributes for when scheduling the events?

[Cali0707]

Hey @S-Mann thanks for your interest here!

The types for the IntegrationSource are defined here: https://github.com/knative/eventing/blob/main/pkg/apis/sources/v1alpha1/integration_types.go, and the CRDs (which also likely need to be updated) are defined here: https://github.com/knative/eventing/blob/main/config/core/resources/integrationsource.yaml

[S-Mann]

Is there an automated way to update the CRDs? I ran the update codegen to reflect the changes I made with the IntegrationSource spec, current code would be meet the issue's expectation? EDIT: Updated the CRDs by copying the ContainerSource CRD's template attribute as @acolombier suggested, this will make the template attribute uniform between them

[Cali0707]

Is there an automated way to update the CRDs?

In the past we had issues with the CRD generators, so currently you need to update manually

[S-Mann]

Okay, I think my PR is ready. Are there test cases or something else to update?

[Cali0707]

@S-Mann there are normally unit tests in each go package you made changes in, you will need to ensure those still pass. Additionally, there are e2e tests that run. However, those take a long time to run so normally I see if they pass on github before debugging any failing ones locally.

When you're ready for a review on the PR, comment /cc @Cali0707 and I'll be assigned to take a look!

[S-Mann]

I ran the unit test for both integrationsource and containersource, they are passing now. I am going to move the PR to ready and mark you as a reviewer.

[S-Mann]

Hey @Cali0707 is there a way to join the knative github org? Do you guys host some meeting to discuss work items? I would like to contribute more.

[Cali0707]

Hey @Cali0707 is there a way to join the knative github org? Do you guys host some meeting to discuss work items? I would like to contribute more.

@S-Mann yes - to join the community create a PR like the one I made a while ago: https://github.com/knative/community/pull/1364

In terms of meetings, all of the meetings can be found in the Knative community calendar. For eventing in particular, we meet at 11am Eastern Time every Thursday. I hope to see you there!