spring-cloud-dataflow icon indicating copy to clipboard operation
spring-cloud-dataflow copied to clipboard

Not able to add IAM role

Open VikasMGowda05 opened this issue 11 months ago • 6 comments
trafficstars

Description: How can we add the pod annotaions, I need to add the IAM roles for all pods which gets launched by SCDF.

Release versions:

we are using "bitnami/spring-cloud-dataflow:2.11.5-debian-12-r2"

Screenshots: Sample screen shot is below @@ Screenshot 2024-12-09 at 1 18 15 PM

Additional context

Below is our values.yaml file

server: image: registry: docker.io repository: bitnami/spring-cloud-dataflow tag: 2.11.5-debian-12-r2 digest: "" pullPolicy: IfNotPresent pullSecrets: [] debug: false composedTaskRunner: image: registry: docker.io repository: bitnami/spring-cloud-dataflow-composed-task-runner tag: 2.11.5-debian-12-r2 digest: "" configuration: streamingEnabled: false batchEnabled: true accountName: default trustK8sCerts: false containerPorts: http: 8080 jdwp: 5005 replicaCount: 1 updateStrategy: type: RollingUpdate startupProbe: enabled: false initialDelaySeconds: 120 timeoutSeconds: 1 periodSeconds: 20 failureThreshold: 6 successThreshold: 1 livenessProbe: enabled: true initialDelaySeconds: 120 timeoutSeconds: 1 periodSeconds: 20 failureThreshold: 6 successThreshold: 1 readinessProbe: enabled: true initialDelaySeconds: 120 timeoutSeconds: 1 periodSeconds: 20 failureThreshold: 6 successThreshold: 1 networkPolicy: enabled: false allowExternal: false allowExternalEgress: false service: type: ClusterIP ports: http: 8080 ingress: enabled: true path: / pathType: ImplementationSpecific hostname: "xyz.com" pdb: create: false minAvailable: "" maxUnavailable: "" pdb: create: false skipper: enabled: false rabbitmq: enabled: false mariadb: enabled: false metrics: enabled: false pdb: create: false externalDatabase: host: "{{RDS-endpoint}}.rds.amazonaws.com driver: com.mysql.cj.jdbc.Driver dataflow: url: "{Database url}" username: password:

VikasMGowda05 avatar Dec 09 '24 07:12 VikasMGowda05

I believe there are podAnnotations in the Bitnami Helm values schema. You can set different for skipper stream deployments and dataflow task launches.

corneil avatar Dec 09 '24 10:12 corneil

We are using this as below as we didn't find the dataflow task launches and it is not working, could you please suggest here

server: podAnnotations: iam.amazonaws.com/role: ' ' service: annotations: iam.amazonaws.com/role: ' '

VikasMGowda05 avatar Dec 09 '24 10:12 VikasMGowda05

@corneil , will you please help here

VikasMGowda05 avatar Dec 09 '24 12:12 VikasMGowda05

If you are trying to add pod annotations to your application you can add them using the following instructions:https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#_annotations

cppwfs avatar Dec 09 '24 13:12 cppwfs

Hi @cppwfs , we have tried that , its not working. Please check the below configuration for reference and Thanks in advance.

deployer: dsp: kubernetes: podAnnotations: iam.amazonaws.com/role:'' serviceAnnotations: iam.amazonaws.com/role:'' jobAnnotations: iam.amazonaws.com/role:''

deployer: default: kubernetes: podAnnotations: iam.amazonaws.com/role:'' serviceAnnotations: iam.amazonaws.com/role:'' jobAnnotations: iam.amazonaws.com/role:''

VikasMGowda05 avatar Dec 09 '24 13:12 VikasMGowda05

It seems like the Bitnami Helm chart doesn't provide for adding a generic block of configuration for dataflow or skipper like we do in our Carvel deployment. The plain kubernetes deployment also provides for extending the spring.cloud.dataflow or spring.cloud.skipper configuration with YAML or environment variables. https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-kubernetes It seems like you will have to use extraEnvVarsCM or extraEnvVars to provide the full properties like

server:
  extraEnvVars:
    - name: SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_PODANNOTATIONS
      value: "iam.amazonaws.com/role: ''"
    - name: SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_JOBANNOTATIONS
      value: "iam.amazonaws.com/role: ''""

corneil avatar Dec 09 '24 13:12 corneil