spring-cloud-deployer-kubernetes icon indicating copy to clipboard operation
spring-cloud-deployer-kubernetes copied to clipboard

Partition batch jobs launched from SCDF don't set deployer properties for worker task pods it launches

Open vrajkuma opened this issue 4 years ago • 0 comments

Discussed the issue in this stackoverflow post : https://stackoverflow.com/questions/69683040/scdf-partition-batch-job-using-spring-cloud-kubernetes-deployer-deployer-pro

Basically in running the partitioned-batch-job sample provided by SCDF in a kubernetes cluster, I am unable to set deployer properties for the worker tasks spawned by the master step. The deployer.<app-name>.kubernetes.<prop-name> properties only apply to the master pod that is created, but the worker pods it spawns using kubernetes deployer don't get them. For example, in my case setting properties for volumes/volumeMounts during the task launch don't get enforced on the worker pods/containers where the volumeMounts are actually needed to process files. These are mounted on the master pod though.

Also, didn't find any documentation on where and how we could set spring.cloud.deployer.properties for worker pods given those are launched by the partitioned task framework

vrajkuma avatar Oct 24 '21 03:10 vrajkuma