Seperate pod assignment config for worker & coordinator

[oasisvali]

This PR updates the affinity, nodeSelector and tolerations values to contain 2 sub-configurations, one for the worker and one for the coordinator. This allows seperate affinity, nodeSelector and tolerations configuration for the workers vs the coordinator pods. This is a common requirement as the resource requirements for a worker pod is usually different compared to a coordinator pod.

This PR enables this change in a non-backward-compatible way. If backward compatibility is required, I can update the PR so that the current format of affinity, nodeSelector and tolerations values are used as a common fallback for worker and coordinator pods.

[oasisvali]

@hashhar any feedback on whether this PR should be updated to be backward-compatible and use the current format of affinity, nodeSelector and tolerations values as a common fallback for both worker and coordinator pods?

[hashhar]

I think preserving backward compatibility is generally a good idea but I don't want it to prevent us from doing the right thing.

I'm fine with current approach you have. cc: @przemekak @radek-starburst

I'll just take care to do a separate release which includes only this change so that people can easily upgrade to get other features and then wait to upgrade further to this version.

[nineinchnick]

I'll close this, since there was no activity here for a long time. Feel free to reopen after rebasing.