aws-cdk icon indicating copy to clipboard operation
aws-cdk copied to clipboard

batch: support multi-node job on Amazon EKS

Open mazyu36 opened this issue 1 year ago • 4 comments

Describe the feature

AWS batch now supports multi-node parallel jobs on Amazon EKS from Jul 11, 2024. But L2 Construct does not support.

MultiNodeJobDefinition is only for multi-node jobs on Amazon ECS.

Use Case

To use multi-node jobs on Amazon EKS.

Proposed Solution

Add new class like MultiNodeEksJobDefinition.

Other Information

When addressing this issue, is submitting an RFC mandatory?

Acknowledgements

  • [X] I may be able to implement this feature request
  • [ ] This feature might incur a breaking change

CDK version used

2.147.0

Environment details (OS name and version, etc.)

all

mazyu36 avatar Jul 19 '24 14:07 mazyu36

Thank you mazyu. We are looking forward to your PRs as always :)

pahud avatar Jul 19 '24 16:07 pahud

I realized that this cannot be implemented at present. EKS configuration is not yet supported in CloudFormation's NodeProperties.

Node properties can't be specified for Amazon EKS based job definitions.

We need to wait for CloudFormation to support this feature.

mazyu36 avatar Jul 20 '24 10:07 mazyu36

This is supported by CloudFormation.

EKS properties are under NodeRangeProperties

delagoya avatar Aug 28 '24 19:08 delagoya

A comment on the above - it made sense to fold the Batch API construct NodeProperties.NodeRangeProperties[].container to NodeProperties.containers[] in the CDK L2 construct when you could only define a single container per NodeRange, but now that you can define multiple containers per NodeRange, the L2 construct should reflect the API more closely.

If a new MultiNodeEksJobDefinition class is created as part of the pull request, it makes more sense for the class to follow the API model and define a readonly nodeRanges: EksNodeRange[] property.

delagoya avatar Aug 28 '24 20:08 delagoya