flink-remote-shuffle icon indicating copy to clipboard operation
flink-remote-shuffle copied to clipboard

Published 20 hours ago • verified publisher icon flink-extended

Support to use StatefulSet + Pvc to deploy the shuffle service

Open Aitozi opened this issue 2 years ago • 2 comments

In the current kubernetes mode, the rss use the hostPath or the emptyDir to store the shuffle data. But, In our company, the local disk resource is very rare and can not meet the storage requirements and It's not the complete storage computing separation architecture I think. I proposal to support to deploy the rss in the way of StatefulSet + PVC , So that we can leverage the cloud storage to store the shuffle data. what do you think of this ? cc @wsry

Aitozi avatar May 08 '22 05:05 Aitozi

@Aitozi Thanks. I think that is a good proposal. Would you like to prepare a PR that?

wsry avatar May 09 '22 03:05 wsry

Yes, I will prepare a PR for it ASAP and will ping you for review, thanks

Aitozi avatar May 09 '22 03:05 Aitozi