flink-containers-example icon indicating copy to clipboard operation
flink-containers-example copied to clipboard

Published 20 hours ago • verified publisher icon alpinegizmo

do i need to create zookeeper and minio for every flink job

Open nealsun opened this issue 5 years ago • 4 comments

In this mode, flink job manger and flink task manager packed together for every single flink job, do i need to create zookeeper and minio for every flink job? or is it the best practice to packed all together to deploy one job?

nealsun avatar Aug 19 '19 03:08 nealsun

Typically you will have one zookeeper cluster, and one distributed filesystem (e.g., minio) that are shared by all of your deployments. Or if you already have a zookeeper set up for Kafka, just use that one.

alpinegizmo avatar Aug 19 '19 12:08 alpinegizmo

got it. Thanks for your reply!

nealsun avatar Aug 19 '19 13:08 nealsun

Why use k8s Job to deploy Job Manager rather than k8s Deployment? i'm using gitlab to deploy my flink job, it's easy to patch a Deployment with new docker image, but Job does not support patch. Can i use Deployment instead of Job to deploy Job Manager ?

nealsun avatar Aug 22 '19 06:08 nealsun

@nealsun I haven't tried using a Deployment for the Flink Master (job manager) -- if you try it, let me know what you find.

alpinegizmo avatar Aug 22 '19 10:08 alpinegizmo