incubator-heron
incubator-heron copied to clipboard
terrible resource is needed when submitting to aurora
I write a simple example to test resource requirement to submit to aurora. It is very terrible that too more resource is needed. Please check below:
my example: builder.newSource(() -> ThreadLocalRandom.current().nextInt(100)) .setName("random-sentences-source");
Config config = Config.newBuilder()
.setNumContainers(1)
.setPerContainerCpu(2)
.setPerContainerRamInGigabytes(3)
.build();
When I submit this topo to aurora, 4.5 cpu and 6.3G memory are allocated to this topology. It seems that Tmaster used 2.5G cpu and 3.3G memory.
My question is how to limit tmaster resource requirement. It seems that Tmaster task using more resource.
In aurora, all containers need to have the same size, and container 0 is mainly for tmaster. Therefore the key is to keep the container size under control. In your example, your cpu and memory usage could be lower if you have 2 containers and 1 core/1.5G per container.
In aurora, all containers need to have the same size, and container 0 is mainly for tmaster. Therefore the key is to keep the container size under control. In your example, your cpu and memory usage could be lower if you have 2 containers and 1 core/1.5G per container.
Let me try, thanks a lot