docker icon indicating copy to clipboard operation
docker copied to clipboard

High CPU utilisation of woker container.

Open mahajanankur opened this issue 5 years ago • 2 comments

I found the membership-manager docker is utilising a lot CPU due to which master and workers are also heating up the environment.

When I stop the membership-manager docker, everything goes to normal.

This might be due to membership-manager has connected to Citus cluster and firing the events repeatedly.

citus_stats

mahajanankur avatar Apr 13 '20 07:04 mahajanankur

Thanks @mahajanankur for reporting this.

~I created two PRs that aims to resolve this issue:~ ~1. https://github.com/citusdata/membership-manager/pull/9 implements a polling mechanism in Membership Manager, so that (a) it will detect the readiness of the coordinator node, (b) properly report that it is ready to accept new Citus worker services~

~2. https://github.com/citusdata/docker/pull/187 (a) updates the docker images to properly detect dependencies, (b) introduce Compose V3 definitions~

~Once they are merged, this issue should be resolved~

Edit: this issue seems unrelated to my fixes

hanefi avatar Apr 13 '20 09:04 hanefi

@hanefi, just to inform you, the issue is still there even after shutting down the membership-manager container. Server with load citus_stats_2

Idle server citus_stats_3

Update: The Citus docker ecosystem pooling the data frequently due to which CPU and I/O gets chocked. This happens even if there is no load or even no schema.

mahajanankur avatar Apr 13 '20 12:04 mahajanankur