Leela Gangavarapu
Leela Gangavarapu
> operator tries to reach each node each time /metrics.json is requested - yes, that's how it's implemented, when prom scrapes operator ep, operator will in-turn reaches out to all...
- ack, we can introduce the classic solution, the rate-limiter assuming prom metrics scraping are DDoS attacks 😅 - however, doesn't prometheus has a scrape interval which can be adjusted,...
> This means that after just one day I'll have 480 dead nodes that the operator is still trying to connect to. - makes sense, noted. - I'll wait for...
@Expro do you also use prometheus?
@Expro then it might be a different, better raise a separate issue. At the same time do you see continuous error logs in operator?
- thanks for checking - that chart definitely shows some issue exist in operator - can you set `VERBOSE` env value to `yes` in operator deployment - this will not...
- woah, three instances of same issue - will take a look at the earliest - I'm assuming https://github.com/kadalu/kadalu/blob/ed620e/kadalu_operator/main.py#L885-L889 is the suspect - I may not be able to reproduce...
- not able to reproduce the high cpu issue in my setup - as the original observation is for the ephemeral nodes, in the linked PR I removed redundant data...
- Woah, I think above rules are too open - Let me take a look and get back - Btw, thanks for testing the kadalu metrics via user workload monitoring...
> This RFE talks about merging the development of Moana project and Storage for Kubernetes. - For this RFE implementation, kadalu-storage-manager (let's say ksm, less typing 😅) should be merged...