bnetzi
bnetzi
Hi, we also got an issue with spark operator slowness , but we found that the real reason for the slowness is scale is the controller delayed queue **mutex.** We...
> Impressive work! A few questions: > > * What do we need K8s burst and QPS for? > * What's the impact on memory and CPU consumption of setting...
> @bnetzi Thanks for the clarification. That makes sense. The default is 50, if I remember correctly. Doubling the Spark operator QPS strikes a balance between allowing for more requests...
@vincentye38 @yuchaoran2011 Thanks for commenting First I would mention that we obviously increased the amount of controller threads while raising the cpu, and it did not help at all. The...
> @bnetzi Thank you for explaining. It's interesting that the cpu utilization is so low. > > > each of the go routines is waiting in line to preform a...
Adding a google doc I've made for illustrating the issue and our solution https://docs.google.com/document/u/0/d/14x76jK7naRjxeFCkqUe7MgoDzZjYETixXXa828o0kSM/mobilebasic
For all the followers if this MR, per the massive change in spark operator design, I've waited with this MR to see if it is still relevant after the changes...
@xyzyng from what @ImpSy wrote it sounds like the main issue with V2 is just some tweaks with the controller runtime configurations (which make sense, since it used for many...
@Balraj06 Hi, for sure tried and it helps for the specific use case I presented. Having said that - it is still a bug due the following reasons: 1. It...
@Balraj06 Sure https://github.com/kubernetes-sigs/karpenter/pull/1305