fabio icon indicating copy to clipboard operation
fabio copied to clipboard

High CPU Utilization

Open codyja opened this issue 5 years ago • 2 comments

Hi, We recently upgrade four completely separate clusters running fabio to 1.5.13 from 1.5.11. Since then two of the sites have all their fabios using a high percentage of CPU. Rolling back to 1.5.11 makes the issue go away.

image

I took a pprof on 1.5.11, and 1.5.13 to try and help identify what's different. The fabios are talking to consul on localhost:8500. Nothing in the configuration is changing when updating versions. We have about 5000k routes within each of the four clusters. We do see a huge memory reduction when moving to 1.5.13 which is nice. Including some pngs of the pprof output. Let me know what else I can provide to help here.

Fabio 1.5.11: fab1 5 11

Fabio 1.5.13 fab1 5 13

codyja avatar Jan 31 '20 20:01 codyja

The two PNGs don't look that much different ... at least not at initial glance. Would it be possible to get samples of approximately the same length from about the same time (similar traffic)? Having the actual prof files would be helpful as well.

aaronhurt avatar Jan 31 '20 20:01 aaronhurt

Hi @leprechau, What samples can I provide here? More CPU charts helpful? The left side of that cpu graph is when running on 1.5.11, and the right side is when running on 1.5.13. Including pprofs from the 2 versions here. pprofs.zip

codyja avatar Jan 31 '20 21:01 codyja