billiard icon indicating copy to clipboard operation
billiard copied to clipboard

getrusage(rusage_self).ru_maxrss is compared against 0 but should always be non-negative

Open frankier opened this issue 9 years ago • 1 comments

I'm filing this issue my progress so far with a bug but it's currently incomplete. I am currently receiving the message "worker unable to determine memory usage" in my log. This is logged here: https://github.com/celery/billiard/blob/cfeac80dfd9136924af066587942c360e9fb67d8/billiard/pool.py#L385 as a consequence of getrusage(rusage_self).ru_maxrss being < 0 but I am having trouble seeing how this is possible. I certainly don't think negative values are intended as error values (at least on Linux) as the current code seems to assume.

To explain why it shouldn't be possible we can look at the journey of ru_maxrss:

  1. It goes from here https://github.com/torvalds/linux/blob/84787c572d402644dca4874aba73324d9f8e3948/kernel/sys.c#L1547 where it's an unsigned long (32 bits or 64 bits usually)
  2. To here https://github.com/torvalds/linux/blob/84787c572d402644dca4874aba73324d9f8e3948/kernel/sys.c#L1613 ; https://github.com/torvalds/linux/blob/9256d5a308c95a50c6e85d682492ae1f86a70f9b/include/uapi/linux/resource.h#L26 where it's a __kernel_long_t which is signed long or signed long long depending on the arch (but again, probably 32 bits or 64bits)
  3. Then it gets converted into a Python type here: https://github.com/python/cpython/blob/2.7/Modules/resource.c#L89

The only way it could end up negative as far as I can see is by overflowing into the sign bit of a 32 bit signed integer, but in this case the RSS would have to be to be larger than 2**31-1 kilobytes which is >2TB. I don't have any processes this big.

frankier avatar Sep 30 '16 07:09 frankier

could you check this on latest release?

auvipy avatar Jul 17 '17 15:07 auvipy