activitysim icon indicating copy to clipboard operation
activitysim copied to clipboard

Measuring big USS on Windows

Open jpn-- opened this issue 3 years ago • 0 comments

The psutil library cannot measure USS values greater than 2^37 bytes on Windows. That's about 128GB. The limitation is hard-coded here.

This limitation probably doesn't matter in practice -- if we're using chunking it's cause we don't want to use that much RAM in one process. If we're not using chunking then it's not important to actually measure USS values -- an RSS measurement is sufficient to get an idea of memory consumption.

If anyone has a known workaround, let us know. Otherwise for now I've set the code to just silently ignore failures to read USS without crashing.

jpn-- avatar Jun 03 '22 17:06 jpn--