tpie icon indicating copy to clipboard operation
tpie copied to clipboard

Resource memory limit exceeded for `tpie::priority_queue`

Open SSoelvsten opened this issue 1 year ago • 2 comments

The following program pushes the unsigned 64-bit integers 1, 2, ..., 20000000 to TPIE's external memory priority queue in reverse and then pulls them afterwards (this essentially just reverses the input). The amount of memory is 32MiB, whereas the 20000000 numbers take up 152MiB.

#include <tpie/tpie.h>
#include <tpie/priority_queue.h>

int main(int argc, char* argv[]) {
  tpie::tpie_init();
  tpie::get_memory_manager().set_limit(32 * 1024 * 1024);

  const size_t MAX = 20000000;

  {
    tpie::priority_queue<size_t, std::less<size_t>> _pq;

    std::cout << "pushing..." << std::endl;
    for (size_t k = MAX; k > 0; --k) {
      _pq.push(k);
    }

    std::cout << "pulling..." << std::endl;
    for (size_t k = 1; k <= MAX; ++k) {
      assert(_pq.top() == k);
      _pq.pop();
    }
  }
  tpie::tpie_finish();
}

The console output when the above is run is

pushing...
pulling...
Resource memory limit exceeded by 1MiB (6%), while trying to increase usage by 2MiB. Limit is 32MiB, but 33MiB would be used.
Resource memory limit exceeded by 3MiB (12%), while trying to increase usage by 2MiB. Limit is 32MiB, but 35MiB would be used.
Resource memory limit exceeded by 5MiB (18%), while trying to increase usage by 2MiB. Limit is 32MiB, but 37MiB would be used.
Resource memory limit exceeded by 7MiB (24%), while trying to increase usage by 2MiB. Limit is 32MiB, but 39MiB would be used.
Resource memory limit exceeded by 9MiB (31%), while trying to increase usage by 2MiB. Limit is 32MiB, but 41MiB would be used.
Resource memory limit exceeded by 11MiB (37%), while trying to increase usage by 2MiB. Limit is 32MiB, but 43MiB would be used.
Resource memory limit exceeded by 13MiB (43%), while trying to increase usage by 2MiB. Limit is 32MiB, but 45MiB would be used.
Resource memory limit exceeded by 15MiB (49%), while trying to increase usage by 2MiB. Limit is 32MiB, but 47MiB would be used.
Resource memory limit exceeded by 17MiB (56%), while trying to increase usage by 2MiB. Limit is 32MiB, but 49MiB would be used.
Resource memory limit exceeded by 21MiB (68%), while trying to increase usage by 2MiB. Limit is 32MiB, but 53MiB would be used.
Resource memory limit exceeded by 25MiB (81%), while trying to increase usage by 2MiB. Limit is 32MiB, but 57MiB would be used.
Resource memory limit exceeded by 29MiB (93%), while trying to increase usage by 2MiB. Limit is 32MiB, but 61MiB would be used.
Resource memory limit exceeded by 33MiB (106%), while trying to increase usage by 2MiB. Limit is 32MiB, but 65MiB would be used.
Resource memory limit exceeded by 39MiB (124%), while trying to increase usage by 2MiB. Limit is 32MiB, but 71MiB would be used.
Resource memory limit exceeded by 45MiB (143%), while trying to increase usage by 2MiB. Limit is 32MiB, but 77MiB would be used.

This is on a Fedora 36 machine and building sources with

  • GCC version 12.2.1 20220819 (Red Hat 12.2.1-2)
  • CMake version 3.24.2
    • CMAKE_BUILD_TYPE = Debug
    • CMAKE_C_FLAGS = -g -O2
    • CMAKE_CXX_FLAGS = -g -O2
  • Boost version 1.76.0 and release 12.fc36

SSoelvsten avatar Nov 11 '22 13:11 SSoelvsten