oneTBB icon indicating copy to clipboard operation
oneTBB copied to clipboard

static_partitioner + global_control triggers an unbounded memory leak

Open lennoxho opened this issue 8 months ago • 2 comments

Description

We found using tbb::static_partitioner while a tbb::global_control is active causes steady and seemingly unbounded memory leaks as tasks were executed.

A minimal repro is attached below.

The issue goes away if we do any of the following

  • Replace tbb::static_partitioner with tbb::auto_partitioner
  • Remove the call to tbb::global_control
  • Wrap the tbb::parallel_for call with tbb::task_arena::execute

Reproduced with the following setup

  • GCC 13.1.0 & GCC 13.2.0
  • Ubuntu 20.04LTS native & Ubuntu 22.04 LTS on WSL 2
  • A range of x86-64 Intel/AMD workstation & server-grade hardware
  • oneTBB 2021.12.0 & the latest commit (fdf1fdb)

Minimal Repro

Makefile

TBB_DIR ?= /mnt/d/Users/LennoxHo/source/onetbb/oneTBB-2021.12.0-install

COMPILE_FLAGS := -g -O2 -Wall -Wextra -fPIC -isystem $(TBB_DIR)/include
LINK_FLAGS := -Wl,-rpath,$(TBB_DIR)/lib -L$(TBB_DIR)/lib -ltbb

TEST_NAME := tbb-leak-test

all : test
.PHONY : all test clean

$(TEST_NAME) : tbb-leak-test.o
	g++ $(COMPILE_FLAGS) $^ -o $@ $(LINK_FLAGS)

%.o : %.cpp
	g++ -c $(COMPILE_FLAGS) $< -o $@

clean:
	rm -f *.o
	rm -f tbb-leak-test

test: $(TEST_NAME)
	./$(TEST_NAME)

tbb-leak-test.cpp

#include <cassert>
#include <cstdio>
#include <sys/resource.h>

#include <oneapi/tbb/global_control.h>
#include <oneapi/tbb/parallel_for.h>
#include <oneapi/tbb/partitioner.h>

void print_rss() {
    struct rusage ru;
    int result = getrusage(RUSAGE_SELF, &ru);
    assert(result == 0);

    printf("Max RSS = %ld kB\n", ru.ru_maxrss);
}

void busy_work(int x) {
    volatile int result = x * x;
    (void)result;
}

int main() {
    using tbb_partitioner = tbb::static_partitioner;

    constexpr int num_threads = 8;
    constexpr auto num_tasks_per_iter = 1'000ull;

    constexpr auto num_iterations = 1'000'000ull;
    constexpr auto num_rss_reporting_interval = 1'000ull;

    tbb::global_control gbl_ctrl{ tbb::global_control::max_allowed_parallelism, num_threads };

    fputs("Starting ", stdout);
    print_rss();

    for (auto i = 0ull; i < num_iterations; ++i) {
        tbb::parallel_for(0ull, num_tasks_per_iter, busy_work, tbb_partitioner{});

        if (i % num_rss_reporting_interval == 0) {
            print_rss();
        }
    }

    fputs("Ending ", stdout);
    print_rss();
}

Steps to reproduce:

  • Copy the attached Makefile & tbb-leak-test.cpp
  • make TBB_DIR=<path to oneTBB installation>
  • Observe the steady increase in memory usage (my runs end with Ending Max RSS = 3079648 kB)

lennoxho avatar Jun 13 '24 03:06 lennoxho