Broken argument passing when threading

[lshamis]

I'm trying to run C++ functions in Python threads. To allow them to run concurrently, I mark them with __release_gil__ = True. Unfortunately, there seems to be some buffering of arguments (or something) and the Python arguments overwrite on their way to C++.

Reproducible test case:

import cppyy
import concurrent.futures

cppyy.cppdef("""
void cpp_fn(const std::string& s) {
    std::cerr << "c++ got " << s << std::endl;
}
""")
cppyy.gbl.cpp_fn.__release_gil__ = True

def job(data):
    print(f"py got {data}")
    cppyy.gbl.cpp_fn(data)

data_samples = ["sample_00", "sample_01", "sample_02", "sample_03"]

with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:
    for data in data_samples:
        executor.submit(job, data)

Output:

py got sample_00
py got sample_01
py got sample_02
py got sample_03
c++ got sample_03
c++ got sample_03
c++ got sample_03
c++ got sample_03

[wlav]

Yes, the buffering is per function. It's being moved in several instances to the call context instead, but some of them are argument dependent (incl. converters for std::string) and take a bit more work. Simplest workaround, for now, is to change the code to not need a buffer:

executor.submit(job, cppyy.gbl.std.string(data))