vmprof-python icon indicating copy to clipboard operation
vmprof-python copied to clipboard

vmprof produces wrong stack traces when code object addresses are reused

Open cfbolz opened this issue 6 years ago • 0 comments

vmprof has the fundamental assumption that code object addresses are unique across the runtime of the program. That is incorrect in many cases, e.g. I think the code object of the global scope of a module dies after importing and the same memory address could be reused by another code object later. If that happens, a random name for the address wins, leading to very confusing stack traces.

An extreme way to show the problem is something like this:

def f(a):
    for i in range(1000000):
        i += 2
    return a + 1

def call(f):
    res = f(2)
    assert res == 3
    return res

code = type(f.func_code)
func = type(f)


def main():
    for i in range(100):
        fc = f.func_code
        fnc = code(fc.co_argcount, fc.co_nlocals, fc.co_stacksize, fc.co_flags, fc.co_code, fc.co_consts, fc.co_names, fc.co_varnames, fc.co_filename, fc.co_name + "_" + str(i), fc.co_firstlineno, fc.co_lnotab, fc.co_freevars, fc.co_cellvars)
        f2 = func(fnc, f.func_globals, f.func_name)
        call(f2)
        print i, id(fnc)

main()

This should show 100 functions f0, ..., f99 but for me on CPython I get just two of them, taking about 50% of the time each, since the memory address of the code object gets reused every other iteration.

This is not a hypothetical problem, I am seeing this occur in practice for large programs such as translating PyPy.

cfbolz avatar May 01 '19 16:05 cfbolz