yaegi icon indicating copy to clipboard operation
yaegi copied to clipboard

feat: improve pprof experience by adding wrappers to interpreted functions

Open david-garcia-garcia opened this issue 2 months ago • 1 comments

When profiling applications using Yaegi (e.g., Traefik with plugins), all interpreted functions appear as anonymous closures in pprof.

This makes profiling impossible. If there was a way of doing this, I could not find it.

I tried to solve this with pprof labels, but they are only supported in CPU analysis and not in memory heap analysis.

I also tried to use reflection to create named intermediate wrappers without success. This is limited or super complex in go (I've done this in the past in IL with .Net).

I am not happy on how little elegant the solution is. I tried to prioritize performance (plus I didn't find a way to accomplish this differently).

What I am doing is assign each anonymous function one of a preset of 500 pregenerated wrappers that will properly show in stack traces and dumps. Then the runtime exposes a method GetWrapperMappings() to get the mappings between these wrappers and the real function names and packages.

Memory overhead is minimal (~125 KB for 500 wrappers) and runtime overhead is also almost negligible.

If you run out of wrappers worst case you might have the more than one function mapped to the same wrapper, which is way better than current situation.

Results

CPU Profile

BEFORE this PR:

File: traefik
Type: cpu
Showing nodes accounting for 1.88s, 87.44% of 2.15s total

1.88s  87.44%  github.com/traefik/yaegi/interp.call.func9
1.88s  87.44%  github.com/traefik/yaegi/interp.genFunctionWrapper.func1.1
1.88s  87.44%  github.com/traefik/yaegi/interp.runCfg

AFTER this PR:

File: interp.test.exe
Type: cpu
Showing nodes accounting for 1.88s, 87.44% of 2.15s total

    0     0%     0%      1.88s 87.44%  github.com/traefik/yaegi/interp.wrapperLevel1
    0     0%     0%      1.88s 87.44%  github.com/traefik/yaegi/interp.wrapperLevel2
    0     0%     0%      1.88s 87.44%  github.com/traefik/yaegi/interp.wrapperLevel6
0.17s  7.91%     0%      1.88s 87.44%  github.com/traefik/yaegi/interp.runCfg

Wrapper Mapping:

wrapperLevel1 → geoblock:*Database:Query
wrapperLevel2 → geoblock:*Plugin:ServeHTTP  
wrapperLevel6 → geoblock::CallServeHTTP

Heap/Allocation Profile

BEFORE this PR:

File: traefik
Type: alloc_space
Showing nodes accounting for 4496MB, 99.87% of 4502MB total

4496MB  99.87%  github.com/traefik/yaegi/interp.genFunctionWrapper.func1.1
4443MB  98.68%  github.com/traefik/yaegi/interp.call.func9.2

AFTER this PR:

File: interp.test.exe
Type: alloc_space
Showing nodes accounting for 4502MB, 100% of 4502MB total

4059MB  90.14%  github.com/traefik/yaegi/interp.wrapperLevel0
 384MB   8.52%  github.com/traefik/yaegi/interp.wrapperLevel2
 353MB   7.83%  github.com/traefik/yaegi/interp.wrapperLevel1

Wrapper Mapping:

wrapperLevel0 → geoblock:*Database:Load
wrapperLevel1 → geoblock:*Database:Query
wrapperLevel2 → geoblock:*Plugin:ServeHTTP

david-garcia-garcia avatar Oct 22 '25 14:10 david-garcia-garcia

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Oct 22 '25 14:10 CLAassistant