Accessing Location lists for inlined functions?

[wcohen]

Hi, I working on some code that uses dyninst to check the variable range list information (https://github.com/wcohen/quality_info/blob/master/dyninsttools/locrangechecks.C). Currently the code doesn't deal well with inline functions and this is a concern as inlined code is common when optimization is turned on. The dyninst symtab interface finds parameters and variables based on the function name, but doesn't provide the location lists for the variables for the inlined functions with getParams() or getLocalVariables(). What would be the best way to get the location lists for the inlined function variables?

[wcohen]

Below is an example program to illustrate:

$ cat lto_test.c
#include <stdio.h>

static void f()
{
	int i;
	for(i=0; i<10; i++)
		 printf("%d\n", i);
}

int main(int argc, char **argv)
{
	int j;
	f();
	for(j=0; j<10; j++)
		 printf("%d\n", j);
	return 0;
}

See the variable j for main, nothing for the inlined function f variable i:

$ gcc -g -O2 -o test lto_test.c
$ ./locrangechecks ~/research/profiling/debuginfo/test
# _init [401000,401040]
unable to find printf
# main [401040,40107a]
argc [401040, 401043] x86_64::rdi
argv [401040, 401043] x86_64::rsi
j [401060, 40106c] x86_64::rbx
j [40106c, 401070] x86_64::rsi
j [401070, 401071] -1(x86_64::rbx)
j [401071, 401079] x86_64::rbx
j [401071, 401079] x86_64::rbx
# _start [401080,4010af]
# _dl_relocate_static_pie [4010b0,4010b5]
# deregister_tm_clones [4010c0,4010f0]
# register_tm_clones [4010f0,401130]
# __do_global_dtors_aux [401130,401160]
# frame_dummy [401160,401170]
# __libc_csu_init [401170,4011d5]
# __libc_csu_fini [4011e0,4011e5]
# _fini [4011e8,4011e8]

I should mention when lto is turned on locrangecheck doesn't find the main variable j (gdb does both variable i and j in the lto compiled code):

$ gcc -g -O2 -flto -o lto_test lto_test.c
$ ./locrangechecks ~/research/profiling/debuginfo/lto_test
# _init [401000,401040]
unable to find printf
# main [401040,40107a]
argc [401040, 401043] x86_64::rdi
argv [401040, 401043] x86_64::rsi
# _start [401080,4010af]
# _dl_relocate_static_pie [4010b0,4010b5]
# deregister_tm_clones [4010c0,4010f0]
# register_tm_clones [4010f0,401130]
# __do_global_dtors_aux [401130,401160]
# frame_dummy [401160,401170]
# __libc_csu_init [401170,4011d5]
# __libc_csu_fini [4011e0,4011e5]
# _fini [401
```1e8,4011e8]

[hainest]

@mxz297 @sashanicolas @kupsch @bigtrak We had some discussion about inlined functions in Dyninst some time ago. Does anyone recall if we had any action items on that?

@sashanicolas We should revisit this once we get separate debug and .dwz files working correctly.

[wcohen]

I have found that there is a getInlines() method for symtabAPI. There are some things are that are a bit unclear in the documentation:

Does the returned vector provide a separate entry for each location inlined? So for a function that is inlined multiple times in another function is there an entry in the vector for each of those instances?

How does one handle the case of inlined functions having inlined functions? Just use the getInlines() in the parent inlined function? Or does getInlines() get all the nested inlined functions?

Where does dyinst record the information about where the inlined function entry point (first instruction) in the data structures?

[wcohen]

For the last questiion in the previous comment the documentation mentions Class InlinedFunction getCallsite() method which looks to provide and address. How does one convert the FunctionBase elements in InlineCollection& returned by getInlines() into InlinedFunction class?

[wcohen]

Did some additional experiments with following test code and the readonly.c (https://github.com/wcohen/quality_info/commit/cbbd4a5839cab2a0ef1362cc9117239ea4705338). Compiled the test code on F33 with $ gcc -g -O3 -o test2 lto_test2.c $ cat lto_test2.c #include <stdio.h> #include <stdlib.h>

static int g(int y) { return ++y; }

static inline int f(int x) { return g(x); }

int main(int argc, char **argv) { int j = atoi(argv[1]); int k = atoi(argv[2]); printf("%d\n", f(j)); printf("%d\n", f(k)); return 0; }

Processed with debug output enabled with "-d" and searching for inlined functions:

./readonly test2 -d -f 'process("PATH").function("*").inline'

Below is the beginning output that shows the processing. Recursive decent parsing of the inlines seems to work as shown by the "f->g" and there are entries for each of the inlined copies of function f. However, the offset look to be the offsets of the function they have been inlined into, main. This offset does not seem to be that useful. Is some other place that is suppose to have the inlined function location in the code? The getCallsite() function just return the file name and line number, which is not useful when working on the binary.

filter: process("PATH").function("").inline filter_inline_func: process("PATH").function("").inline inlined: atoi offset: 401050 /home/wcohen/lto_test2.c:16 inlined: atoi offset: 401050 /home/wcohen/lto_test2.c:17 inlined: f offset: 401050 /home/wcohen/lto_test2.c:18 f->g inlined: g offset: 401050 /home/wcohen/lto_test2.c:11 inlined: f offset: 401050 /home/wcohen/lto_test2.c:19 f->g inlined: g offset: 401050 /home/wcohen/lto_test2.c:11

[wcohen]

GCC provides DW_AT_entry_pc addresses for inlined function, but those appear to have incorrect values (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99654).

GCC provides .debug_rnglist section that lists the areas of code associated with each inlined function. It would be useful to have dyninst expose the debug_rnglist information for inlined functions like the getLocationList() method for localVar. Having a list of instructions associated with the inlined function would provide a way of working around the incorrect DW_AT_entry_pc values. It would also open up some additional abilities to do data flow analysis on the inlined function.

The clang compiler provides DW_AT_low_pc and DW_AT_high_pc at times, but also appears to use range lists.

[wcohen]

It looks like the FunctionBase class getRanges() method provides the desired information. This isn't described in the documentation. Doing some sanity check of the data with ranges_sanity (https://github.com/wcohen/quality_info/blob/wcohen/funcranges/dyninsttools/ranges_sanity.C). Earlier attempts to modify the locrangecheck.C to use getRanges() ended up with range vector with 900+ elements for reset_early_page_tables, a relatively simple function, for /usr/lib/debug/lib/modules/5.12.7-300.fc34.x86_64/vmlinux. For the /usr/lib/debug/lib/modules/5.12.11-300.fc34.x86_64/vmlinux ended up with verify_cpu having -1297924.

range_sanity is recursively looking the ranges inlined functions. Each actual function (non-inlined) is preceded by a "=====" . Looking at the output of "./range_sanity /usr/lib/debug/lib/modules/5.12.11-300.fc34.x86_64/vmlinux" See that verify_cpu is a real function with 0 elements in the range list. The fedora 5.12.7-300 kernels reset_early_page_tables only has 1 element. So looks like the issue is in the locrangecheck.C code.

The 0 elements reported for some function made me take a look to see where that was occurring. Some look to be normal C functions like the following in vmlinux 5.12.7:

zbud_zpool_destroy 0 compare_single 0

Others seemed to be cold sections of functions, which would be expected that the region information is stored with the main part of the function:

tick_switch_to_oneshot 2

tick_switch_to_oneshot.cold 0