returning the procedure's Local Variables ?

[dcabale]

Hey there, RADStudio Memory management documentation says this about procedure's local variables:

These "Local Variables" can be accessed, using the IDE debugger, with a [Name, Value] pair list, such as this :

The application Stack is accessible by FastMM5, at the memory allocation event. Considering this, would FastMM5 be able, on that event, to return also these Local Variables, just as the IDE debugger does? Thanks for your support.

[pleriche]

Theoretically it would be possible to walk the stack backwards to the calling procedure and obtain the values of the local variables that are stored on the stack, but the local variables held in registers would not be accessible.

Practically though it's not really feasible. Obtaining the local variables would be computationally expensive. If the local variable values are required later they would need to be copied, increasing memory overhead as well. Finally, when it comes to interpreting the local variables there would have to be code that is able to convert the local variable values to human readable form.

[dcabale]

1. Thanks for your quick answer.
2. Please excuse my ignorance on this, but what fundamental difference does exist, at the time of a memory block allocation, between what is available to FastMM and what is available to the RADStudio IDE, displaying the Local Variables table? How does the IDE read the registers ?
3. Let me put aside your computer-expensiveness argument, as my goal would be to analyze what the application does, and thus set in Debug configuration only.
4. About the local variables interpretation in human readable form : I suppose this could be done with the help of the .map file, couldn't it? Again, the IDE does that.

[pleriche]

(2) The debugger displays the variables for the code block in which the debugger has stopped. At that point the registers contain the values for the local variables. If I understand you correctly you want FastMM to capture the local variables for the routine that called into FastMM (via GetMem, FreeMem, etc.). At the point that FastMM gains control the registers will contain the parameters for the e.g. GetMem request. FastMM would have to walk the call stack backwards through all the callers and collect whatever local variables it can get off the stack. Many of the local variables would have been in registers before the call sequence, thus not available anymore.

(4) The IDE uses debugger visualizers. They are many and they can be complex. Consider for example records, you would probably want to drill down into the fields for them. It's not a simple task to emulate this runtime.

I'm sure it will be possible to do something in line with what you're asking, but the quality of the results may disappoint due to (2), with the consequence that doing (4) would probably not be a good investment.

[dcabale]

1. FastMM would have to walk the call stack backwards through all the callers and collect whatever local variables it can get off the stack.

Correct.

2. Many of the local variables would have been in registers before the call sequence, thus not available anymore.

I'm not following you there : the local variables should be stored in the stack for each called function: from RADStudio Memory management documentation and from Memory Management: Frame Allocation Is that not correct ? What is the "registers" you are referring to ?

3. If someone would ask you to do it, how would you emulate IDE-like debugger /variable visualizer?

Thanks for your help.

[pleriche]

What is the "registers" you are referring to ?

I am referring to the CPU registers. With optimization enabled the CPU registers will be used to store many local variables (instead of the stack). Before a subroutine is called the local variables of the caller that are held in registers will either be pushed onto the stack or shuffled to non-volatile registers if they need to persist beyond the call, or otherwise they will just be discarded. Determining the local variables of the caller from the callee will thus be very hard, and in many cases impossible.

[dcabale]

Thanks for your reply

1. It is no question to "enable optimization" when it stands for program analysis. => I expect the local variables to be held in the stack frame, not in CPU registers
2. How is that the Delphi IDE can retrieve the caller's local variables, when the program is stopped at a callee's breakpoint ?

[pleriche]

With optimization disabled the local variables will be on the stack, and in that case it would be possible to collect the local variables for all the callers, provided you have access to all the debug information the debugger has.

Assuming you are able to get to all the local variables of the caller routine like the debugger does, you would still face the issue of how and where you're going to store all those values in case you need to display them later in a crash dump. If you were to capture all the local variables of the entire call stack every time GetMem, FreeMem, etc. is invoked you would run out of memory very quickly.

The point I made in my first response stands: It's too expensive both computationally as well as in storage required to make adding capturing of local variables together with stack traces practical. It's also not a small development undertaking to develop the code to be able to present captured local variable values in human readable form.

[dcabale]

Let me precise my goal: my goal is to use FastMM as a back-bone for building my own call-graph application, as a code-comprehension tool Here is how I imagine the graph display:

• Nodes would represent TObject's (or descendants) instances or methods. Big nodes for instances (constructors), and small nodes for methods.
• Relations (arrows) would link the nodes: the origin of the relation is the caller method; the destination of the relation is the called mathod or, if that method is a constructor, a new TObject's (or descendants) instance. For doing that:
• call-stack snapshots would be taken at each GetMem(), just as FastMM does so far.
• Each call in the call-stack would be set as a node : as said, small nodes for methods and large nodes for TObject's instances.
• The stack-frames (the method's local variables) would be used as attributes for the small nodes (form of key /value pairs).
• TObject's public members would be used as attributes for the large nodes (member's name and type)

But let's come back to your arguments:

1. "issue of how and where you're going to store all those values in case you need to display them later in a crash dump." My intent is not to use that feature in a crash dump. As I said before, I would use FastMM as a back-bone of my program.
2. "all the local variables of the entire call stack every time GetMem, FreeMem, etc. is invoked you would run out of memory very quickly".."It's too expensive both computationally as well as in storage required to make adding capturing of local variables together with stack traces practical" Correct. Given this, parts of the application only could be zoomed in. Could be only a single method. How ? (just an idea) we could use the "Allocation Group" feature that would filter the allocated blocks, minimizing the overhead. But I believe you better than me could bring the best solution for this issue.
3. "It's also not a small development undertaking to develop the code to be able to present captured local variable values in human readable form." The IDE already does that. May not Embarcadero help us with this?

Anyway, if you consider that FastMM cannot be held as the back-bone for such a tool, then what would you advise to satisfy my need ?

[pleriche]

What you're describing would be a great debugging tool, if you can pull it off. In my opinion that is a big "if". You have to ask yourself why this has not been done before. I believe the answer lies in the issues I have touched on in my previous responses, and likely additional stumbling blocks that may only be revealed once you try to get it off the ground.