ghidra
ghidra copied to clipboard
Ability to have more than `Byte.MAX_VALUE` calling conventions
Subj. This can be used to generate custom calling conventions with different input and output destinations.
For now you cannot apply more than 127 calling conventions using HighFunctionDBUtil.writeOverride
, it prints
ERROR Unable to assign calling convention `__r39_r56` - allocation capacity exceeded (CallingConventionDBAdapterV0)
That seems rather excessive and is likely abusing the intention of reuseable "named" calling conventions. I assume this is being done to customize storage on a function-by-function basis or do you intended on reusing each named convention to multiple functions? Our intent is to add custom storage capability to Function Definitions in a manner similar to that of Function.
I am reaching a similar issue here reverse engineering hand-generated code; so without a consistent calling "convention".
The information from the "calling convention" I do not see in the custom storage is the list of affected (and symmetrically, unaffected) registers; from which the true output registers can be drawn (using the custom storage).
A large number of calling convention allows for a fine-grained analysis, without which I would be missing the complete effect of the function (returning a structure across multiple registers) or on the contrary been swamped with non-existant side-effects.
But in my case being able to set the killed by / unaffected by lists on a function by function basis would also solve my issue.
For taylored situations as you describe (e.g., pure assembly code) we are evaluating an approach which would allow a function-specific tayloring of affected/unaffected registers, etc.. We are also evaluating similar storage-specific details for function definitions (i.e., data type). We prefer not to blow-out the number of defined calling conventions used at this point in time.