Regression in 5.1.7379-dev (f55a1eef) breaks propagation of return types when overriding a call type for calls inlined from a function

[WeiN76LQh]

Version and Platform (required):

• Binary Ninja Version: 5.1.7379-dev (f55a1eef)
• OS: macOS
• OS Version: 15.2
• CPU Architecture: M1

Bug Description: Prior to the regression call type overrides applied via the API or through the UI as a user, would work as expected. However version 5.1.7379-dev kind of broke them and now calls from inlined functions do not propagate the return type of the applied call type override at the site where the function was inlined. This is specifically an issue with calls in functions that have been inlined. Overrides elsewhere seem to work fine.

Steps To Reproduce: Please provide all steps required to reproduce the behavior:

1. Load a copy of DYLD Shared Cache.
2. Load a library like FeatureStore or anyone that makes calls to stub functions.
3. Find a call to a stub function. I've been testing with _objc_alloc.
4. Try to apply a call type override to it with a different return type and observe that the variable being assigned the result of the _objc_alloc doesn't change to use the return type of the call type override. You might need to be in MLIL to observe a variable assignment, often in HLIL the result of the call is passed directly to another call.
5. Now stop the function being inlined by going to j__objc_alloc and editing its function properties and turning off function inlining. I find often times toggling this setting doesn't work. Sometimes it requires multiple attempts but usually changing the function workflow from inherited to the same workflow just not the inherited option, at the same time as unticking the function inlining checkbox, will actually get the change to stick.
6. Try setting a call type override, with a different return type, to the j__objc_alloc call and observe that the variable being assigned the result will take on the return type of the call type override.

Expected Behavior: Overridding the type of a call in an inlined function (at the location where its been inlined, not the actual function) should result in the return type being propagated to the variable its being assigned to.

Additional Information:

• This is not a problem for non-inlined calls (calls that aren't from inlined functions).
• In 5.1.7372-dev (c3488220) things worked as expected. I originally noticed this issue in the latest version (was 5.1.7399 at the time) and went back and found where the regression occurred so its still broken on latest.

[WeiN76LQh]

Actually I'm finding on latest (5.1.7400-dev (6eeff4a8)) even for non-inlined functions, the call type override is there (if I go to change it I see it has updated from being changed before), but the return type isn't being propagated to the variable its being assigned to. Manually re-applying the call type override as a user does nothing. I have to manually set the variable's type.

Although there seems to be some inconsistency because sometimes it does work for non-inlined calls.

[bdash]

Thanks for filing this! I had noticed myself that this wasn't working but was still in the "am I doing something wrong" phase so hadn't gotten around to writing up a bug report.

[WeiN76LQh]

I believe there is a related issue I'm also seeing in MLIL where direct assignments are not propagating types as I'd expect. I have a variable that is assigned a value and then used as an argument to a call. Thats its entire existence. Instead of taking on the type of the value it is being assigned it is choosing the type of the argument for the call it is passed to. If I override the call type, the variable will adjust its type according to the argument of the call type override.

I can kind of see the logic here but shouldn't the type of the assigned value take precedence? I made sure the value it was being assigned had a type specified by the user and that didn't help either. It will only ever take on the type of the call parameter unless I explicitly set the variable's type.

[WeiN76LQh]

Another thing I'm seeing, that may or may not be part of the issue, is that modifying IL, specifically MLIL to change the target of a call is not updating the call type. So I'll notice a variable which is assigned the return result of a modified MLIL call will have the type of the original call target's return type. If I then go to override the call type then that will also show the type of the original call target. If I then override the call type with the updated call target's type then the variable will end up with the expected return type.

Thinking this might be an unrelated issue that I'm just also observing now I modified my plugin to not always override call types. Also I'm unsure if this is occurring for all instances or just some.

[plafosse]

@emesare This seems to correspond to

commit f55a1eef2bb62212a934c01058b2a32faddbbcec
Author: Mason Reed <[email protected]>
Date:   Sun May 4 21:20:32 2025 -0400

    Bump API (skip-note)

which corresponds with

commit 178887ba4d503838c444a0c976ac5538117fd2fb
Author: Mark Rowe <[email protected]>
Date:   Sat May 3 23:19:08 2025 -0700

    [Rust] Fix a pre-existing formatting issue
    
    CI is complaining about it.

commit 9bd1a1c5d54f46e250274538e8faee23d5520a88
Author: Mark Rowe <[email protected]>
Date:   Sat May 3 22:39:50 2025 -0700

    [Rust] Avoid leaking a reference within {Medium,High}LevelILFunction::ssa_form
    
    Return a Ref<_> so that the underlying core object will have its
    reference count decremented when the caller drops the object.

commit c9253993ee4085880468cf9345519f45adfa77a3
Author: Daniel Roethlisberger <[email protected]>
Date:   Sat May 3 14:00:57 2025 +0200

    [SharedCache] Avoid reading header fields outside of the bounds of the header
    
    Use mappingOffset as an upper bound for the header size, and avoid
    reading any header fields from beyond that offset.
    
    Co-authored-by: Mason Reed <[email protected]>

I'm not sure if this is related IDK how it could be.

[emesare]

Cannot seem to get it to repro by applying a call type override to an inlined function, both you and @bdash seem to be having this problem so it is clearly an issue I just must be doing something wrong. If you have time to create a simple video showcasing the issue, or some instructions like "go to 0xfoo and change the return type in call type adjustment" I can try again to reproduce.

https://github.com/user-attachments/assets/3c853049-fb5b-47f8-bd0f-edaff49ca52d

[WeiN76LQh]

I just followed the instructions in the original post and observed the issue, here's a more detailed set of instructions:

• Currently on Binary Ninja 5.1.7500-dev (1da2316a) for macOS.
• Analyzing a copy of iOS 18.1.1 for an iPhone 16 Pro Max (iPhone17,2_18.1.1_22B91 downloadable here).
• I have the DSC extracted and put into a project.
• I double click open the main DSC file.
• I didn't wait until initial auto-analysis had completed and loaded the FeatureStore library (/System/Library/PrivateFrameworks/FeatureStore.framework/FeatureStore).
• I waited for analysis to complete.
• I went to the function +[FSFCurareInteractionStream getWithStreamId:sourceType:] at address 0x21b21b48c.
• There is only one call to _objc_alloc in that function at MLIL index 12 with the address 0x21fc02c18.
• The variable it was assigning to had the type void*.
• In MLIL I right-click override call type on the line the call is made.
• The type was void calltarget() and I changed it to uint64_t calltarget().
• The variable type remained void*.
• I override the call type again and changed it to some more accurate uint64_t* _objc_alloc(void*).
• The variable type remained void*.
• I re-analyzed the function and again the variable type did not change.
• I modified the return type of _objc_alloc and observed no change to the variable type.
• There is a singular call to _objc_msgSend within the function that takes the result of the _objc_alloc call as its first argument (self).
• In MLIL I right-click override call type on the line the call is made.
• The type was void* calltarget(void* self, uint8_t* sel, int64_t initWithStreamId, int64_t sourceType) and I changed it to void* calltarget(char* self, uint8_t* sel, int64_t initWithStreamId, int64_t sourceType).
• The variable type that is assigned the result of _objc_alloc changed from void* to char*.

Hopefully thats enough to replicate what I'm experiencing, let me know if not.

[emesare]

I can confirm the issue with your detailed repro steps, however I am not sure this is a regression, I went and tested 5.0 and the behavior is also as described, and inspecting the commits through the time period in which you noticed this issue yields no real leads. I also noticed some peculiar behavior where i was initially having the override work and it all the sudden stopped working, this was not loading from a database and I was swapping builds in-between, could be some race condition i started to constantly lose, not sure.

Anyways, seeing as I could manually set the returned variable type consistently I am going to bump this down to impact medium and remove the regression tag, seeing as I could reproduce on 5.0, and 4.2 was basically frozen when i tried to load.