Patching native methods

[mnns]

Hi, is it possible to add before/after code on native calls (Pinvoke)?

Thanks

[Windows10CE]

Yes, you can use NativeDetour, the docs for which are here https://monomod.dev/api/MonoMod.RuntimeDetour.NativeDetour.html

Just a warning though that using MonoMod detours on P/Invokes can be... odd in ways you might not expect. I can't list any of them off the top of my head, just keep it in mind if it starts causing problems.

[nike4613]

Expanding more on what @Windows10CE said, patching P/Invokes portably is not really something that MonoMod supports at the moment. If you know your target runtime, OS and architecture, you should be safe (and frankly, I'm not sure the OS and arch matter all that much). With that said, here are some of the known caveats:

• On CoreCLR (and I believe Framework) the runtime shares IL stubs by signature, and the entrypoint MonoMod finds for a P/Invoke is that IL stub. This means that 1. the patch won't be applied when calling from native code, and 2. will be invoked for every P/Invoke with that signature.
• Older Framework doesn't use IL stubs, but instead a (large) collection of assembly stubs. I suspect that MonoMod finds the address of those, and patching that would lead to the same issue as above.
• I'm not sure how Mono does P/Invoke, but it doesn't have the same kinds of stubs that CoreCLR and Framework use, so it may generate the marshalling code at the callsite in older versions. I'm fairly sure that recent Mono does a very similar thing to CoreCLR though.
• If you do patch the actual native function instead of the P/Invoke stub, you also need to make sure that the runtime actually transitions back into the correct GC mode, because otherwise it will collect objects that shouldn't be collected and cause all kinds of hard to diagnose issues.
  • If you're on .NET 5 or higher, using UnmanagedCallersOnly and making the patch target a normal function pointer to that method should be enough.
  • On older runtimes, you'll want the patch target to be the result of something like Marshal.GetFunctionPointerForDelegate, as that returns a wrapper which transitions the GC accordingly before calling the delegate.
  • Once you have done that though, you can't call the original without quite a bit more work, especially if it might need to be thread safe.

---

All of that being said, in the (hopefully) near future, NativeDetour will be removed as part of my rewrite, and at some point I will attempt to create a sane way to detour native methods. This will absolutely have the same requirements as UnmanagedCallersOnly (that is, blittable parameters and return type), and handle all of the funky requirements of detouring native methods.

[mnns]

Thanks for the great answers. Maybe you guys can help me with a general direction, what I'm trying to achieve is: I have a C# game that's writing logs to file system.

Oddly enough it uses a managed C# library (#1, "LogManager.dll") that in turn uses a native library (#2, "PrimLogger.dll") to do so. Internally, the "PrimLogger.dll" uses Win32 API CreateFile().

I want to be able to understand, during runtime, which C# managed library invoked any native library that uses Win32 API CreateFile() or ReadFile() API for example. I also want to block this behaviour (or detour if possible, maybe even returning different values).

I tought about maybe writing a program that hooks CreateFile() and ReadFile() on the CLR process (which has the native library loaded?). Problem is I'm not sure how to pinpoint which managed library called the native library that invoked this function.

Thanks a lot.

[nike4613]

This is now fully possible with the NativeHook type in the reorganize branch, though you will need to get a pointer to the function you want to invoke yourself. There are currently no utilities to get one from a P/Invoke method, though that could (and probably should) be added at some point.