IDA-Pro-SigMaker icon indicating copy to clipboard operation
IDA-Pro-SigMaker copied to clipboard

Is it just me or is making XREF sigs painfully slow?

Open belmeopmenieuwesim opened this issue 1 year ago • 13 comments

It is working great for me, only for the exception that making XREF sigs are painfully slow. I just had to wait more than 15 minutes for it to process 70 XREFs. Is this normal?

Binary size is about 50MB.

belmeopmenieuwesim avatar Aug 03 '24 11:08 belmeopmenieuwesim

FindSignatureOccurences should probably return early whenever it detects more than 1 hit. That would speed it up already by a bunch.

belmeopmenieuwesim avatar Aug 03 '24 12:08 belmeopmenieuwesim

I was able improve performance by more than 20x by using SIMD and doing the optimization trick in previous comment.

belmeopmenieuwesim avatar Aug 04 '24 04:08 belmeopmenieuwesim

That's actually how I did it in my first implementation: https://github.com/A200K/IDA-Pro-SigMaker/blob/2fa5492953f3ca9fc60bceed315627183545f97f/IDA%20Pro%20SigMaker/Main.cpp#L50

I didn't think it would do that much of a difference and I wanted to clean up the code, but that's mostly because I usually don't have big database / binary sizes. Thanks for the feedback, I'll consider your suggestion!

A200K avatar Aug 05 '24 01:08 A200K

@belmeopmenieuwesim I implemented your fix in my latest release. Would you mind comparing the performance improvement to your SIMD optimization? What exactly did you do?

It was really unusably slow. How didn't anyone else mention this before lol

A200K avatar Aug 05 '24 02:08 A200K

@A200K the SIMD code seems to be on here: https://github.com/A200K/IDA-Pro-SigMaker/compare/main...belmeopmenieuwesim:IDA-Pro-SigMaker:main

w00tzenheimer avatar Aug 06 '24 00:08 w00tzenheimer

That‘s really cool! I didn’t think there was that much room for improvement, compared to the IDA API. I won‘t be able to work on the project for a few weeks, but I‘ll definitely try to look further into/implement/add this when I can.

Thanks for the input @belmeopmenieuwesim @w00tzenheimer !

A200K avatar Aug 06 '24 19:08 A200K

planning to add 'make signature from selected code' option?

GCHE1 avatar Aug 07 '24 11:08 GCHE1

also i tested this plugin vs sigmakerEx, am I supposed to change some settings to make it work properly?

on this plugin https://i.imgur.com/LntijOe.png result: Error: Signature left function scope with option "continue when leaving function scope" signature is very huge

SigMakerEx: Finding signature for 0049441F. Address SIG: 0x0049441F, 23 bytes 8, wildcards. IDA: "E8 ?? ?? ?? ?? E8 ?? ?? ?? ?? C7 45 FC FF FF FF FF 85 F6 74 0B 6A 04" "\xE8\xCC\xCC\xCC\xCC\xE8\xCC\xCC\xCC\xCC\xC7\x45\xFC\xFF\xFF\xFF\xFF\x85\xF6\x74\x0B\x6A\x04", "x????x????xxxxxxxxxxxxx"

GCHE1 avatar Aug 07 '24 12:08 GCHE1

@GCHE1 „Copy selected Bytes“ option should be the closest thing to „make signature from selected code“

That‘s odd. You could try the „Operand types“ button and uncheck everything except for the DATA and CODE type operands in order to restrict wildcarding and achieve shorter signatures (at potential cost of signature stability over binary revisions).

If that doesnt help - would you mind sharing your binary, or at least some more screenshots of the code, as well as the first 30 Bytes my plugin created?

A200K avatar Aug 07 '24 17:08 A200K

@GCHE1 „Copy selected Bytes“ option should be the closest thing to „make signature from selected code“

That‘s odd. You could try the „Operand types“ button and uncheck everything except for the DATA and CODE type operands in order to restrict wildcarding and achieve shorter signatures (at potential cost of signature stability over binary revisions).

Hello,

Unchecking "Immediate Value" allows me to generate signature, result:

Your Plugin: Match @ 49441F Signature for 49441F: \xE8\x00\x00\x00\x00\xE8\x00\x00\x00\x00\xC7\x45\x00\x00\x00\x00\x00\x85\xF6\x74\x00\x6A\x04 x????x????xx?????xxx?xx So, the provided signature has 23 bytes and 13 wildcards.

SigMakerEx: Finding signature for 0049441F. Address SIG: 0x0049441F, 23 bytes 8, wildcards. IDA: "E8 ?? ?? ?? ?? E8 ?? ?? ?? ?? C7 45 FC FF FF FF FF 85 F6 74 0B 6A 04" "\xE8\xCC\xCC\xCC\xCC\xE8\xCC\xCC\xCC\xCC\xC7\x45\xFC\xFF\xFF\xFF\xFF\x85\xF6\x74\x0B\x6A\x04", "x????x????xxxxxxxxxxxxx"

https://pastebin.com/fQDR19AM

If you still need this binary, let me know where can I send it

GCHE1 avatar Aug 07 '24 20:08 GCHE1

Seems fine to me, to be honest. Might change the default settings for the operands in the future though.

A200K avatar Aug 08 '24 10:08 A200K

Seems fine to me, to be honest. Might change the default settings for the operands in the future though.

Seems fine for me too, thanks for help and have a great day Sir.

GCHE1 avatar Aug 08 '24 21:08 GCHE1

@A200K The AVX2 signature scanner over here https://github.com/qis/signature is even faster (even without TBB!) than the hacked together SIMD implementation I made inside my fork. For me atm it is sufficient. But if you really want to go max performance, you can incorporate https://github.com/qis/signature into your project. It is single header and code is very nice.

belmeopmenieuwesim avatar Aug 12 '24 09:08 belmeopmenieuwesim