mandiant
triage binary ninja backend failures
from capa-testfiles, these files fail to process today:
failures for binja:
- 0761142efbda6c4b1e801223de723578.dll_
- 1038a23daad86042c66bfe6c9d052d27048de9653bde5750dc0f240c792d9ac8.elf_
- 112f9f0e8d349858a80dd8c14190e620.exe_
- 253309d8b3675d3cc61d4bf23aa15d4b.dll_
- 2dae11cc5f86f5399b560b8837c26274b7e09431deed669b0844fef44e917915.exe_
- 2f7f5fb5de175e770d7eae87666f9831.elf_
- 3b13b6f1d7cd14dc4a097a12e2e505c0a4cff495262261e2bfc991df238b9b04.dll_
- 4122acca2f9ea98fc3f3ad040688e4ce.exe_
- 44d40faf3f1fe4ed969befab7afcd2f0.exe_
- 49a34cfbeed733c24392c9217ef46bb6.exe_
- 54390bda109aab7fc006b8b4ead5b6c2.dll_
- 55d77ab16377a8a314982f723fcc6fae.exe_
- 5f66b82558ca92e54e77f216ef4c066c.exe_
- 5fbbfeed28b258c42e0cfeb16718b31c.exe_
- 648fc498110b11b4313a47a776e6ba40.exe_
- 6cc148363200798a12091b97a17181a1.exe_
- 7f15b1a47bbe031334e23653879e9661f4b8cde80c307548328fdd3aed87ca46.exe_
- 91a12a4cf437589ba70b1687f5acad19.exe_
- 92d8ea10ea30e8b534334a1c9857a455.exe_
- 94d3c854aadbcfde46b2f82801015c31.exe_
- 971e599e6e707349eccea2fd4c8e5f67.exe_
- 9b7ccaa2ae6a5b96e3110ebcbc4311f6.dll_
- 9ff8e68343cc29c1036650fc153e69f7.exe_
- a74ee8200aace7d19dee79871bbf2ed3.exe_
- a933a1a402775cfa94b6bee0963f4b46.dll_
- b5f0524e69b3a3cf636c7ac366ca57bf5e3a8fdc8a9f01caf196c611a7918a87.elf_
- c3341b7dfbb9d43bca8c812e07b4299f.exe_
- e87076a1182ba40758e1d7258442c1ee23bf71ac77fad9f3babc707dce11c144.exe_
- eaad7dfc78304b977d3844cc63577152.exe_
- ee3b869b668abec332d07c66d1a39f6dbf3a598cc1325b57a0504f8d24ac2e28.dll_
- kernel32.dll_
- mimikatz.exe_
I tested 112f9f0e8d349858a80dd8c14190e620.exe_ and it seems to be just slow:
I will look into other samples when possible as well
Well I see it crashes now:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Users/xusheng/capa/capa/main.py", line 1103, in <module>
sys.exit(main())
^^^^^^
File "/Users/xusheng/capa/capa/main.py", line 994, in main
capabilities, counts = find_capabilities(rules, extractor, disable_progress=args.quiet)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xusheng/capa/capa/capabilities/common.py", line 75, in find_capabilities
return find_static_capabilities(ruleset, extractor, disable_progress=disable_progress, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xusheng/capa/capa/capabilities/static.py", line 168, in find_static_capabilities
function_matches, bb_matches, insn_matches, feature_count = find_code_capabilities(ruleset, extractor, f)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xusheng/capa/capa/capabilities/static.py", line 114, in find_code_capabilities
for bb in extractor.get_basic_blocks(fh):
File "/Users/xusheng/capa/capa/features/extractors/binja/extractor.py", line 58, in get_basic_blocks
for mlil_bb in f.mlil.basic_blocks:
^^^^^^
File "/Applications/Binary Ninja.app/Contents/Resources/python/binaryninja/function.py", line 1039, in mlil
raise ILException(f"Medium level IL was not loaded for {self!r}")
binaryninja.exceptions.ILException: Medium level IL was not loaded for <func: x86@0x467464>
This is quite similar to https://github.com/mandiant/capa/issues/2249
With the fix https://github.com/mandiant/capa/commit/9dd8b7a86448ff7882966347c755631b4c589f9b, it is no longer crashing due to the IL not being loaded. However, it gets to a different crash:
lot of lines omitted
......
DEBUG capa.capabilities.static: analyzed function 0x7a83b0 and extracted 13 features, 0 matches in 0.01s static.py:179
DEBUG capa.capabilities.static: analyzed function 0x7a83c0 and extracted 13 features, 0 matches in 0.00s static.py:179
DEBUG capa.capabilities.static: analyzed function 0x7a83d0 and extracted 27 features, 0 matches in 0.01s static.py:179
DEBUG capa.capabilities.static: analyzed function 0x7a8401 and extracted 10 features, 0 matches in 0.00s static.py:179
DEBUG capa.capabilities.common: analyzed file and extracted 96040 features common.py:35
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Users/xusheng/capa/capa/main.py", line 1101, in <module>
sys.exit(main())
^^^^^^
File "/Users/xusheng/capa/capa/main.py", line 995, in main
meta.analysis.layout = capa.loader.compute_layout(rules, extractor, capabilities)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xusheng/capa/capa/loader.py", line 663, in compute_layout
return compute_static_layout(rules, extractor, capabilities)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xusheng/capa/capa/loader.py", line 641, in compute_static_layout
assert addr in functions_by_bb
^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
@williballenthin I tried a few other files, and it seems capa runs fine on it with binja backend. I am curious when you say "fail to process", does it just crash like for 112f9f0e8d349858a80dd8c14190e620.exe_, or it fails to pass certain unit tests?
The "IL cannot be loaded" also happens with b5f0524e69b3a3cf636c7ac366ca57bf5e3a8fdc8a9f01caf196c611a7918a87.elf_ at 0x8091b80
Weird enough, if I add --restrict-to-functions 0x8082d40 to restrict the analysis to the offending function, then everything is fine
I am triaging these and will update the status here. Please be aware the test is down on top of the recent fixes #2500 .
| Sample | Status |
|---|---|
| 0761142efbda6c4b1e801223de723578.dll_ | ✅ |
| 1038a23daad86042c66bfe6c9d052d27048de9653bde5750dc0f240c792d9ac8.elf_ | ✅ |
| 112f9f0e8d349858a80dd8c14190e620.exe_ | ✅ |
| 253309d8b3675d3cc61d4bf23aa15d4b.dll_ | ✅ |
| 2dae11cc5f86f5399b560b8837c26274b7e09431deed669b0844fef44e917915.exe_ | ✅ |
| 2f7f5fb5de175e770d7eae87666f9831.elf_ | Crash like https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 |
| 3b13b6f1d7cd14dc4a097a12e2e505c0a4cff495262261e2bfc991df238b9b04.dll_ | ✅ |
| 4122acca2f9ea98fc3f3ad040688e4ce.exe_ | packed |
| 44d40faf3f1fe4ed969befab7afcd2f0.exe_ | ✅ |
| 49a34cfbeed733c24392c9217ef46bb6.exe_ | ✅ |
| 54390bda109aab7fc006b8b4ead5b6c2.dll_ | ✅ |
| 55d77ab16377a8a314982f723fcc6fae.exe_ | AuotIt |
| 5f66b82558ca92e54e77f216ef4c066c.exe_ | ✅ |
| 5fbbfeed28b258c42e0cfeb16718b31c.exe_ | ✅ |
| 648fc498110b11b4313a47a776e6ba40.exe_ | Crash like https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 |
| 6cc148363200798a12091b97a17181a1.exe_ | ✅ |
| 7f15b1a47bbe031334e23653879e9661f4b8cde80c307548328fdd3aed87ca46.exe_ | Crash like https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 |
| 91a12a4cf437589ba70b1687f5acad19.exe_ | packed |
| 92d8ea10ea30e8b534334a1c9857a455.exe_ | AutoHotKey |
| 94d3c854aadbcfde46b2f82801015c31.exe_ | hits a binja bug https://github.com/Vector35/binaryninja-api/issues/6191, but capa finishes fine |
| 971e599e6e707349eccea2fd4c8e5f67.exe_ | packed |
| 9b7ccaa2ae6a5b96e3110ebcbc4311f6.dll_ | Crash like https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 |
| 9ff8e68343cc29c1036650fc153e69f7.exe_ | ✅ |
| a74ee8200aace7d19dee79871bbf2ed3.exe_ | ✅ |
| a933a1a402775cfa94b6bee0963f4b46.dll_ | ✅ |
| b5f0524e69b3a3cf636c7ac366ca57bf5e3a8fdc8a9f01caf196c611a7918a87.elf_ | Crash like https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 |
| c3341b7dfbb9d43bca8c812e07b4299f.exe_ | Crash like https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 |
| e87076a1182ba40758e1d7258442c1ee23bf71ac77fad9f3babc707dce11c144.exe_ | ✅ |
| eaad7dfc78304b977d3844cc63577152.exe_ | ✅ |
| ee3b869b668abec332d07c66d1a39f6dbf3a598cc1325b57a0504f8d24ac2e28.dll_ | ✅ |
| kernel32.dll_ | the initial binja analysis runs longer than expected (https://github.com/Vector35/binaryninja-api/issues/6177), but eventually capa is able to complete the analysis |
| mimikatz.exe_ | ✅ |
fyi probably some of these are .NET files for which we don't expect Binja to do very much (and a failure is fine).
fyi probably some of these are .NET files for which we don't expect Binja to do very much (and a failure is fine).
Thx for letting me know! I am now only looking for crashes, and even if it is .NET files the analysis should not crash
I have triaged all of the files and here is a brief recap:
- Most of the files are just fine with the fix in PR #2500
- A handful of them now crashes as mentioned in https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478. This is a new crash discovered only after the original crash is fixed. This crash is still related to the IL being unavailable, and it seems to break some assumptions made by capa. Also, interestingly, I tested if I run capa with binja >= dev 4.3.6482, which has the fix to https://github.com/Vector35/binaryninja-api/issues/6171, then the crash is gone since the IL will always be available (or not). In other ways, the new crash is caused by the IL initially available, but become unavailable afterwards. Technically, this can be avoided if we tell people to use the dev version, but I want capa be able to run with the stable binja, so I will look into it and see what is happening
- There are a few packed binaries, as well as AutoIt, AutoHotKey ones that are excluded
In other words, the newly noticed crash in 2) is the only thing remaining for this issue to resolve
So the failure in https://github.com/mandiant/capa/issues/2406#issuecomment-2490264478 is indeed due to https://github.com/Vector35/binaryninja-api/issues/6222. Which is quite obscure, and I will look into it.
It is also not caused by https://github.com/mandiant/capa/issues/2516