BruteShark icon indicating copy to clipboard operation
BruteShark copied to clipboard

Published 20 hours ago • verified publisher icon odedshimon

BruteSharkCli is killed when trying to use network map module on a directory of pcaps, no output created

Open syloktools opened this issue 3 years ago • 2 comments

Output from terminal:

xxx@xxx:/xxx$ sudo ./BruteSharkCli -m NetworkMap -d /xxx/data/packets/servers/dailylogs/2019-05-02/ -o /xxx/results [+] Start analyzing 11 files [+] Start processing file : daemonlogger.pcap.1556805601 [+] Finished processing file : daemonlogger.pcap.1556805601 [+] Start processing file : daemonlogger.pcap.1556816401 [+] Finished processing file : daemonlogger.pcap.1556816401 [+] Start processing file : daemonlogger.pcap.1556820001 [+] Finished processing file : daemonlogger.pcap.1556820001 [+] Start processing file : daemonlogger.pcap.1556803424 [+] Finished processing file : daemonlogger.pcap.1556803424 [+] Start processing file : daemonlogger.pcap.1556809201 [+] Finished processing file : daemonlogger.pcap.1556809201 [+] Start processing file : daemonlogger.pcap.1556830801 [+] Finished processing file : daemonlogger.pcap.1556830801 [+] Start processing file : daemonlogger.pcap.1556827201 [+] Finished processing file : daemonlogger.pcap.1556827201 [+] Start processing file : daemonlogger.pcap.1556834401 Killed

If I run it against one file: xxx@xxx:/xxx$ ./BruteSharkCli -m NetworkMap -i /xxx/data/packets/servers/dailylogs/2019-05-02/daemonlogger.pcap.1556812801 -o /xxx/ [+] Start analyzing 1 files [+] Start processing file : daemonlogger.pcap.1556812801 [+] Finished processing file : daemonlogger.pcap.1556812801 [+] Successfully exported network map to json file: /xxx/resultsBruteShark Network Map.json [+] Successfully exported network nodes data to json file: /xxx/BruteShark Network Nodes Data.json [+] Successfully exported extracted files to: /xxx/Files [+] BruteShark finished processing

Size of directory is 912M. Is there a limit. System has 8 gig of memory.

syloktools avatar Jan 15 '22 08:01 syloktools

Hi @robertnixon2003 ! Thanks for creating this issue. There is no built-in limitation at BruteShark, also there is no log containing the phrase "killed". Therefore I tend to believe it some kind of operating system lack of resource.

I can suggest few ways to investigate the issue:

  1. Make sure the folder exists and it has write permissions.
  2. Run all files one by one to ensure that this behavior is not related to a specific file.
  3. Clone this project and run it at debug mode (for accurate exception and stack trace).

Feel free to contact with any further questions

odedshimon avatar Jan 15 '22 21:01 odedshimon

Hi guys, I also encountered the same problem. See below

jan 26 19:56:05 qa-br-vostro kernel: Tasks state (memory values in pages):
jan 26 19:56:05 qa-br-vostro kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
jan 26 19:56:05 qa-br-vostro kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/[email protected]/app.slice/vte-spawn->
jan 26 19:56:05 qa-br-vostro kernel: Out of memory: Killed process 16242 (BruteSharkCli) total-vm:10105664kB, anon-rss:4961504kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:10280kB oom_score_adj:0
jan 26 19:56:05 qa-br-vostro kernel: oom_reaper: reaped process 16242 (BruteSharkCli), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
jan 26 19:56:05 qa-br-vostro systemd[1]: [email protected]: A process of this unit has been killed by the OOM killer.
jan 26 19:56:05 qa-br-vostro systemd[1504]: vte-spawn-48f92492-8cbe-4dec-90b7-ef408d10d774.scope: A process of this unit has been killed by the OOM killer.

File:
-rw-r--r-- 1 g0043780 g0043780 385M jan 26 17:01 Boot-2601-all.pcapng

cmd:
./BruteSharkCli -i ../Plataformas/Boot-2601-all.pcapng -o ../Plataformas/

 ./BruteSharkCli --version
BruteSharkCli 1.0.0.0

free -h
               total        used        free      shared  buff/cache   available
Mem.:          7,6Gi       4,4Gi       217Mi       706Mi       3,1Gi       2,3Gi
Swap:          975Mi       480Mi       495Mi

uname -a
Linux qa-br-vostro 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux

Strace (last lines):

sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3322908672, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3322908672, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 59
fstat(59, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(59, "MemTotal:        8013480 kB\nMemF"..., 1024) = 1024
close(59)                               = 0
mprotect(0x7fbe26866000, 4284416, PROT_READ|PROT_WRITE) = 0
read(53, "\266o\321\302m\t\25X\344\202\6\35\265\204XB\376$a\333\202J\n\230\20\376\301\253[\370C5"..., 4096) = 4096
sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3318521856, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3318521856, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 59
fstat(59, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(59, "MemTotal:        8013480 kB\nMemF"..., 1024) = 1024
close(59)                               = 0
mprotect(0x7fbcc2650000, 91545600, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd6ffec000
mprotect(0x7fbd6ffec000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd4e8da000, 2093056, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd6ffee000, 140009472, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd8577d000, 139882496, PROT_READ|PROT_WRITE) = 0
read(53, "\272\275k/\vd\356,\210Ww\364\233\24Sj\351\207\23\257#\354\232\332\230\2\273\225L\373\256\315"..., 4096) = 4096
mprotect(0x7fbd227b1000, 91676672, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd5ffea000
mprotect(0x7fbd5ffea000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd5ffec000, 140013568, PROT_READ|PROT_WRITE) = 0
read(53, "\214\5\0\0\0\0\0\0s\352\315\26\264\351\t@j\5\0\0j\5\0\0\1\0^\0\2=\30J"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd4ffe8000
mprotect(0x7fbd4ffe8000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd4ffea000, 140013568, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbdd0582000, 91676672, PROT_READ|PROT_WRITE) = 0
read(53, "ramData> </CustomParams> </Sched"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd3a22e000
mprotect(0x7fbd3a22e000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd3a230000, 140017664, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbda857a000, 91676672, PROT_READ|PROT_WRITE) = 0
read(53, " </CustomParams> </ScheduleEvent"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd2a22c000
mprotect(0x7fbd2a22c000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd2a22e000, 140017664, PROT_READ|PROT_WRITE) = 0
read(53, "x\361\22\211\275\206\265\255\30\343\22\21\303\35(\326\335\255s\272\253\252\241\260\322\226\24$\322h&\316"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcfa21e000
mprotect(0x7fbcfa21e000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcfa220000, 140017664, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd98578000, 91680768, PROT_READ|PROT_WRITE) = 0
read(53, "vent> <ScheduleEvent> <InstanceD"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcda21a000
mprotect(0x7fbcda21a000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcda21c000, 140021760, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd78574000, 91684864, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcca218000
mprotect(0x7fbcca218000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcca21a000, 140021760, PROT_READ|PROT_WRITE) = 0
read(53, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcaa214000
mprotect(0x7fbcaa214000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcaa216000, 140021760, PROT_READ|PROT_WRITE) = 0
read(53, "\350IoR\265)G\214\204\325\0364\245\347D\267\370r\220\353\251u\312\256t\235Vi\20\370\20\v"..., 4096) = 4096
mprotect(0x7fbd68573000, 91680768, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc9a212000
mprotect(0x7fbc9a212000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc9a214000, 140021760, PROT_READ|PROT_WRITE) = 0
read(53, "\340\352\305;\270\352K\202\337\365\373\244\323\215\367n\370\267\355N\240P0\242\305\233:S\321\240\365\254"..., 4096) = 4096
mprotect(0x7fbd58571000, 91684864, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc8a210000
mprotect(0x7fbc8a210000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc8a212000, 140025856, PROT_READ|PROT_WRITE) = 0
read(53, "\365\357\240\271\245dr\24#u\311U8\356\231EH\301\203\1\330\344_+\327QU`a\r\364?"..., 4096) = 4096
mprotect(0x7fbdb860a000, 65536, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc7a20e000
mprotect(0x7fbc7a20e000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc7a210000, 140025856, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd427b8000, 91684864, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc6a20c000
mprotect(0x7fbc6a20c000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc6a20e000, 140025856, PROT_READ|PROT_WRITE) = 0
read(53, "\27\7\0\4\0\0GC\351\30\0\0\1\340\30)\204\320\r=\256}\23?\35\256}\7\205\200\262\267"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc5a20a000
mprotect(0x7fbc5a20a000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc5a20c000, 140029952, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd327b6000, 91684864, PROT_READ|PROT_WRITE) = 0
read(53, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096) = 4096
mprotect(0x7fbdb861a000, 65536, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc4a208000
mprotect(0x7fbc4a208000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc4a20a000, 140029952, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd027a8000, 91688960, PROT_READ|PROT_WRITE) = 0
read(53, "\330\326\205]\220jY\327o\212X\251{\235h\272\250\245w\315\232\374\350\334\213;\f\200\16\352\226\3"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc3a206000
mprotect(0x7fbc3a206000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc3a208000, 140029952, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc2a204000
mprotect(0x7fbc2a204000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc2a206000, 140034048, PROT_READ|PROT_WRITE) = 0
read(53, "o\211\250\304\247\352\16u\224\340v\10#)1{\f\335=\243\230U$\242\213\207\201\2h\256T\216"..., 4096) = 4096
mprotect(0x7fbdb862a000, 65536, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbce27a5000, 91688960, PROT_READ|PROT_WRITE) = 0
+++ killed by SIGKILL +++
Morto

fariaalex avatar Jan 27 '22 12:01 fariaalex