Opening syscalls view crashes Sysdig Inspect

[danielmitterdorfer]

Observed behavior

When opening the "Syscalls" view, Sysdig Inspect starts to read the trace file but the UI freezes and all UI elements disappear after processing 16 - 20% of the trace file.

This problem reproduces always with the attached trace file (I did not try others).

Steps to reproduce

1. Open "Sysdig Inspect"
2. Open trace.scap.zip (does not contain any sensitive info; it was captured in a newly created Virtualbox VM)
3. Click on "Syscalls" in the bottom left corner

System info

The trace was captured on:

# uname -a
Linux packer-virtualbox-iso-1512307732 4.4.0-101-generic #124-Ubuntu SMP Fri Nov 10 18:29:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
# sysdig --version
sysdig version 0.8.0

The system running Sysdig Inspect:

# uname -a
Darwin io 15.6.0 Darwin Kernel Version 15.6.0: Mon Oct  2 22:20:08 PDT 2017; root:xnu-3248.71.4~1/RELEASE_X86_64 x86_64
# Sysdig Inspect version
Version 0.2.0 (0.2.0)

Additional info

In the MacOS system log we see the following:

10/12/17 13:16:26,989 diagnosticd[129]: error evaluating process info - pid: 1698, puniqueid: 1698
10/12/17 13:16:27,151 ReportCrash[1643]: Saved crash report for Sysdig Inspect Helper[1698] version 0 to /Users/daniel/Library/Logs/DiagnosticReports/Sysdig Inspect Helper_2017-12-10-131627_io.crash

See more details in the crash report Sysdig Inspect Helper_2017-12-10-131627_io.crash.zip.

[davideschiera]

Thanks @danielmitterdorfer for reporting the issue. I can replicate the issue on my side, I'll dig more soon.

[davideschiera]

The problem seems to be related to sysdig itself. You should be able to replicate this as well:

$ csysdig -r trace.scap
Could not initialize reader: invalid block type

[danielmitterdorfer]

That's odd. I ran csysdig -r trace.scap on the machine where I have created the trace file and it presented at least an initial screen (so I did not get the same error message as you). As soon as I hit a key (e.g. F6) in that UI, csysdig crashes immediately.

I created a trace again today on a "proper" machine (i.e. not a Vagrant box) and Sysdig Inspect could open that trace just fine and has also shown the Syscalls view.

So I think it boils down to the fact that the trace that I have attached originally is really fishy. Not sure if you want to handle the case of corrupt trace files explicitly in Sysdig Inspect and present an explicit error message though.

[davideschiera]

What version of sysdig do you have on your VM (and where you use csysdig if different)? And can you try creating a capture with the initial VM and see if you have the same problem?

At this point I see 3 issues:

1. it seems like you managed to generate a corrupted file with sysdig, I'd try to investigate this a little more but might need more information from you
2. when you use a corrupted file with Sysdig Inspect, the server crashes and it doesn't recover
3. the error is reported to the frontend, but no useful information is displayed

It makes sense to address 2 and 3 immediately, as you suggest.

[danielmitterdorfer]

What version of sysdig do you have on your VM (and where you use csysdig if different)?

This was probably lost in all the information provided in the original message: It's in all cases Sysdig version 0.8.0.

can you try creating a capture with the initial VM and see if you have the same problem?

I created a trace in the very same VM once more but did not start any processes while capturing (i.e. the VM was pretty silent). I could load that trace file successfully in sysdig.

Then I ran the exact scenario that I did initially: I ran a benchmarking tool in another ssh session on this VM and captured a trace with sysdig. This time, I could not load the trace. So it seems to be related somehow to what's captured by sysdig when the tool is running.

[davideschiera]

Would it be possible to update sysdig? Latest version is 0.19.1 (released on 10/5/2017), there's a good chance that the bug was fixed between the 2 releases.

Thanks again for the help!

[danielmitterdorfer]

Oh sorry! I was under the impression that I have already the latest version as I installed it just recently. I'll update soon and report my findings.

[danielmitterdorfer]

I've downloaded the Debian x86_64 package for sysdig 0.19.1. The download URL for the Debian package at https://www.sysdig.org/wiki/latest-packages/ reports 0.17.0 as latest version but given your hint, I just adjusted the URL and could then successfully download 0.19.1.

Then I ran the test scenario again I've mentioned yesterday in the Vagrant box and the result is the same. After ~ 20%, the UI freezes.

However, this time I could successfully navigate in csysdig hence I'm uploading the new trace file in case it is of interest to you: trace_019_1.scap.zip

[davideschiera]

The issue can be still reproduced with latest release 0.3.3 (https://github.com/draios/sysdig-inspect/releases/tag/0.3.3).

For the moment, the best workaround is to start with higher level views (e.g. list of processes) and then use the "Syscalls" view to narrow down the scope of system events to expose.

Sorry about the persisting issue.