turbinia
turbinia copied to clipboard
Add lsof debug statement in PostProcessLosetup
In #1122 we've discovered issues with file handles stuck on loop devices after a Task completes as well as some race conditions causing the postprocessor to fail detaching the disk.
For future troubleshooting, add a lsof <device_path>
debug statement in PostProcessLosetup
underneath the debug statement losetup retry check
.
We'll also need to add lsof
to the Turbinia worker docker image and come up with a way to track system dependencies, something that we currently lack (only track Job dependencies).
One option we talked about here is since we run into some complex loop/disk/partition handling related issues from time to time is to have a debug method that dumps lots of other state around the loopback devices and other block devices (e.g. lsblk, gdisk -l, losetup -a) and is only enabled when the DEBUG_TASKS variable is set (which can be set on a per request basis).