CalebQ42
Error extracting
Hi
I am seeing an error while extracting.
"Unsupported file type. Inode type: 5"
This means that your archive has a character device in it. I never could find an archive with one when creating the library and I didn't know exactly what to do with it, so I just left it as "unsupported". That being said, I just did some research and it should be fairly simple to add support.
v0.7.11 should (in theory) adds support for character devices. As I mentioned, I don't have an archive to test it with, so please let me know if it works.
we are observing SIGSEGV.
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x2 addr=....]
goroutine 2531 [running]: io.ReadAll({0x0, 0x0}) github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo.func1() github.com/CalebQ42/squashfs/internal/data/fullreader.go:181 +0x58 created by github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo github.com/CalebQ42/squashfs/internal/data/fullreader.go:172 +0x150 exit status 2
Could you provide the archive that's giving this error? I can't reproduce with the archives I use for testing.
The problem seems to be something with a file's fragment either not being parse correctly, or something not working quite right with the de-compressor.
Unfortunately, I can not share the archive. But a few more data points:
io.ReadAll({0x0, 0x0}) /usr/lib/golang/src/io/io.go:701 +0xd9 github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo.func1() github.com/CalebQ42/squashfs/internal/data/fullreader.go:181 +0x68 created by github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo github.com/CalebQ42/squashfs/internal/data/fullreader.go:172 +0x165 panic: mkdir /tmp/extract-squash3679600936/usr/lib64/abc: permission denied [recovered] panic: mkdir /tmp/extract-squash3679600936/usr/lib64/abc: permission denied
Unfortunately, I can not share the archive. But a few more data points:
io.ReadAll({0x0, 0x0}) /usr/lib/golang/src/io/io.go:701 +0xd9 github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo.func1() github.com/CalebQ42/squashfs/internal/data/fullreader.go:181 +0x68 created by github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo github.com/CalebQ42/squashfs/internal/data/fullreader.go:172 +0x165 panic: mkdir /tmp/extract-squash3679600936/usr/lib64/abc: permission denied [recovered] panic: mkdir /tmp/extract-squash3679600936/usr/lib64/abc: permission denied
This seems to be a separate issue.
This seems to be due to a permissions issue. The squash archive has permissions that is set when extracting and can cause issues if a folder doesn't allow you to write files inside of it. v0.7.13 adds an option to ignore the permissions or you can extract it with root permissions.
In the future it should probably be set-up so permissions are not set on a folder until after it's contents are extracted to prevent this issue.
v0.8.0 Fixed some things that may be related to the issues you are having.
Thanks
unfortunately, I still see the issue.
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x48b159]
goroutine 802 [running]: io.ReadAll({0x0, 0x0}) /usr/lib/golang/src/io/io.go:701 +0xd9 github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo.func1() go/src/github.com/CalebQ42/squashfs/internal/data/fullreader.go:181 +0x68 created by github.com/CalebQ42/squashfs/internal/data.FullReader.WriteTo go/src/github.com/CalebQ42/squashfs/internal/data/fullreader.go:172 +0x165 panic: unexpected EOF [recovered] panic: unexpected EOF
I found an error that wasn't being reported correctly. In v0.8.1 You'll still get an error, but it'll be a usable error instead of a panic.
I see "unexpected EOF" whereas when I run "unsquashfs" command line utility I am able to extract successfully.
Could you give the output of unsquashfs -s. It should give me some info so I could create an analogous archive that I could test on.
Found a valid SQUASHFS 4:0 superblock on ABC.sqsh Creation or last append time XXXXXX Filesystem size YYY Kbytes (ZZZ Mbytes) Compression xz Dictionary size PPP Filters selected: x86 Block size PPP Filesystem is exportable via NFS Inodes are uncompressed Data is compressed Fragments are compressed Always_use_fragments option is not specified Xattrs are compressed Duplicates are removed Number of fragments 9 Number of inodes 435 Number of ids 5
The problem appears to be that the fragment for a file isn't as large as it should be. Since I'm not getting errors for fragments for any archives I've tested, I'm currently inclined to think that the problem is more that unsquashfs isn't as stringent with fragment length. On exp branch I've made it ignore specifically Unexpected EOF errors for fragments, if you could give that a try, then check the file (such as with using Meld) to see if there's any difference between this library or unsquashfs. This is at least the best theory I have at the moment.
Wanted to give you an update.
My current guess is that it's related to decompression. XZ isn't well supported in Go, with most of the libraries simply using cgo or not supporting all features (namely LZMA filters). The current XZ library I use is the only one that is pure Go and supports LZMA filters, but it seems to be abandoned, with the last update being in 2020. Then when I was looking around, I learned that a new version of the XZ format was release last year (2022), which could also be contributing to the issues.
To be clear, it's still very possible that my library is at fault; there's a lot of moving parts and without the archive causing the problem it's impossible to know what exactly is going on.
It's been a while, but if you're still interested in using this library could you try with the newest version v0.8.4 as it might fix your issues. In particular I've added the ability for file fragments to be sparse (filled with zeroes) even though this isn't documented, but it does align with the errors you were getting.