Import of splice package breaks some software execution

[sb10]

github.com/hanwen/go-fuse/v2/splice is a dependency of a dependency of a dependency of a dependency of a package that I import in my software. In one code path of my software, I don't use go-fuse, but the import remains.

My software does an exec.Command("a_certain_executable"), and when the import is in place, it dies with errors like:

error while loading shared libraries: libc.so.6: cannot open shared object file: Error 24

If I comment out the init() method of splice.go (https://github.com/hanwen/go-fuse/blob/f57e95bda82d40dd4b2ceb1065fbd98c834ba23b/splice/splice.go#L36), I don't get the error.

While this is a very rare case (only this certain executable suffers like this), I believe it's quite bad for a package to have system-wide side-effects that cause issues with other software, merely on import.

I don't really understand what this init() method is doing, or how it is causing the side-effect that it causes, but is there any way to refactor splice.go so that it doesn't do this on an init(), but only when really required to do so by some method call?

[hanwen]

you can disable splice completely by doing cd fuse; mv splice_darwin.go splice_linux.go.

When I do that over here, the resulting binaries are still linking to libc.so.6. Are you executing in a restricted container that doesn't have libc installed?

[sb10]

No, just a normal operating system (ubuntu).

For my code path that does use go-fuse, I obviously need it to work correctly, so I can't just disable splice completely, unless it's really not needed for anything. And if its not needed, you can do a release of go-fuse with the init() removed?

[hanwen]

splice is a performance optimization that reduces amount of data copying between fuse and the kernel. You can disable it at a slight performance cost.

I can't make a release that disables it; without understanding the problem, it's hard to predict if the release would actually fix it.

[hanwen]

https://stackoverflow.com/questions/7876706/sh-error-while-loading-shared-libraries-libc-so-6-cannot-open-shared-object-f - you need to increase your max open file limit.

[sb10]

It's not an open file limit issue, or at least not in a way that I can fix. I'm an order of magnitude under my limit, and the only thing changing between working and not working is that init() function.

Can you make a release that calls the code that's currently in the init only on demand, and not on import?

[sb10]

Even without understanding the problem, can you agree that it's better to not have code that has system-wide side-effects on import?

Can you describe in words what this code is doing, and what effects it might be having on the system? That might make it easier to debug.

[hanwen]

Look,

$ grep 24 /usr/include/asm-generic/errno-base.h 
#define	EMFILE		24	/* Too many open files */

the kernel says you have too many open files. I suggest to use strace to find where your file descriptors are going. Or look under /proc/self/fd/.

I'm happy to fix problems in go-fuse, but I haven't seen any indication there is a problem that is go-fuse's fault. If you want to convince me, please send me a minimal reproducer.