AppImageKit icon indicating copy to clipboard operation
AppImageKit copied to clipboard

Build cmake dependancy (mksquashfs) error (gcc 10.2.0)

Open kyx0r opened this issue 4 years ago • 26 comments

2020-11-10-135355-1818x1233-scrot
picture uploader

I hope one day you get rid of this abomination called Cmake. Wouldn't it be nice if I could make a patch for this, but I literally can't because of all this junk in the way.

gcc AppImage.c -o AppImage Keep it simple, stupid.

kyx0r avatar Nov 10 '20 19:11 kyx0r

L146 of build/mksquashfs-prefix/src/mksquashfs/squashfs-tools/mksquashfs.h misses a extern. Adding it fixes the problem.

Can reproduce. this has been fixed upstream, but not on appimagekit.

srevinsaju avatar Nov 10 '20 20:11 srevinsaju

AppImageKit hasn't changed, so I don't see why this would occur anywhere.

Feel free to send a patch.

I hope one day you get rid of this abomination called Cmake. Wouldn't it be nice if I could make a patch for this, but I literally can't because of all this junk in the way.

gcc AppImage.c -o AppImage Keep it simple, stupid.

That's just utter nonsense. By the way, you don't have to build anything yourself if you don't want to. Just get the pre-built binaries.

TheAssassin avatar Nov 10 '20 20:11 TheAssassin

That's just utter nonsense. By the way, you don't have to build anything yourself if you don't want to. Just get the pre-built binaries.

I went on a rant a bit, I know this likely wouldn't be acceptable for a project this size, but that does not mean supporting unity build is impossible. C is c, all you need is the compiler to work with it.

It's true that AppImageKit did not change. It's probably due to the gcc changes. See here for info https://aur.archlinux.org/packages/appimage-git/ leads to https://wiki.gentoo.org/wiki/Gcc_10_porting_notes/fno_common

kyx0r avatar Nov 10 '20 20:11 kyx0r

We can't actively support third-party packaging. Bundling on Gentoo/Arch is anyway pretty useless usually.

As said, PRs welcome. @srevinsaju seems to know where the issue is, so I hope to see one soon. Another extern won't break anything elsewhere, I guess.

TheAssassin avatar Nov 10 '20 21:11 TheAssassin

This is the fix which is missing in AppImageKit's dependencies:

https://github.com/plougher/squashfs-tools/commit/fe2f5da4b0f8994169c53e84b7cb8a0feefc97b5

which happened 11 months ago. But, I am sure that it built clean 3 months ago

Looking at the commit message, it seems to be something which happened due to GCC v10. I remember seeing a lot of patches on Arch Linux's AURs.

srevinsaju avatar Nov 10 '20 21:11 srevinsaju

Perhaps we need to update a commit ID in our external project builds?

TheAssassin avatar Nov 10 '20 21:11 TheAssassin

2020-11-10-161920-1706x539-scrot Adding extern on L146 of build/mksquashfs-prefix/src/mksquashfs/squashfs-tools/mksquashfs.h still did not do it for me, now there is less of 'multiple definition' than before but looking around at the sources I can't quite make sense of why I still get the error. I had to modify the build.sh file to not remove the ./build directory each run, idk what is the correct way to get the build system to not overwrite the source code each time I build. But it looked like it did not overwrite mksquashfs.h at least. But idk what goes there in cache.

kyx0r avatar Nov 10 '20 21:11 kyx0r

Okay it was a cache thing problem in the end. removing process_fragments.o and recompiling did the trick. But now there is bunch of this stuff.

2020-11-10-163037-2558x1410-scrot

Guess I'll have to wait for someone who knows how these dependencies are built cause, I am no build system expert here.

kyx0r avatar Nov 10 '20 21:11 kyx0r

If you try to add extern to those symbols you see redefined in the image above you will get runtime.4.o: warning: relocation againstlibhandle' in read-only section .text.startup.main' which means that you have to add -fPIC somewhere in the build process. Where can I do that?

kyx0r avatar Nov 10 '20 22:11 kyx0r

We already build the runtime as position-independent executable, actually... Strange...

TheAssassin avatar Nov 10 '20 23:11 TheAssassin

@TheAssassin Well I've decided to give this a second look. And I guess my years of working with C reminded me that linkage errors from .h files can be also resolved by using keyword static as that will tell the linker to only make 1 copy of the said symbol. So basically this is what you want to do to fix the build:

diff --git a/squashfs-tools/mksquashfs.h b/squashfs-tools/mksquashfs.h
index 1beefef..b650306 100644
--- a/squashfs-tools/mksquashfs.h
+++ b/squashfs-tools/mksquashfs.h
@@ -143,7 +143,7 @@ struct append_file {
 #endif

 extern struct cache *reader_buffer, *fragment_buffer, *reserve_cache;
-struct cache *bwriter_buffer, *fwriter_buffer;
+extern struct cache *bwriter_buffer, *fwriter_buffer;
 extern struct queue *to_reader, *to_deflate, *to_writer, *from_writer,
        *to_frag, *locked_fragment, *to_process_frag;
 extern struct append_file **file_mapping;
diff --git a/src/patches/squashfuse_dlopen.h b/src/patches/squashfuse_dlopen.h
index da3c34f..78dd1d9 100644
--- a/src/patches/squashfuse_dlopen.h
+++ b/src/patches/squashfuse_dlopen.h
@@ -23,9 +23,9 @@

 #define LIBNAME "libfuse.so.2"

-void *libhandle;
-int have_libloaded;
-const char *load_library_errmsg;
+static void *libhandle;
+extern int have_libloaded;
+extern const char *load_library_errmsg;

 #define LOAD_LIBRARY \
 if (have_libloaded != 1) { \
diff --git a/build.sh b/build.sh
index b69e99a..aba9e24 100755
--- a/build.sh
+++ b/build.sh
@@ -63,10 +63,8 @@ cd "$HERE"
 git submodule update --init --recursive

 # Clean up from previous run
-[ -d build/ ] && rm -rf build/

 # Build AppImage
-mkdir build
 cd build

 # make sure that deps in separate install tree are found

You will need the last patch also, because otherwise the 1st 2 pathes will get overwritten each time. Also when working with this and patching make sure to run

make clean

in ./build directory each time.

Looking forward for cooperation from the maintainer and fix these cmake dependencies asap.

kyx0r avatar Nov 12 '20 21:11 kyx0r

I was about to fix an issue in AppImageKit and submit a PR, but I can't even build it because of:

/usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: read_fs.o:(.bss+0x0): multiple definition of `fwriter_buffer'; mksquashfs.o:(.bss+0x400c90): first defined here

:-/

realnc avatar Sep 23 '21 16:09 realnc

Which system are you trying to build on? Best use CentOS 6 as the build system.

probonopd avatar Sep 23 '21 19:09 probonopd

Which system are you trying to build on? Best use CentOS 6 as the build system.

I'm not going to move from my Gentoo install to an ancient and hopelessly outdated system like CentOS 6 that is completely useless in 2021.

realnc avatar Sep 23 '21 19:09 realnc

Then you will have no fun in working on AppImageKit, sorry. Maybe you could use a virtual machine or container?

probonopd avatar Sep 23 '21 19:09 probonopd

https://www.linuxvmimages.com/images/centos-6/

probonopd avatar Sep 25 '21 21:09 probonopd

https://en.wikipedia.org/wiki/Software_portability

realnc avatar Sep 25 '21 21:09 realnc

There is no reason to not merge fixes that allow building on newer systems as well. These errors are usually very much justified. Compilers become smarter over time...

@probonopd's responses are just infantile. I'm sure @realnc has heard about Docker. That's not the point, though. Instead, I'd have invited @realnc to contribute a fix. But I'm sure they're pretty much not motivated by now.

Also, CentOS 6 has long been EOL and should not be used for obvious reasons.

@realnc, I guess @probonopd just wants to ensure the thing can still be built on the target environment, CentOS. Should you find a fix (or be willing to collaborate on one) to allow building on Gentoo as well, please don't hesitate.

P.S.: Yes, old distros are often outdated. But the way the runtime works mandates building on old distros. Nobody's asked you to work on that. The CI is going to make sure the binary can still be built, so, fixes welcome.

TheAssassin avatar Sep 25 '21 21:09 TheAssassin

@probonopd just wants to ensure the thing can still be built on the target environment, CentOS.

Exactly. Sorry if I was not clear. (Actually when I was working on the runtime it built on CentOS 6 and I hope it still does - I had forgotten that apparently we are now building on CentOS 7. For me the runtime.c is pretty much feature complete by now (except for small improvements like fixing the create/modification/... times that won't require new dependencies), so I hope the build requirements won't need to be incremented anymore, indefinitely.)

(The older I get the more annoyed I am for everything to be updated all the time, and the known good things to be no longer supported. I know someone who still used Ubuntu gutsy 2 years ago - shows that I am not alone.)

probonopd avatar Sep 25 '21 23:09 probonopd

Bump, 3 years after reported and still a problem.

theofficialgman avatar Dec 11 '23 15:12 theofficialgman

@theofficialgman is there a special reason you need to build this yourself rather than using the provided binaries?

If you'd ask me, we should freeze our build environment indefinitely. I don't like moving targets.

probonopd avatar Dec 12 '23 18:12 probonopd

@theofficialgman is there a special reason you need to build this yourself rather than using the provided binaries?

I don't see how this question is relevant to the bug report.

I am able to use the official appimages but have to call them directly with QEMU in my CI due to this other bug https://github.com/AppImage/AppImageKit/issues/965

At the end of the day, appimagekit should be able to be compiled on modern systems without having to resort to 5+ year old tooling. As it is right now, appimagekit can't be built on even the oldest supported ubuntu LTS (Focal, notice that I didn't say ELTS).

theofficialgman avatar Dec 12 '23 18:12 theofficialgman

I don't see how this question is relevant to the bug report.

The classic use case is prebuilt binaries for us. Supporting anything but the official build environment is additional effort for us to test and maintain whereas it has little relevance for everyday development.

At the end of the day, appimagekit should be able to be compiled on modern systems

Agreed. Contributions welcome!

appimagetool was even extracted out of this repository a couple of months ago, so this issue may have to be transferred there anyway. Please don't hesitate to test that repository, though. It should work better, as after the move, I've spent some time debugging and fixing build problems there.

TheAssassin avatar Dec 12 '23 18:12 TheAssassin

AppImageKit hasn't changed, so I don't see why this would occur anywhere.

This is possibly because since gcc-10, the -fno-common flag is the default. More here: https://www.linuxquestions.org/questions/programming-9/multiple-definition-errors-when-linking-on-arch-using-gcc-10-1-0-a-4175675444/#post6124558

is there a special reason you need to build this yourself rather than using the provided binaries?

In a docker container using qemu in a GitHub actions runner, I can't extract appimagetool-aarch64.AppImage after using dd if=/dev/zero of=linuxdeploy-$ARCH.AppImage conv=notrunc bs=1 count=3 seek=8. I get Failed to get fs offset for /proc/self/exe. As a solution, I want to build appimagetool as part of a Dockerfile build.

I tried here with LInux Deploy https://github.com/netpanzer/netpanzer/pull/197

And here with appimagetool. I know those are separate from this issue; I'm just giving the reason why I want to build appimagetool. Hmmm... actually it should build in there because it's a focal image and using gcc-9...

andy5995 avatar Feb 01 '24 03:02 andy5995