Feature request: provide an AppImage for CodeLite

[fusion809]

Hi,

I'd like to suggest that you developers of CodeLite provide an AppImage for CodeLite. This way users of all modern Linux distributions should be able to use CodeLite without any problem. See AppImages are a cross-distribution packaging format that are designed to be run without a hitch on any modern Linux distribution. The beauty of them is that they do not need to be installed. They are merely made executable (with chmod +x) and run (e.g., with ./CodeLite-9.2.0-x86_64.AppImage). No extra tools or root privileges are required to run AppImages either.

Thanks for your time, Brenton

[fusion809]

A related issue I started on the probonopd/AppImages repository is probonopd/AppImages#108.

[fusion809]

Oh and please add a metainfo file to this program. As presently my attempts to build an AppImage for CodeLite are failing because it is missing. Specifically I'm getting the error:

WARNING: AppStream upstream metadata is missing, please consider creating it
         in usr/share/metainfo/codelite.appdata.xml
         Please see https://www.freedesktop.org/software/appstream/docs/chap-Quickstart.html#sect-Quickstart-DesktopApps
         for more information.
Error opening file '/home/fusion809/GitHub/mine/packaging/AppImages/recipes/meta/codelite/CodeLite/CodeLite.AppDir/usr/share/metainfo/codelite.appdata.xml': No such file or directory
Failed to generate a template

[fusion809]

If you's end up refusing to provide an AppImage please at least add a configure option to enable binreloc (https://github.com/limbahq/binreloc).

[eranif]

If you's end up refusing to provide an AppImage

I am not refusing, its just has lower priority.

[fusion809]

I am willing to take care of this, but my attempts so far have failed. I think if you add a binreloc cmake option (i.e., adding the cmake option --enable-binreloc or something similar) I can produce a working AppImage.

[probonopd]

Seems to compile forever, resulting in

The job exceeded the maximum time limit for jobs, and has been terminated.

on Travis CI.

https://travis-ci.com/probonopd/codelite/builds/115667414

[probonopd]

Why was this closed @fusion809?

[fusion809]

Well, because the stale bot was going to close it and I was sick of getting notifications about it being imminent, lol.

[probonopd]

I had pinged that stupid bot so that it went away. Not doing anything is imho never a good reason to mark something as "wontfix". It can also mean "no one had the time to look at it yet."

[eranif]

This is the bot behavior. writing something in the comment of the bug will remove the label automatically and will silence the bot for a month

[dghart]

I just spent a couple of days playing with this. I had almost no previous appimage experience, and no great understanding of cmake either. Reading various threads from various places, I now have less understanding of cmake (there's a comment at the end of this thread: "See you in the ward for those suffering from cmake related nervous disorders in the near future..."

Still, I gather that cmake doesn't need much tweaking to make relocatable binaries; just use DESTDIR. And it seems that's true for simple projects: I managed to appimage an app in debian jessie (the oldoldstable version) which runs in buster and in openSUSE 15.1. I did this using linuxdeploy following these instructions.

Trying with CodeLite was less successful. I can create an appimage but it fails to run successfully, even in the same distro. There usually isn't an error message, it just aborts. Running in gdb doesn't give any extra information, and strace was no help either.

I've run out of time and inclination but, in case you want to take over, I've paste-binned a description of what did.

Anyway, I suspect there is no need to alter the CodeLite build setup to build an appimage. Of course, if you find you do need to make alterations to make the appimage run...

[probonopd]

What happens if you do not mount it but insteaed run it like this?

./path/to/CodeLite-13.0-x86_64.AppImage

Is the AppImage available for download so that I can test/debug?

[dghart]

It 'runs' with the same problems, but only if LD_LIBRARY_PATH is set correctly, which it won't be as the exact mount-point is unknown until the mount has succeeded. When I tested, I had first to mount the appimage by hand; then I could use its libs in LD_LIBRARY_PATH to run a duplicate with ./path/to/CodeLite-13.0-x86_64.AppImage.

The image needs its rpath to be set to "$ORIGIN/../lib:$ORIGIN/../lib/codelite", but my attempts to do this with cmake have failed. BUILD_RPATH_USE_ORIGIN should work, but it's far too new to be available in the jessie cmake version (3.0.2).

Is the AppImage available for download so that I can test/debug?

I've now uploaded it temporarily to www.4Pane.co.uk/temp/CodeLite-13.0-x86_64.AppImage (NB it's 231MB). Good luck!

[probonopd]

You could

• Move or symlink libraries so that they can be loaded from $ORIGIN/../lib; or
• Use patchelf to change the rpath after the compiler has run; or
• Export LD_LIBRARY_PATH in a custom AppRun script, setting it to $HERE/usr/lib/... (where $HERE) would be a path relative to the AppRun script itself)

[probonopd]

Getting

me@host:~$ /home/me/Downloads/CodeLite-13.0-x86_64.AppImage 
/home/me/Downloads/CodeLite-13.0-x86_64.AppImage: symbol lookup error: /home/me/Downloads/CodeLite-13.0-x86_64.AppImage: undefined symbol: _ZTI16clThemedListCtrl

Where should _ZTI16clThemedListCtrl be coming from?

[dghart]

You could Move or symlink libraries so that they can be loaded from $ORIGIN/../lib; or Use patchelf to change the rpath after the compiler has run; or Export LD_LIBRARY_PATH in a custom AppRun script, setting it to $HERE/usr/lib/... (where $HERE) would be a path relative to the AppRun script itself)

Thanks, I'll have a go.

/home/me/Downloads/CodeLite-13.0-x86_64.AppImage: undefined symbol: _ZTI16clThemedListCtrl

That comes from one of the CodeLite libs, lib/codelite/libplugin.so The error disappears if you run the appimage with LD_LIBRARY_PATH=/tmp/$(mt-point)/usr/lib/codelite/AppRun

However, while looking for the symbol inside the image, I noticed that there were duplicated wxWidgets libs in usr/lib/ (they should be only in usr/lib/codelite) and the extras were from a different wxWidgets build. That might account for the segfaults. How can I get rid of them, either:

1. by adding them to an exclude list, perhaps? Is there a command-line option or something equivalent that would do this?
2. or manually: I can do appimage-extract and delete them from usr/lib/, but how do I rebuild the appimage without them being added back? Is there an available re-squash script? I tried calling mksquashfs squashfs-root/ which 'worked', but the resulting appimage doesn't have an offset so is probably bad in other ways too. (However, running it does give a better segfault ;) one that gdb can show a backtrace from.)

[probonopd]

However, while looking for the symbol inside the image, I noticed that there were duplicated wxWidgets libs in usr/lib/ (they should be only in usr/lib/codelite)

Do you have an idea where the second set came from?

How can I get rid of them

Unpack the AppImage, make your modifications, run ./appimagetool-*.AppImage ./squashfs-root

[dghart]

Do you have an idea where the second set came from?

They are installed on the system. I don't know why some were grabbed from there.

Unpack the AppImage, make your modifications, run ./appimagetool-*.AppImage ./squashfs-root

Ah, thanks. I'd missed appimagetool.

I've now made various changes, with partial success: I've made some temporary CodeLite cmake changes so that the plugin and wx libs are installed to usr/lib (DebuggerGDB.so is still in debuggers/ and so when creating the appimage I needed to set LD_LIBRARY_PATH in buster, but not in jessie :/ ). For the record, after prepending the path to the correct wxwidgets install to my $PATH, the compile line was (mkdir -p build-dbg3 && cd build-dbg3/ && cmake -G "Unix Makefiles" VERBOSE=1 -DCMAKE_BUILD_TYPE=DebugFull -DWITH_PCH=0 -DCL_PREFIX=$(pwd)/../CLgtk3-install/usr -DPLUGINSDIR=$(pwd)/../CLgtk3-install/usr/lib -DCOPY_WX_LIBS=1 .. && make -j16 && make install) though that requires those cmake changes. The resulting appimages build OK, and they run cleanly; but only on the same installation: the jessie one runs on jessie, the buster one on the building buster but not on a buster virtualbox guest. Elsewhere they start but then abort without a useful error message, even when run in gdb, even when unsquashed and run direct.

I'd guess there is a system lib conflict somewhere, but I've failed to find it; e.g. ldd doesn't show anything wrong, and LD_DEBUG gives only false alarms about missing symbols (it gives the same ones even when CodeLite runs successfully).

I've run out of ideas. In case anyone wants to test them, I've again uploaded the two faulty appimages: www.4Pane.co.uk/temp/CodeLite-13plus-buster.AppImage www.4Pane.co.uk/temp/CodeLite-13.0-Jessie.AppImage

[probonopd]

Did you try to compare a working to a non-working invocation using strace?

[dghart]

Yes. It doesn't show anything that is helpful, at least to me. Part of the problem is that, when starting, CodeLite creates and runs lots of threads. I suspect that one or more of those is the cause but I can't prove it; nor can I prevent the threads from running.

I also tried copying the jessie gcc libs/dependencies/bundling gtk2, using appimage-helper-scripts. It didn't help.

[probonopd]

Is it possible to build an AppImage on Ubuntu xenial?

[dghart]

No, not a successful one.

I first built the standard way, and it runs fine on xenial. It failed on debian buster without any useful output (even in gdb); on debian jessie it failed with the output '...libstdc++.so.6: version `GLIBCXX_3.4.21' not found'; and on ubuntu bionic it 'Failed to load module "canberra-gtk-module"'.

I therefore rebuilt, adding the gcc etc dependencies as above. This time, on jessie it now segfaults without any useful bt, even when just running --appimage-extract :/. And bionic dies with a SIGILL from /usr/lib64/ld-linux-x86-64.so.2.

While I'd like to see a working CodeLite appimage and would welcome any help to produce one, I'm afraid my available time is exhausted.

[probonopd]

Thanks, I highly appreciate the time you have been putting into this.

Let me ask differently:

Is it possible to build CodeLite on Ubuntu xenial on Travis CI? If yes, then I can have a look at converting it into an AppImage.

[dghart]

I've no experience of Travis CI, but I can't think why it shouldn't be possible. However you will not be able to succeed with a standard CodeLite 13.0 tarball; it fails on old versions like xenial. I've therefore uploaded a fixed (and more recent) CodeLite tarball for you to test.

CodeLite uses the wxWidgets toolkit.The other problem with xenial is that the version of wxWidgets that it provides, wx3.0.2, is very old. I've not tested CodeLite with it so it might be OK, but I strongly suggest you use wx3.1 instead. There isn't a xenial build for that in the CodeLite wx repo either, so I've uploaded a tarball of the required debs (which are built against GTK+3). Presumably that will need to be extracted/installed into Travis, and their dependencies (and CodeLite's) installed...

Once its dependencies are installed and CodeLite tarball extracted, the way I've been building it is: (mkdir -p build-dbg3 && cd build-dbg3/ && cmake -G "Unix Makefiles" VERBOSE=1 -DCMAKE_BUILD_TYPE=DebugFull -DWITH_PCH=0 -DCL_PREFIX=$(pwd)/../CLgtk3-install-4appimage/usr -DCOPY_WX_LIBS=1 .. && make -j<no_of_cores> && make install)

Then:

cd CLgtk3-install-4appimage
cp usr/share/applications/codelite.desktop .
mv usr/lib/codelite/debuggers/debuggers/*.so  usr/lib/codelite
for f in *.so; do patchelf --set-rpath '$ORIGIN' $f; done
mv usr/lib/codelite/*.so -> usr/lib/
for f in usr/bin/*; do patchelf --set-rpath '$ORIGIN/../lib' $f; done
ARCH=x86_64 VERSION=13.9 /path/to/linuxdeploy-x86_64.AppImage --appdir . --output appimage

Please let me know if you need more information about any of this.

[probonopd]

Thanks @dghart for putting effort into this.

you will not be able to succeed with a standard CodeLite 13.0 tarball; it fails on old versions like xenial. I've therefore uploaded a fixed (and more recent) CodeLite tarball for you to test.

Does this mean you will have additional work to maintain this forever? Could the upstream code be changed so that it contains your changes?

CodeLite uses the wxWidgets toolkit.The other problem with xenial is that the version of wxWidgets that it provides, wx3.0.2, is very old.

It should be no problem to compile a later version of wxWidgets, since wxWidgets is going to be bundled privately alongside the application anyway it should not make any difference in terms of size.

[probonopd]

I have started working on this over at https://travis-ci.com/probonopd/codelite-appimage/ but am running into Travis CI build time limits. Can I initially disable some parts to shorten build times?

[dghart]

Does this mean you will have additional work to maintain this forever? Could the upstream code be changed so that it contains your changes?

CodeLite 14.0 has now been released and, IIRC, it has the xenial fix. But even if not, it's not long until the ubuntu 20.04 release...

Can I initially disable some parts to shorten build times?

I think the only easy way to do that is to pass -DNO_CORE_PLUGINS=1 to cmake. But that will result in a CodeLite build that's noticeably less useful, so it should be for testing only.

I don't suppose Travis CI will let you use ccache, and retain its data between runs?

[probonopd]

I don't suppose Travis CI will let you use ccache, and retain its data between runs?

It does. But it only caches stuff when the build succeeded. Hence I will start with -DNO_CORE_PLUGINS=1 and go from there.

[probonopd]

Finally, we are getting somewhere :100:

Using https://github.com/probonopd/codelite-appimage/blob/simplify/.travis.yml I was able to build https://github.com/probonopd/codelite-appimage/releases/tag/continuous.

But it seems to be looking for files in absolute paths like

/usr/share/codelite/astyle.sample
/usr/share/codelite/gdb_printers
/usr/share/codelite/gdb_printers/
/usr/share/codelite/images
/usr/share/codelite/index.html
/usr/share/codelite/php.sample
/usr/share/codelite/rc
/usr/share/codelite/rc/menu.xrc
/usr/share/codelite/svnreport.html
/usr/share/codelite/templates
/usr/share/codelite/templates/

This prevents the application from being launched.

Can the source code be changed so that those paths are opened from a location relative to the main executable? More information: https://docs.appimage.org/reference/best-practices.html#binaries-must-not-use-compiled-in-absolute-paths

For test purposes, when I manually put the files there, then the AppImage works:

./CodeLite-9-x86_64.AppImage --appimage-extract usr/share/codelite/*
sudo mv squashfs-root/usr/share/codelite /usr/share/
./CodeLite-9-x86_64.AppImage

[dghart]

But it seems to be looking for files in absolute paths like...

Yes, that's what your cmake line says to do. If you want to 'install' to a more local dir, you need to set CL_PREFIX to something else. For me, that was: -DCL_PREFIX=$(pwd)/../CLgtk3-install-4appimage/usr

but ofc the name doesn't matter. I don't think you need to mkdir first.

For test purposes, when I manually put the files there, then the AppImage works

Good :) But so does mine in xenial. It fails elsewhere though. Can you test your appimage in e.g. debian, or ubuntu >16.04.