djfun
Package for Easier Installation
Below I'll outline some packaging possibilities. Bundling dependencies with container formats on Linux, and for easier distribution on Windows and Mac may require a licensing change. The feature-newgui branch is where all development for the next release is happening.
Linux
- Snap Package
- AppImage
- Flatpack
- Deb & RPM Repos
macOS
- py2app
- Homebrew
Windows
- cx_Freeze + Installer
Current Dependencies on feature-newgui Branch
- numpy
- pillow-simd
- qt5
- pyqt5
- ffmpeg 3.3
Mainly pyqt, which is licensed under the GPL. The other dependencies are under BSD, MIT or LGPL. Also: don't forget ffmpeg as a dependency.
If pyqt is the only hold up we could consider moving to PySide which has LGPL. Or dual license under GPL and MIT? Making the GPL version the one we ship containerized with dependencies.
so the code in the repository would be MIT and the packaged version with the dependencies GPL. I would be fine with that.
Would it be possible to give the user the option to use their own ffmpeg build if they want? Because we cannot distribute a compiled version with the libfdk_aac encoder. https://trac.ffmpeg.org/wiki/Encode/AAC
On OS X we can also package it within Homebrew which is a pretty popular way to get things over there.
I would also suggest that we hold off on focusing on packaging too much until we make the transition to qt5, as it would likely lessen the complication for some of the aforementioned platforms.
Homebrew could be an option. I don't use macOS much, but when I do that's who I get syncthing up and running :-)
Was a transition to qt5 ever decided on? Changing to qt5 would not make the dependency hell any easier, you're just swapping out a handful of libraries with another handful of newer libraries.
@djfun The source code for packaged version would still need to be made easily accessable. Depending on how that gets shipped on other platforms it's not necessarily straight forward such as PyInstaller which creates an auto-extracting exe, and puts the source files in automatically generated locations. So we might need to create a separate repository for it, or offer strictly as a compressed file that can be downloaded from the website.
Releasing GPL version outside of github would probably be ideal so all the innovation happens under the MIT license and we don't have to deal with pull requests that are incompatible with the MIT version. That said if you are not completely set on sticking with MIT, I'd be willing to work on a strictly GPL codebase as well.
I work for Canonical on Ubuntu and Snapcraft. I've been contacted by Wayne directly via Telegram who directed me at this issue. You can find the documentation for Snapcraft (the tool used to create and publish snaps) on https://snapcraft.io
Given that this project is developed using Python and Qt4 you'll be interested in the documentation for the snapcraft python plugin and you should use the desktop-qt4 helper, see the CasterSoundboard snapcraft for an example for an example of how to include a desktop helper. You'll also require a number of interfaces to be enabled, again the CasterSoundboard snap is a useful reference.
Once you've landed a snapcraft.yaml in this GitHub project you can quickly automate the commit, build (on multiple architectures) and publish to process by using https://build.snapcraft.io.
I invite you to join the Snapcraft Discourse forum. The entire Snapcraft and Snap team are present there, so if you have questions ask there :-)
@IamDH4 Yes, you'd be swapping out for a newer set of libraries that
- Aren't EoL'd
- Are more easily accessible and
- Have better cross-platform integrations, HiDPI support and a slew of other nice-to-haves.
While perhaps not a requirement, i think it'll reduce some headaches long term.
Updated setup.py to reflect changes in the feature-newgui branch in commit 8c9914850e9987d4f05e8b88dedb058ffbb4f53f. Changed resource references to use "frozen" path when available. I haven't dug much deeper but this should take care of bundling all the python dependencies (E.G. numpy, pillow-simd, and pyqt5), and creates a portable binary / resource set.
I think the next step is figuring out packaging on each platform, and create build scripts for those platforms.
Linux - I need to read up on snaps a bit, pkgbuild on Arch, and possibly a deb. Windows - We should probably just wrap the resulting freeze with an installer that includes the needed dlls. macOS - Need to wrap everything up in a .app container.
I updated the dependency list to reflect the feature-newgui change from Qt4 to Qt5.
Specified ffmpeg as ffmpeg 3.3 since older versions don't work.
If and when all license issues are sorted out I would be willing to help package this as a Flatpak and potentially adding it to flathub.
@djfun Have you decided on what to do with licensing? Convert to GPL or dual license? Can we get away with a dual license on this same repo, and just make it clear all bundled versions of the application are released under the GPL license?
Is there anything else that needs to be done except switching avp to GPL?
@tassaron2 freeze.py is fully functional under Linux and Windows now correct?
@IamDH4 freeze.py currently works using whatever "PIL" package is installed. There seem to be no problems with freezing on Linux. I have not managed to build Pillow-SIMD on Windows, since there is not much documentation on it and I don't have Windows at home to devote my time to figuring it out. But freezing on Windows does work using standard Pillow.
There are potential issues with making a freeze with Pillow-SIMD, since it uses CPU features that may or may not exist on the target system. This is the reason why it's not included in the standard Pillow version according to the creator @homm. There is currently no simple way to make Pillow-SIMD widely-compatible and take advantage of all the features.
We could do a freeze with only SSE4 enabled for maximum compatibility across x86 systems, and maybe provide two other frozen versions with more/less compatibility. Another solution would be changing back to the old render method that did not use alpha_composite, so standard Pillow would be faster.
@tassaron2 Pillow supports alpha-composite acording to the docs.
http://pillow.readthedocs.io/en/3.4.x/reference/Image.html
The reason pillow-simd was used is for it's performance.
I mention alpha_composite because it's twice as slow using standard Pillow. So the export could probably be faster using standard Pillow if we avoided using alpha_composite when possible. I don't necessarily think this is the best solution but I mentioned it as a possibility.
Supposedly most people have SSE4 support nowadays (it's been around for a decade), and the AVX instructions compared in that chart I linked above are not that much faster, so I think we could stand to have one Windows installer for hopefully 90% of people using Pillow-SIMD with SSE4. Users with old/weird hardware could do a manual installation and use standard Pillow.
People with the newest CPUs could then achieve slightly better performance by doing a manual installation to enable AVX2, which is hard to do on Windows. But the speed difference isn't as major, apparently. All this is based on doing research over the past week; I don't really have any experience with this stuff.
I do plan to devote more time to compiling Pillow-SIMD on Windows and I can make a Windows installer eventually if no one else volunteers. Currently the freeze in a zip is about 300MB because we aren't doing anything to remove unused portions of Qt, so that's not easy to distribute. We could look into adapting something like Hatchet to trim Qt, but it would be a big project.
I have initial Flatpak support working, with some temporary workarounds and limitations.
- There is no desktop entry/launcher item, because that requires both a desktop entry file and an icon. The first step in solving this is creating an application icon.
- Most, but not all encoding options work. I have fixed most of them but a few odd ones remain.
~~- Some features are not enabled in the ffmpeg build, partly due to license issues. I also wanted to start small and not add anything unnecessary.~~
~~- The app is launched via a wrapper because the
setup.pyscript is broken.~~
You can find it here: https://github.com/AsavarTzeth/flathub/tree/com.github.djfun.audio-visualizer-python
You can test it like this:
Download
git clone https://github.com/AsavarTzeth/flathub.git; cd flathub
git checkout com.github.djfun.audio-visualizer-python
Build & Install
flatpak-builder --repo=/home/$USER/testrepo app com.github.djfun.audio-visualizer-python.json --force-clean
flatpak --user remote-add --no-gpg-verify --if-not-exists testrepo /home/$USER/testrepo
flatpak --user install testrepo com.github.djfun.audio-visualizer-python
Run
flatpak run com.github.djfun.audio-visualizer-python
Update
flatpak --user update
Providing an AppImage would have, among others, these advantages:
- Applications packaged as an AppImage can run on many distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
- One app = one file = super simple for users: just download one AppImage file, make it executable, and run
- No unpacking or installation necessary
- No root needed
- No system libraries changed
- Works out of the box, no installation of runtimes needed
- Optional desktop integration with
appimaged - Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
- Can optionally GPG2-sign your AppImages (inside the file)
- Works on Live ISOs
- Can use the same AppImages when dual-booting multiple distributions
Here is an overview of projects that are already distributing upstream-provided, official AppImages.
Notable Python based AppImages include Ultimaker Cura and PrusaControl.
I tried testing the Flatpak, having never used one before, on my home system running Lubuntu 16.04 . After running your commands I had to install this runtime:
flatpak --user remote-add --from gnome https://sdk.gnome.org/gnome.flatpakrepo
flatpak --user install gnome org.freedesktop.Platform//1.6
When launching, the console is spammed with a million GTK errors parsing my weird theme. If I change to a default theme, these errors go away, but there's still an error that stops it from launching as Pillow can't find libjpeg.so.8.
On Fedora 25 the same commands produce a working program, but when I try to export the video I get a strange TypeError I've never seen before (somehow the ffmpeg command has an integer in it instead of a string). Which commit did you use to make this?
@tassaron2 I did mention that you had to install that runtime. It is possible to install it from both sdk.gnome.org and flathub.org.
As for your errors, I thank you for sharing. I will try to reproduce it next week.
The commit used should always be the one in the manifest file. Whenever I get a good build I commit and push to github, followed by syncing my flatpak repo with my web server.
Quick question, are you using flatpak from the ubuntu repositories or from the official ppa? It could be a case of flatpak being too old. That might also explain why you had to explicitly install the runtime; newer versions will search all your added remotes and suggest it for you.
Oh well, it was worth asking. It is late here, so I will try to reproduce your errors tomorrow.
I failed to reproduce your errors on any of my Ubuntu 16.04 machines.
What I did find though is that after a new build flatpak --user update does not detect any updates from https://repo.tzeth.com while my local repository works fine. I would bet the issue lies in how I currently export to the local repo and then rsync to the web server.
I might have to do some more research on how to best sync an ostree (flatpak repo) with another host, and how to maintain a flatpak repo in general.
There were some issues with my first Flatpak repo which were entirely my fault.
I have now taken the time to read the documentation carefully. I then made a new gpg signed repo and made sure that updates work correctly. I tested this on both Arch Linux and on a second machine running Ubuntu 16.04.
Removing the old remote
flatpak --user uninstall com.github.djfun.audio-visualizer-python
flatpak --user uninstall org.freedesktop.Platform
flatpak --user remote-delete tzeth
Install Runtimes
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install flathub org.freedesktop.Platform 1.6
or
flatpak remote-add --if-not-exists gnome https://sdk.gnome.org/gnome.flatpakrepo
flatpak install gnome org.freedesktop.Platform 1.6
Installation
flatpak remote-add --if-not-exists tzeth https://tzeth.com/repo/tzeth.flatpakrepo
flatpak install tzeth com.github.djfun.audio-visualizer-python
Run
flatpak run com.github.djfun.audio-visualizer-python
Updating
flatpak update
Currently based on commit 20905230fe6df814fdaa6173b266dd7de51bd269
Trying to make a package of this application myself was one factor that made me burn out in 2017, which resulted in the project going dormant for a few years. Therefore I'm not going to package it myself; I will instead focus on making the program more stable and make it installable via Pip. I'll also fix any bugs related to "freezing" the app if that's needed for an EXE or something.
The Pillow-SIMD dependency is no longer required when installing with Pip, because that makes the Pip package easier for me to maintain. However Pillow-SIMD does have performance advantages, so it would be ideal to include in packages. But a working package with regular Pillow is better than having no package! Any help is appreciated.
I tried updating @AsavarTzeth's Flatpak and put my work in a new branch. This Flatpak seems to work for me.
I first built the Flatpak using flatpak-builder and put it in a folder called repo. I copied this folder into a virtual machine running Ubuntu 22.04, then installed Flatpak, added the Flathub repo, and installed org.kde.Platform//5.15-22.08 from Flathub. After doing this, I'm able to install Audio Visualizer from the repo folder.
One issue I realized however is that Flatpak docs say not to use com.github in your application id. They recommend setting up a GitHub Page and changing it to io.github, which would be an easy fix (just publish the readme maybe?).
For Flathub we also need an icon.