SpacingBat3
Package webcord as a flatpak
Is your feature request related to a problem? Please describe. With rise of immutable linux distros like Silverblue, Kinoite and especially Steam OS 3, it's more needed than ever to have everything available as flatpaks. All 3 mentioned distros use flatpaks as their primary means of getting software,
Describe the solution you'd like Preferably, webcord gets published on flathub.org. But having a .flatpak installer in Releases page would be good too.
Describe alternatives you've considered
- Snaps: are largely only present in Ubuntu and a small number of Ubuntu based distros, and manjaro, are also slow to startup due to compression.
- AppImages do not integrate well, require not only more disk space but a whole separate app to manually integrate it with the OS, if you can call creating a .desktop file proper integration.
- None of these two provide the security of flatpak and it's heavily used Portals system.
- I tried installing webcord rpm in a fedora toolbox, and the app worked fine, but terminal had to be used for installation and the app has to be launched from the terminal every time too. Overall not very good experience, and doesn't work on Steam OS 3 due to lack of podman in the default image.
Additional context I am aware of the decision to wait out on distro-agnostic packaging formats mentioned in #66, however it's been nearly a year and a lot has changed since then.
From what I saw, not much changed within Electron Forge for making a Flatpak target. (At least) the issue #2662 still blocks me from using it as a build target. And while there's a workaround pointed here, it could be a bit confusing for those who want to package WebCord on their own and could not since this maker fails to install the required dependencies.
However I had some plans about working on the alternative to Electron Forge CLI and maybe even some of their makers. They were using a lot of deps (at least once I looked at this project), some of which were even deprecated – in fact, Forge dependencies was the reason why I couldn't release my app once: it depended on sabotaged colors library, on which suprisingly WebCord depended as well, yet it still was functional as it used the untouched safe variant (it now uses my own library: @spacingbat3/kolor, as an anti-trust action to the libraries like that).
Preferably, webcord gets published on flathub.org.
It is unlikely I will publish my app to the third-party repositories, at least for now. The more likely is I will publish .flatpak files for each of the architectures.
Right now I'll flag this issue as upstream and wontfix, but either a fix on the Electon Forge's side or developing my own maker will make me to revisit this issue.
It would be possible to extract the appimage, like heroic did it
It's not the problem with packaging (or creating a tarball), it's the problem with maintaining it. Any solution given would require from me to either develop my own maker or/and work on separate action for Flathub (if I would like to publish it there). It would be a hell to manually bump the metadata, resolve errors, build and publish packages for every Linux distributable format which is there currently. This is why a solution to automatize the entire process is needed.
Perhaps you could use flatpak-external-data-checker so the bot automatically bumps for you? Though that's if you decide to extract the AppImage. It's more of a workaround than something you want, but I just wanted to chime in and let you know that this exists if ever you are interested with some automation.
It would be possible to extract the appimage, like heroic did it
It's not the problem with packaging (or creating a tarball), it's the problem with maintaining it. Any solution given would require from me to either develop my own maker or/and work on separate action for Flathub (if I would like to publish it there). It would be a hell to manually bump the metadata, resolve errors, build and publish packages for every Linux distributable format which is there currently. This is why a solution to automatize the entire process is needed.
on top of what TheEvilSkeleton said, I'd also be down to maintain the flatpak on flathub, if that's ok with you I also got a working build already that just needs some little tweaks to work optimally
flatpak built by @axtloss is a repackaged appimage, which can work fine, i guess, but that'd be basically strapping a toolchain on top of a toolchain.
I tried building webcord "the intended way" by following their official documentation. They have an electron base-app so your electron app doesn't take more space than absolutely needed, and it all can be neatly organized into a single manifest you give to flathub and it builds & publishes the app for you if you throw in some AppStream metadata.
Unfortunately I was met with a funky npm error I was not able to troubleshoot as my specialty is not javascript at all. I even find it a little funny because the culprit is around your fork of kolor :)
the experiment can be found here: https://github.com/vega-d/WebCord_flatpak_build and npm logs here: https://github.com/vega-d/WebCord_flatpak_build/issues/1
In my opinion if a flatpak version (published on flathub or not) sees the light of day, .rpm and maybe even .appimage can be dropped as pretty much all distros (and especially fedora) support flatpaks right now, making it easier to maintain since this way you don't have to maintain and test webcord in as many packaging formats.
First of all, thank you all for your work and research for packaging Flatpaks 👍. This showed me how important it is for the community. Hopefully, it will be a matter of time to have WebCord packaged into Flatpaks or/and published on Flathub repository.
Unfortunately I was met with a funky npm error I was not able to troubleshoot as my specialty is not javascript at all. I even find it a little funny because the culprit is around your fork of kolor :)
@vega-d I've checked the logs and it seemed to actually fail on @spacingbat3/jsonc-parser package, which is not a problem as WebCord 3.2.0 now removes the use of JSONC format. So instead packaging WebCord 3.1.4 try to package WebCord 3.2.0 to see if that will work any better. You are also free to use other Node package manager, like yarn.
(...) rpm and maybe even .appimage can be dropped (...)
Even if there will be Flatpaks published, I'm definitely not going to deprecate any other package formats. These do no harm to anyone and I believe there could be some group of people who prefer distribution-specific packages.
So instead packaging WebCord
3.1.4try to package WebCord3.2.0to see if that will work any better.
Just tried, now I am getting same ENOTCACHED error on a completely different npm package.
I also meant jsonc-parser and not kolor, not sure why kolor even crossed my mind?
I believe there could be some group of people who prefer distribution-specific packages.
I think it'd be quite small, especially on fedora.
You are also free to use other Node package manager, like
yarn.
I am not completely sure how to use yarn, but dependency parser script for flatpak-builder mentions supporting it so gonna look from there...
it'd also be possible to release webcord as a tarball, which would make the packaging process easier and not a workaround like my appimage solution
It'd also be possible to publish my version with the appimage workaround on flathub first and once the native build works, or a tarball is released, it'd be very easy to just move to that other build type. As I said already, I'd also be ready to maintain the flatpak on flathub. Edit: Here's the flatpak manifest that I'm using right now, works with no issues com.github.spacingbat.webcord.txt
After an intense 10+ hour session of having very non pleasurable intimate time with electron I have finally came to a working flatpak that works just as well as any other releases you currently have.
It involves a two step process of using electron builder to compile a binary with all it's dependencies into a tar.gz archive, and then repacking that into a .flatpak.
The process is not exactly "how the founding fathers intended" but I'd say it's better than repackaging an AppImage. Also as a byproduct you have a workable .tar.gz tarball someone might appreciate.
(btw it also can do pretty much every distro-specific packaging format under the sun including snap, mac and windows binaries of all kinds, and it does that in parallel, even with integration into github CI, you might want to look into it more)
Shell scripts are available here: https://github.com/vega-d/WebCord_flatpak_build
it builds straight from master of your repo.
As a sidenote, this can be easily adapted to be published on Flathub, arm builds included.
I've digged through Flatpak and I believe I have found a cause of previous issues and the current ones we need to find a way for.
Notes about previous issues
Digging through the Flatpak documentation, I've noticed this:
(...) By default,
flatpak-builderdoesn’t allow build tools to access the network. This means that tools which rely on downloading sources will not work. Therefore, Node.js packages must be downloaded prior to running the build. Setting theelectron_config_cacheenvironment variable means that these will be found when it comes to the build. (...)
Flatpak Documentation, Electron section, The application module
This tells you more about what you did wrong:
Since even simple Node.js applications depend on dozens of packages, it would be impractical to specify all of them as part of a manifest file. A Python script has therefore been developed to download Node.js packages with NPM and include them in an application’s sources.
Flatpak Documentation, Electron section, Bundling NPM packages
A few things has been described there, so let me summarize the issues with your previous work:
flatpak-builderhas no access to the network at build time, so either all modules needs to be downloaded before build is going to be initialized or (as described in Flatpak docs) generate a JSON file as a cache snapshot;- all these errors were due to missing dependencies at cache – I guess you haven't generated it, but copied from sample application;
- a lot of other issues are with your manifest as well – for instance,
npm run packagecan be used instead ofmakeand flags could be added to disable update notifications and build stable releases;
I've been experimenting with Flatpaks recently on my own, and I might push the repository similar to yours, except with functional manifest file and generated-sources.json.
The current issues
However, a new issues to discuss on that has recently come to my mind:
- Can we really publish it to Flathub and keep the cache up-to-date without manual interaction? How?
- If it isn't possible, is it still worth it to develop any separate repository for packaging? Why not fix the issues with
electron-forge?
And since I need to re-implement a maker for AppImages anyway (the current one's code is terrible), I'll probably work on Flatpaks as well.
A few things has been described there, so let me summarize the issues with your previous work:
* `flatpak-builder` has no access to the network at build time, so either all modules needs to be downloaded before build is going to be initialized or (as described in Flatpak docs) generate a JSON file as a _cache snapshot_; * all these errors were due to missing dependencies at cache – I guess you haven't generated it, but copied from sample application;
Oh I am well aware of how caching is required when building a flatpak. (in fact, I have an app on flathub written in python that has caching working perfectly fine)
My original issues with jsonc-parser and later other packages weren't from a complete lack of cache, but rather due to how npm absolutely refused to either cache properly or use the cache it made because I was giving it generated-sources.json with the correct links and summaries I got from the script.
As far as I managed to troubleshoot it, the culprit is somewhere in npm. And for heck's sake I can't find where exactly.
- for instance,
npm run packagecan be used instead ofmake
I purposefully avoided using scriptlets from npm run, as they call into electron-forge since that's what you have in your package.json.
- flags could be added to disable update notifications and build stable releases;
That I agree with. I could implement all of that, but it was out of scope for my experiment. The goal was making an initial flatpak with an automated building process. While it does compile right now, I admit there's still a long way to go in terms of stuff to improve.
Can we really publish it to Flathub and keep the cache up-to-date without manual interaction? How?
When you push an update to the manifest (through git) to flathub repo, a bot automatically picks it up and sets it to be built on Flathub's CI. It creates two virgin-clean sandboxes to build in (one for arm and another for x86-64). This way every time you push an update to Flathub manifest the cache is guaranteed to be generated from scratch. This is not an issue to be concerned about.
- If it isn't possible, is it still worth it to develop any separate repository for packaging?
A separate repo for packaging is inevitable if you're looking to publish on flathub. It's management and updating can be handled automatically with some github actions and a couple shell scripts though.
And since I need to re-implement a maker for AppImages anyway (the current one's code is terrible), I'll probably work on Flatpaks as well.
While trying to make flatpak I ~~accidentally~~ made AppImage instead a couple of times. If you edit yaml configuration file by adding a - AppImage next to - dir you will also get a perfectly working appimage while compiling the tarball. (Also works for snap and more)
My original issues with jsonc-parser and later other packages weren't from a complete lack of cache, but rather due to how npm absolutely refused to either cache properly or use the cache it made because I was giving it generated-sources.json with the correct links and summaries I got from the script. As far as I managed to troubleshoot it, the culprit is somewhere in npm. And for heck's sake I can't find where exactly.
Yeah, I've just noticed how this script really works and I think at best it would be to fix Electron Forge. There's a lot of problems with Flatpaks right now and their packaging is far from being ideal. And while that would mean I wouldn't publish it to Flathub, I think it's just something I will need to eventually work on.
So if I understand it correctly, you dismiss the idea of packaging as a flatpak because npm doesn't work directly in the flatpak building process? Completely ignoring the fact that both me and @vega-d gave solutions to that and even proved that they work properly? You're purposely choosing the harder way, which will also take longer, for no apparent reason.
So if I understand it correctly, you dismiss the idea of packaging as a flatpak because npm doesn't work directly in the flatpak building process? Completely ignoring the fact that both me and @vega-d gave solutions to that and even proved that they work properly? You're purposely choosing the harder way, which will also take longer, for no apparent reason.
Not really, as I still need to know how to package it in the first place to create a maker. And I will use what I've learned from this to make it working. Apart from that, makers have an access to already installed packages, so it might be easier to just create a flatpak than playing with the generation script.
And I still need to write the AppImage maker anyway – probably I start from this one before working on the Flatpaks since I have a better experience with bare (minimum tools) AppImage packaging.
well to package it you just add the flatpak manifest (which me and vega already made) to flathub and let the bot build it. the maker would optimally create a .tar.gz when using vegas manifest (which is better) or an appimage for the manifest I provided. The repo from vega also has a tarball script already in it.
additionally, vega seems to have made a script to build appimage and tarballs already, as they mentioned it here
[...] I'd also be down to maintain the flatpak on flathub, if that's ok with you
-- @axtloss, https://github.com/SpacingBat3/WebCord/issues/116#issuecomment-1140500714
@SpacingBat3 what are your thoughts?
[...] I'd also be down to maintain the flatpak on flathub, if that's ok with you
-- @axtloss, #116 (comment)
@SpacingBat3 what are your thoughts?
Why not? If someone wants to host a WebCord on Flathub by themselves, they can do it without my prior confirmation. I probably won't host it myself, at least now.
Right now I will take my time to work on developing a minimal AppImage maker, then move into minimal Flatpak maker. Why they're minimal? As I want to use as least tools as possible, removing the complexity and dependence on third-party modules and binaries. I will also write them to do tasks asynchoniusly.
Why not? If someone wants to host a WebCord on Flathub by themselves, they can do it without my prior confirmation.
I just thought I'd ask before because I know that flathub prefers consent from the developer/maintainer of the project if someone else is packaging the flatpak.
Why not? If someone wants to host a WebCord on Flathub by themselves, they can do it without my prior confirmation.
I just thought I'd ask before because I know that flathub prefers consent from the developer/maintainer of the project if someone else is packaging the flatpak.
When it comes to packaging and hosting, I feel like the community should be allowed doing it, unless developers themselves want to adopt it. I believe AUR itself has similar policy, except "trusted user" is who adopts the package and I doubt original developers can gain control on the packages maintained by someone else.
Today our (me and @axtloss) flatpak manifest was merged into flathub.
The manifest file can be found here: https://github.com/flathub/io.github.spacingbat3.webcord
And the app can be installed through software stores on many distros and from flathub directly
A couple questions regarding Metadata though.
@SpacingBat3 how would you prefer to be credited, on flathub? right now we put a placeholder "Contributors of Webcord", but we can change it to your nick name, your legal name or would you like it to keep as is?
Also, would it be possible to throw in this GitHub Workflow file into your /.github/workflows? It triggers a build to be made and published on flathub when you make a release. Having to keep a fork of WebCord around solely for monitoring releases is kind of silly
@SpacingBat3 how would you prefer to be credited, on flathub?
You can just add a link to my repo as a homepage/source repository. I believe Flatpaks supports long descriptions, so I would also advise you to use first paragraph / short description for a notice like: a community maintained Flathub package of WebCord and probably then write the general description about what WebCord is. You can then add an additional section that flathub repository is maintained by (...) and source repository is maintained by SpacingBat3.
You can just add a link to my repo as a homepage/source repository.
That's already there in the metainfo file. There are links to your repo, how to submit an issue and how to translate. It's all really neatly presented imo.
What I'm not sure about is what to put here, right under the name at the very top of the page:
It only supports a short phrase, and has tag name of <developer_name> so I'm not too sure how to credit you there in a way that's acceptable. That's why I'm asking.
It only supports a short phrase, and has tag name of <developer_name> so I'm not too sure how to credit you there in a way that's acceptable. That's why I'm asking.
You can just put there SpacingBat3, unless there's a way to add an author email/homepage, then you can add full author details from package.json.
I've added Electron Forge Flatpak maker until I won't develop a better ones on my side, so simple Flatpaks will be distributed in WebCord v3.4.0 releases. Probably they will lack a proper OS integration through.
Unfortunately, flatpaks still needs to be delayed in GitHub actions, but at least they can be enabled for the individuals that wish to try building them on their own with WEBCORD_BUILD=true variable. And I know a lot more what's causing this issue, so there's a little hope it will be fixed someday and that I won't repeat it in my own maker.