Discuss how to package Rust/Cargo applications (and other non-standard build systems)

[alexlarsson]

trafficstars

From @moosingin3space on February 6, 2017 17:38

When I read the documentation for flatpak-builder, I see that it wants a project to support the Build API. Rust applications that build with Cargo don't fit as nicely into this mold. I'm curious what the expectation is for a non-standard build system. Should the build system be extended to produce Flatpaks itself, or should flatpak-builder be extended to support arbitrary build systems, in a similar manner to Arch Linux PKGBUILDs?

Copied from original issue: flatpak/flatpak#536

[alexlarsson]

Oh, the build api just makes it easier to build with f-b, you can build whatever you want if you add a custom makefile. Alternatively we're making that not required by https://github.com/flatpak/flatpak/pull/516

[alexlarsson]

From @TingPing on February 6, 2017 18:49

The "Build API" should probably not be mentioned in the first place, it is extremely autotools specific and even already flatpak-builder supports CMake and Meson.

[alexlarsson]

From @moosingin3space on February 6, 2017 18:59

So, regarding Rust/Cargo specifically, we need:

• The ability to run cargo fetch or cargo vendor or something outside the sandbox, so all dependencies can be fetched.
• Run cargo build --release inside the sandbox
• cp some files around to fit the structure of a flatpak

Is all this possible with flatpak-builder?

I'm also interested in packaging Servo, which has a more complicated build system. The system in #516 will support this sort of packaging?

[alexlarsson]

Its not generally possible to run something outside the sandbox like that. The whole point of the build is that we have all the build dependencies inside the runtime, and don't rely on host side tooling. You can however, disable the network sandboxing for a particular module, so that you can run something that downloads dependencies. (At least unless you specify --sandbox to flatpak-builder.)

Still, while this works, it doesn't really match how flatpak-builder normally works. Normally the json specifies the source up to sha256sums of every piece of sourcecode, and flatpak verifies that, so that the build is completely reproducible.

[alexlarsson]

From @lilianmoraru on February 15, 2017 11:47

I've also hit this issue today. I want to make my Rust server runnable only from Flatpak.

[alexlarsson]

From @daa84 on March 9, 2017 18:14

And how about build tools? If i want to download rustc/cargo for build how to do this?

[eyelash]

BTW I created a flatpak for an application that depends on a Rust binary by installing Rust inside the build sandbox. This manifest might be helpful to other people trying to flatpak Rust applications.

[nedrichards]

This kind of functionality would also be very useful for packaging Electron apps inside a Flatpak. They mostly use npm or yarn to specify their dependencies and it'd be much more upstream developer friendly to be able to look inside the package.json and get the dependencies from there at the source download stage. Currently you can pretty easily build flatpak bundles as part of the electron tooling, but that just gives you an x86-64 file and doesn't really give a big advantage for the developer over something like appimage (other than the usual advantage of actually working).

It just feels like language specific package managers (pip, nuget, cargo, npm etc. etc.) are 'a thing' that flatpak should think about a broad solution for (not necessarily just parsing npm manifests!) if we want to be the brain dead easy to implement answer to a lot of problems developers have.

[alexlarsson]

For the record, i made a rust SDK here: https://github.com/flathub/flathub/pull/89

[adityashah1212]

How about having an ability to run a script/command at download stage as a part of sources, but block its access to everything except for the download directory and network?

[adityashah1212]

I have one more idea for this, how about having a plugin for flatpak-builder to download the packages for each respective build mechanisms. Something like maintaining a script for example for each build mechanisms. This script is shipped with flatpak-builder and flatpak-builder triggers the script with respective files (packages.json/Cargo.toml) etc. The script downloads the packages for you and stores them in download dir.

[Ayush1325]

I am interested in improving Rust + Flatpak development and thus would like to revive this discussion.

I know about the existence of flatpak-cargo-generator, but it's not the ideal solution since you need to maintain another file, which needs to be kept in sync with Cargo.lock file.

Meson also has support for Rust now (although things like build.rs are not supported), so I would like to know why cargo isn't directly supported by flatpak-builder. Is it simply that no one has bothered to implement it, or is there something more to it?

[TingPing]

@Ayush1325 flatpak-builder must have a static list of unchanging sources. So you have to have an intermediate tool like flatpak-cargo-generator to get this list.

In order for this to change I think Cargo.lock needs to gain sha hashes of each project listed. Then it would be safe for flatpak-builder to parse it at build time knowing that the sources are reproducible.

EDIT: Turns out Cargo.lock does contain checksums. So I think its reasonably safe to add support for this into flatpak-builder directly.

[ssokolow]

For future reference, this is what an entry in a Cargo.lock with version = 3 schema looks like:

[[package]]
name = "clap"
version = "3.2.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71655c45cb9845d3270c9d6df84ebe72b4dad3c2ba3f7023ad47c144e4e473a5"
dependencies = [
 "atty",
 "bitflags",
 "clap_derive",
 "clap_lex",
 "indexmap",
 "once_cell",
 "strsim",
 "termcolor",
 "textwrap",
]

checksum is SHA-256 and, for leaf dependencies, the dependencies key is omitted. In my experience, path dependencies omit the checksum, but you can't upload crates which use them exclusively to crates.io (it's possible for an entry to have both for easier development), so that shouldn't be relevant.

[TingPing]

So I think a feature that:

• Adds a cargo buildsystem that runs the normal cargo commands to build/test/install
• Add a cargo-lock source type that expands into regular sources, erroring if not reproducable.

Is pretty reasonable if somebody wants to work on that. The hard part is we don't have a toml parser. I guess tomlc99 would do.

[chriskilding]

Are there any updates on the proposed cargo buildsystem? It would certainly make Flatpak packaging of Rust apps much more convenient.

[chriskilding]

I've opened #564 to have a go at implementing this. I haven't worked on flatpak-builder before, so will gratefully accept any help :)