Proposal for making flatpak-builder more workable for development workflows and other IDEs

[refi64]

TL;DR: There are two ideas I have to make using flatpak-builder easier for both IDEs and command-line tools.

Right now, if I modify a module in a Flatpak, the entire thing is rebuilt. This is great for build consistency, but terrible when it's the module for the project you're working on. GNOME Builder works around this by using --stop-at= to stop at the current module and then running flatpak build manually. Of course, this is very much specific to GNOME Builder.

Another problem is parsing the manifests. Flatpak-builder now supports both JSON and YAML, meaning that any IDE with built-in Flatpak support has to both parse both formats in the exact same way, as well as resolve all the imported module files.

Here's my idea to solve the two problems:

• flatpak-builder --reuse-build-dirs=module-name will reuse the same build directory for the given module. This would basically set up all the sources, then do a build, all in the build directory that was used before.
• flatpak-builder --dump-manifest my-manifest would parse the manifest (either JSON or YAML), parse all the modules that are in separate files, put them all together, and print out the resulting JSON.

Thoughts? :grin:

[TingPing]

flatpak-builder --dump-manifest my-manifest would parse the manifest (either JSON or YAML), parse all the modules that are in separate files, put them all together, and print out the resulting JSON.

I agree that would be nice, and the logic is already there (the manifest is added to the end app/runtime).

[alexlarsson]

In my work on flatbox I also want to be able to use parts of f-b. For instance, to be able to run the build steps from individual modules in flatbox, but in an already entered sandbox.

However, the existing f-b is very much targeted towards "build the entire thing, repeatable", and adding other features in that model is kinda weird. So, I think the solution is to add either make libflatpak-builder, or to make a separate flatpak-builder-tool executable that does these kinds of things.

[louib]

@refi64 @alexlarsson I agree that flatpak-builder is not really suitable as a development tool from the command line right now. But I feel like it could be perfectly suitable for a terminal based development workflow with just a few changes.

First, I feel like there could be an option to update the project files in the cache when a source module is configured with "type": "dir". Maybe something like --sync-local-modules. The other option would be a --reuse-build-dirs, as was proposed by @refi64, which could be used along --keep-build-dirs to reduce the time it takes when rebuilding a module that changed (most of the time, the current module).

Currently, I use f-b for development on the terminal by:

1. Setting the type of the current (last) module to "dir" in the flatpak manifest (and changing the path accordingly).
2. building the project once using --keep-build-dirs and -v.
3. locating the flatpak commands from the output above and using the commands along with an rsync to update and compile quickly my code from the flatpak sandbox.

It works, but I feel like the last step could be handled by f-b.

[toger5]

This would indeed be super nice. It is a clean way to develop to not let the packagemanager pull all the dev files and insted use the flatpak sdk's. what is the state on this?

[louib]

I have a proof of concept for a --reuse-build-dirs option but would like to hear @alexlarsson's opinion on that option before going further.

[refi64]

So I actually tried this a while back, just as an ugly hack to avoid rebuilding Chromium from scratch while doing quick tests. However, it turned out to be surprisingly...messy, and not as useful as I had thought it would be. (That being said, to be frank, I don't remember all the reasons why exactly, I just remember dropping it.)

[1player]

Has the situation changed since last update? I'm trying to hack on a project that is much easier to build in Flatpak than on the host, but the workflow I should adopt with flatpak-builder is lost on me, as I have to nuke my build and .flatpak-builder directories every time I make a change, which isn't... ideal.

[louib]

@1player I suggest you have a look at fenv. I've been using it for a few months and it is focused on local development workflows. There are no official releases of the app yet so you will have to build it from source though.