beeware
Gtk4 migration
Describe your changes in detail
This PR migrates from Gtk3 to Gtk4, from3to4, and taking also into consideration Gtk5, from4to5.
Note the following:
- [x] This changes required GTK4 v4.10 at least (Primary, because we use
Gtk.AlertDialogto implement TogaDialog. Otherwise, we either rebuild it from scratch usingGtk.Windowor using deprecated Gtk API). - [x] Layout mechanism has been completely refactored.
- [x] Applying styles on widgets has been changed.
- [ ] Tree widget needs a refactoring by one of the new Gtk4 widgets instead of removed Gtk3 widget (
Gtk.TreeModel). - [ ] Menus and Commands will be addressed in #1931
- [x] Fixes some bugs.
Audit checklist:
- [x] app
- [x] fonts
- [x] icons
- [x] images
- [x] keys
- [x] paths
- [x] window
- [x] activityindicator
- [x] base
- [x] box
- [x] button
- [ ] canvas
- [ ] dateinput
- [ ] detailedlist
- [x] divider
- [x] imageview
- [x] label
- [ ] multilinetextinput
- [ ] numberinput
- [x] optioncontainer
- [ ] paswordinput
- [ ] progressbar
- [ ] scrollcontainer
- [ ] selection
- [ ] slider
- [x] splitcontainer
- [ ] switch
- [ ] table
- [ ] textinput
- [ ] timeinput
- [ ] tree
- [ ] webview
What problem does this change solve?
This PR migrate from Gtk3 (have a lot of deprecated functions and old UI) to Gtk4 and does not violate what will be in Gtk5.
Related issues:
Refs #1931 Refs #1935
PR Checklist:
- [x] All new features have been tested
- [x] All new features have been documented
- [x] I have read the CONTRIBUTING.md file
- [x] I will abide by the code of conduct
Wow - this is a big chunk of work :-) Thanks for taking on this project.
As I indicated in #1935, we're in the middle of a widget audit right now, so there's a lot of churn happening with widget implementations - but we're also building GUI test cases for all widgets. Those tests (and the test probes in the gtk/tests_backends folder) will be a key part of acceptance testing for this PR.
At present, this PR is failing CI because of basic formatting (towncrier and pre-commit failures) - I'd strongly recommend setting up your development environment as described in the Toga docs to avoid these issues. Once those problems are resolved, you're going to hit a bunch of problems with the testbed app - this is the test harness we're using to validate GUI backends. That testbed will require modifications to run GTK4 code, as will the GTK probes (defined in the gtk/tests_backend folder) that allow the GTK backend to be tested. Procedurally, I'm unlikely to take a deep dive into this code until those tests pass.
@MuhammadMouradG: We recommend you install pre-commit into your own environment as shown here, so you won't need to wait for CI to run it.
Suspend the work on this PR for one week because I will be unavailable on the internet.
Until then, We need decisions on the following problems:
- The implementation of dialogs depend on the two widgets that are deprecated and replaced by new ones on >v4.10, the problem is that some platforms are still using version of Gtk4 less than 4.10 and the new widget still not exist. So, what do you think about the best decision?
- The package of webkit2 for Gtk4 have a bug, see this, so, what is the best decision for that.
- It is not a problem but a sepetated task, the
treewidget needs reimplemented by usingGio.ListModelbecause it depends on the `Gtk.TreeModel' that has been deprecated.
Suspend the work on this PR for one week because I will be unavailable on the internet.
Until then, We need decisions on the following problems:
- The implementation of dialogs depend on the two widgets that are deprecated and replaced by new ones on >v4.10, the problem is that some platforms are still using version of Gtk4 less than 4.10 and the new widget still not exist. So, what do you think about the best decision?
I'm not sure I fully understand the issue here. Are you saying that there aren't any dialogs in 4.0-4.10? Or that the 4.0-4.10 dialogs use the 3.X API, but that API been deprecated in more recent versions of GTK? Looking at the docs for GTK.Dialog, I can see that it's been deprecated, but it's been deprecated in favor of Gtk.Window... which definitely exists.
I guess I'm not understanding the migration path here. But - we can't require users have GTK 4.10 binaries, as that version is only a couple of months old. So, we'll need to use an older API, even if it is marked as deprecated.
- The package of webkit2 for Gtk4 have a bug, see this, so, what is the best decision for that.
Again, I'm not fully sure I understand the question. If the package is broken, it's broken. That would seem to be a blocker on GTK4 adoption. I don't know what the alternative approach is.
- It is not a problem but a sepetated task, the
treewidget needs reimplemented by usingGio.ListModelbecause it depends on the `Gtk.TreeModel' that has been deprecated.
Can Gio.ListModel be used for Gtk.Tree in GTK3? i.e., could we do the migration to ListModel now?
Hi again!
Are you saying that there aren't any dialogs in 4.0-4.10? Or that the 4.0-4.10 dialogs use the 3.X API, but that API been deprecated in more recent versions of GTK?
I mean the second one.
Looking at the docs for GTK.Dialog, I can see that it's been deprecated, but it's been deprecated in favor of Gtk.Window... which definitely exists.
In more details, the current implementation of Toga.MessageDialog and the others that inherit from it, uses Gtk.MessageDialog as the native widget for the backend implementation. The problem is that this Gtk.MessageDialog widget has been deprecated in favor of new widget Gtk.AlertDialog in version "4.10". This Gtk.AlertDialog provides a simple migration but will produce an issue for users with older than "4.10" API.
However, I didn't notice that this GTK.Dialog, which Gtk.MessageDialog inherits from it, was deprecated in favor of Gtk.Window, so, as I understand you propose to use Gtk.Window instead of Gtk.AlertDialog and Gtk.MessageDialog which fulfills the purpose we want and provides support to users with older API.
I guess I'm not understanding the migration path here.
The intention of this PR, in addition to what I refer in the PR topic, is to minimize the effort of future migration to the new releases of "GTK" by making it is just as changing the version of the "GTK"; This means avoiding using any methods/classes that have been or will be deprecated in the next or current releases.
But - we can't require users have GTK 4.10 binaries, as that version is only a couple of months old. So, we'll need to use an older API, even if it is marked as deprecated.
And therein lies the problem. If we make an effort in this migration but we used deprecated methods/classes, what is the purpose of the migration then? Now, we are putting effort into this migration because we ignored the carefully choice of methods while we were using Gtk3, of course, this is also due to the a lot of changes in Gtk versions, but isn't it logical that using deprecated methods will make it difficult for future migration to newer versions and will make the current effort is useless? What do you think?
Again, I'm not fully sure I understand the question. If the package is broken, it's broken. That would seem to be a blocker on GTK4 adoption. I don't know what the alternative approach is.
So, you propose that the best decision now is to suspend the migration until we have a solution to this problem.
Can
Gio.ListModelbe used forGtk.Treein GTK3? i.e., could we do the migration toListModelnow?
From the Gtk documentation and as I understand, yes with more features. For more details see this.
But - we can't require users have GTK 4.10 binaries, as that version is only a couple of months old. So, we'll need to use an older API, even if it is marked as deprecated.
And therein lies the problem. If we make an effort in this migration but we used deprecated methods/classes, what is the purpose of the migration then? Now, we are putting effort into this migration because we ignored the carefully choice of methods while we were using Gtk3, of course, this is also due to the a lot of changes in Gtk versions, but isn't it logical that using deprecated methods will make it difficult for future migration to newer versions and will make the current effort is useless? What do you think?
I agree it's a challenge. It's a challenge that I'd very much like the GTK developers to engage with, because it's entirely of their own making. It's not a problem we have on any of the other backends; deprecations are introduced slowly, and old APIs are preserved. You can get new functionality if you want to use the new APIs, but the old APIs continue to work without being noisy about it to end users. On the other hand, there are APIs that we're using on the GTK backend that have undergone two phases of deprecation in the GTK versions in the effectively 5 year window of OS releases that we support. And, sure, the old APIs still work... but they're noisy, and I've lost track of the number of users who report the deprecations as bugs in the GTK backend.
But - GTK is what it is, so we need to accomodate it. There's essentially 4 options:
- Force users on old versions of GTK to upgrade.
- Use the oldest API, and make a case-by-case decision to upgrade as availability allows
- Provide multiple implementations of a single widget.
- Provide 2 backends.
1 isn't practical, as there are still-supported and completely reasonable versions of Linux (e.g., Ubuntu 20.04) that don't provide any stable GTK4 packages as part of their officially supported repository. Sure, you could download and compile from source - but that's a pretty high barrier of entry to put on usage of Toga.
Option 4 is the nuclear option - start from scratch with an independent GTK4 backend. Although this might sound attractive, it actually won't fix the problem, because GTK is happy to deprecate APIs in the middle of a major cycle. Having an isolated GTK4 backend won't help us in this Dialog discussion, unless we set the minimum GTK4 version to 4.10... in which case, almost nobody will be able to use the GTK4 backend; and if we do that in anticipation of the fact it will take us a while to get the GTK backend to completion, I guarantee that by the time 4.10 is commonplace and 4.20 is available, there will be some other API that has undergone a "likes are now florps" transformation. Or, GTK5 will be out, and we'll have 3 GTK backends... which is way too many backends :-)
Essentially, a "no deprecation warnings on GTK" goal is essentially unattainable if we're going to support as many Linux users as possible. The best we can do is to minimize our own headaches, which is what option (2) and (3) are.
Option 2 is making a case-by-case evaluation of which new APIs we can adopt. We did this recently with the set_wmclass API on the main window - when we introduced the use of that API, it wasn't deprecated; we carried the deprecated API until it was clear nobody was actually benefitting from it any more, at which point we removed it. In the Dialog case, 4.10 deprecates the Dialog APIs, but almost nobody is on 4.10, so nobody will see the deprecation warning from the old APIs. Eventually, most users will be on 4.10, so we can upgrade the GTK backend to use the new API, thereby silencing the backend warning.
Option 3 is something we are also doing in the current codebase for a handful of cases. For example, Android didn't introduce justification to it's text APIs until Android API 26, so we gate the logic that uses that API. There's a handful of other APIs where we've done this. It's usually a simple "if/else" to make a single API call optional - but in theory, there's no reason we couldn't do this on whole widgets, assuming there's a need. As long as we've go test coverage, and the maintenance overhead of having 2 completely different widget implementations isn't too onerous, I can see this working well.
Again, I'm not fully sure I understand the question. If the package is broken, it's broken. That would seem to be a blocker on GTK4 adoption. I don't know what the alternative approach is.
So, you propose that the best decision now is to suspend the migration until we have a solution to this problem.
Webkit is just 1 widget; we can do a lot of work on the migration before that issue becomes critical. However, it would absolutely be a blocker on a hard switch over to GTK4.
Can
Gio.ListModelbe used forGtk.Treein GTK3? i.e., could we do the migration toListModelnow?From the Gtk documentation and as I understand, yes with more features. For more details see this.
In which case, patches welcome :-) I've just completed an audit of Table (#2011) and Tree (#2017), so we have full runtime test coverage of those widgets now. If we're able to transparently change the underlying data source to use ListModel rather than ListStore/TreeStore, that seems like a good "future-proofing" improvement that can be done independent of the GTK4 upgrade.
From a quick survey of the ListModel API, it also looks like it has improved potential for abstracting data access back to the underlying data model, so we don't need to maintain a parallel copy of the data. If that's an option, then it sounds like something worth pursuing.
Thanks for details.
Now, we have a clear vision on the issues that I mentioned at the description of this PR. A last thing and for this PR to be useful to other contributors, and for all issues discussed to be complete, what do you think the best option we can use, 2nd or 3rd, for Dialog widget? I.e., we can provide the two implementations of dialogs one for > "v4.10" and other for < "v4.10", as I did here. Or we can use what we proposed, using Gtk.Window in favor of Gtk.Dialog, which overcomes the headache of widget deprecation, but we will spend some effort.
Finally, what the status of this PR?
Thanks for details.
Now, we have a clear vision on the issues that I mentioned at the description of this PR. A last thing and for this PR to be useful to other contributors, and for all issues discussed to be complete, what do you think the best option we can use, 2nd or 3rd, for
Dialogwidget?
I honestly don't have any firm opinion at this point, mostly because I know almost nothing about GTK4. At the very least, option 2 will be easiest to implement for now, and will satisfy the majority of GTK4 users in the immediate term. We can always migrate from 2->3 in the future (where "future" could be very soon, if it needs to be :-) )
Finally, what the status of this PR?
In what sense? You're the one who opened it, so... it's whatever status you want it to be :-)
From my perspective, it's marked as a draft, currently has merge conflicts with main, and is failing the Linux testbed tests. Those three issues are the reason I haven't taken a detailed look at it; and I'm unlikely to do a deep review until they are - especially the last one, since the testbed is now our "does it work?" standard. I'd also expect to see the outstanding audit PRs (currently #1903, #1969, #1984, #1996, #2011, #2017, #2020, #2025) merged into this PR, as they have major API updates and complete testbed tests. The failing GTK backend tests are less of an issue, and they should all be deleted in the very near future, so I wouldn't worry too much about those.
I'm happy to take a preliminary review pass if that's what you're asking; let me know if that's the case.
At the very least, option 2 will be easiest to implement for now, and will satisfy the majority of GTK4 users in the immediate term.
Sorry for being confused, but did you mean by option2 providing the 2 implementation or using Gtk.Window?
I'm happy to take a preliminary review pass if that's what you're asking; let me know if that's the case.
Yes, please.
- On the subject of which: the testbed app is where the real action is. From my perspective, this PR isn't serious until the testbed app is running and passing. So far, this PR isn't barely touching the gtk/tests_backend folder.
Yes, I know it is important stage, but the problem is that this testbed tests stop in the importing phase of webkit package, so, what do you think is the solution to this?
As a side note, I manged importing WebKit on debian 12 by install libwebkitgtk-6.0-dev package.
- You're going to want to integrate the outstanding audit PRs fairly soon, as they include some pretty radical changes to the GTK widget implementations. If you merge https://github.com/beeware/toga/pull/2025, https://github.com/beeware/toga/pull/1996 and https://github.com/beeware/toga/pull/1903 into this PR, you should get all the relevant changes. This also covers almost all the gtk/tests folder deletions.
I will do, I'm just waiting until these PRs be merged with main branch, except if these will not merged soon!
Yes, I know it is important stage, but the problem is that this testbed tests stop in the importing phase of
webkitpackage, so, what do you think is the solution to this?
In the short term - commenting out the import (or any other code that can't be executed) with a big FIXME/TODO comment is fine. However, a real solution for the problem will be a blocker on this PR being merged.
As a side note, I manged importing
WebKiton debian 12 by installlibwebkitgtk-6.0-devpackage.
Is that webkit1 or webkit2? Based on the package name, it sounds like the former, and we definitely need webkit2.
- You're going to want to integrate the outstanding audit PRs fairly soon, as they include some pretty radical changes to the GTK widget implementations. If you merge [widget Audit] toga.DetailedList #2025, [widget Audit] toga.OptionContainer #1996 and Add additional testbed tests for font #1903 into this PR, you should get all the relevant changes. This also covers almost all the gtk/tests folder deletions.
I will do, I'm just waiting until these PRs be merged with main branch, except if these will not merged soon!
I'd expect them to get merged over the next couple of weeks, as Windows and Android support is completed for each of them. However, the GTK portions of these PRs are all completed, and should only see minor modifications going forward.
The more I dig into this, the more concerns I have about exactly how we're going to be able to land this PR.
Here's a survey of the state of GTK4 support at present in all the still-supported versions of the "big" distros:
- Ubuntu
- 20.04 - Not available
- 22.04 - 4.6
- 23.04 - 4.10
- 23.10 - 4.10
- Debian
- buster - Not available
- bullseye - 4.8
- bookworm - 4.8
- sid - 4.8
- Fedora
- 37 - 4.8
- 38 - 4.10
- 39 - 4.11
- RHEL
- 6 (RHEL) - Not available
- 7 (CentOS 7) - Not available
- 8 (Centos Stream 8) - N/A
- 9 - 4.4
When it comes to testing - we're using Github Actions for CI, so that means we have access to Ubuntu 20.04 and Ubuntu 22.04 for testing purposes. Since 20.04 doesn't have GTK4 at all, that means in practice, we're only in a position to test GTK 4.6, and we won't be in a position to test anything more recent around this time next year when Ubuntu 24.04 becomes available in Github Actions configurations.
(Yes, I know we can tell Github to use different base images - but there literally isn't a stable OS base image we can use at present that includes GTK 4.12 - we'd have to bake our own. But at that point, we're not testing on a OS image that anyone has in practice)
I'm really not sure where this leaves Toga's GTK4 story.
On the one hand, it's very clear from GTK's documentation that they're charging forward, with no intention of supporting GTK3, and some discussions referencing GTK5. They clearly consider GTK4+ to be the future. But there are plenty of Linux users who don't have access to GTK4 at all through their stable channels.
I'm not sure how we can accomodate both bleeding edge API development, and the fact that there are Linux distributions that are supported and in active use that won't have access to that bleeding edge.
I really don't have an appetite for a standalone toga-gtk4 backend - especially since the idea of "GTK4" as a stable target seems to be a complete mirage anyway (c.f., 3 APIs for CSS loading in 5 micro versions). At the moment, it seems like our best option is to have lots of "if version > X: else..." logic in the code, multiple testbed backend implementations so we can have clearly independent probes for testing various configurations, and try to minimize these branches with judicious use of the silencing of warnings.
Yes, this is going to be a major complication on the codebase... but I'm not sure I see a better option. If you've got ideas, I'm open to other suggestions.
FYI: #2075 has now landed, so the widget audit is complete; we now have 100% test coverage for all widgets and features.