[WIP] Qt6 Conversion

[greghope667]

Summary of changes

Updates Plover to use Qt6 instead of Qt5 This is work-in-progress, see below for details

Pull Request Checklist

• [x] Changes have tests
• [x] News fragment added in news.d. See documentation for details

Progress

• [x] Convert Qt5 api calls to Qt6
• [x] Remove resource compiler (not supported in PyQt6)
• [x] Test for Windows/Mac/Linux
• [x] Include resources (icons) in built package
• [x] Update Windows/Mac/Linux build scripts
• [x] Figure out how this breaks plugins and fix them (Plugin manger updated, other plugins to follow)

Release plan

• [ ] Public testing & bug fixes
• [ ] Update version numbers
• [ ] Point dependencies back to PyPI
• [ ] Upload to PyPI
• [ ] Build official binary release

[mkrnr]

Thank you so much for working on this! This migration is incredibly important for the maintainability of plover.

I'm happy to test this the next days on Windows, Linux, and macOS.

Let me know if there's anything else I can do to help.

[greghope667]

@mkrnr Any testing you can do would be very helpful. Post any error messages or not-working features here.

If there are any plugins you use that you can test that would be useful, although I expect only non-graphical plugins have a chance of working without updates.

The packaging systems need fixing too (see all the red in the CI right now...) . I'll take a look into that over the next few days/weeks but if anyone knows how all the packaging scripts work then help would be appreciated.

[Robberduckzilla]

Launches successfully on Apple Silicon Macbook Air (M2 chip). Qt6 takes system theme (Dark) too, neat!

[mkrnr]

Here are some tests on Windows 11 64 bit using a Georgi:

tox r -e launch

all built-in functionality works flawlessly:

• Add translation
• Lookup
• Paper Tape
• Suggestions
• Add/create/edit/delete dictionary
• Writing steno
• Anything else I tried

Warnings during start, maybe the one regarding gui_gt can be fixed:

warning: no previously-included files found matching '.gitignore'
adding license file 'LICENSE.txt'
warning: no previously-included files found matching 'plover\gui_qt\*_rc.py'
warning: no previously-included files found matching 'plover\gui_qt\*_ui.py'
warning: no previously-included files found matching 'plover\gui_qt\.gitignore'
no previously-included directories found matching '.github'

tox r -e test

only failures are in:

test\gui_qt\test_dictionaries_widget.py ..................................FFF

This fails on the master as well and is likely fixed by #1599.

tox r -e plugins_install

Executed successfully. Is there a way to test this at this point? Because when I then run tox r -e launch it recreates the env because the env type changes.

tox r -e plugins_install -- C:\Users\mkoer\git\plover_regenpfeifer

Executed successfully. Don't know how to test this, see above.

tox r -e packaging_checks

packaging_checks: commands[0]> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- packaging_checks
packaging_checks: exit 2 (.02 seconds) C:\Users\mkoer\git\greghope667-plover> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- packaging_checks
.pkg: _exit> python c:\users\mkoer\git\greghope667-plover\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta
  packaging_checks: FAIL code 2 (81.95=setup[81.94]+cmd[0.02] seconds)
  evaluation failed :( (82.56 seconds)

tox r -e release_prepare

release_prepare: commands[0]> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- release_prepare
release_prepare: exit 2 (0.01 seconds) C:\Users\mkoer\git\greghope667-plover> bash --noprofile --norc -eo pipefail -c " . ./plover_build_utils/functions.sh; python=python; \"$@\"" -- release_prepare
.pkg: _exit> python c:\users\mkoer\git\greghope667-plover\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta
  release_prepare: FAIL code 2 (75.69=setup[75.67]+cmd[0.01] seconds)
  evaluation failed :( (76.28 seconds)

[greghope667]

Thanks for running the tests, that's great to know this works for other people.

Warnings during start, maybe the one regarding gui_gt can be fixed:

Don't think these warnings happened for Qt5 - I'll look at what's causing these messages

Executed successfully. Is there a way to test this at this point? Because when I then run tox r -e launch it recreates the env because the env type changes.

You might be able to launch plover directly using the environment created by tox. Enter the environment (something like source .tox/dev/bin/activate.bat, might be a slightly different path), then run with python -m plover. I think there's some way to directly run commands through tox in a given environment too but will have to check.

I'm going to look at packaging this, then hopefully we'll be able to share this and get some beta testers for a Qt6 Plover release.

[mkrnr]

I'll give that a try next days, thanks!

We'll also have to look into updating plover_plugins_manager to Qt6 since that's such an integral part of plover. Did you look into that already by any chance?

[greghope667]

I've had a quick look at the plugin manager - there are a few small changes needed there for Qt6 but shouldn't be anything too major. We'll need to update both this project and the plugin manager together though as they're mutually dependent.

[mkrnr]

Awesome to see your progress on this, now with the plugin manager migration!

I'd like to help you testing this but when I launch with tox (tox r -e launch) the plugin manager is still not included. Could you share how you test this? I also tried building the packages on windows but was not successful.

[greghope667]

Most of my recent testing has been through the CI system in GitHub, just fixing errors and getting the builds working. The CI versions have the plugin manager bundled (which was causing build failures using a non-patched Qt5 plugin manager version) - you can download these as build artefacts. Seems to work for the Linux AppImage, but the Windows exe doesn't yet run. I've yet to sort out how to use the plugin manager successfully when launching locally from tox.

Getting local builds to work is not easy - you have to have almost exactly the same setup (including library versions) that the build scripts need else you'll get a bunch of unhelpful error messages from pip. I've also run into issues where some of the download sources for build scripts are rate limited so stop working after a few attempts. Honestly the build system needs work to be simplified, it's a little unmanageable right now.

[mkrnr]

That makes a lot of sense to focus on getting the Github CI build system to work.

The macOS DMG installs and starts perfectly fine (macOS 13.4.1, M1 chip) but the Plugins Manager is not showing up. The CI build looked fine to me but I'll continue looking into that one.

On Win11 64bit I get ValueError: 'plugins_manager/icon.svg' must be only a file name for both the Installer and ZIP version. You probably get the same. I also get this error message when running the AppImage on Ubuntu.

Let me know if I can do more tests. I'll also try to work on solutions. Thanks again for working on this, I would love to buy you a coffee!

[greghope667]

I've sorted out the bundling of assets now, and I think mostly everything works (well, it's likely there's some bugs hiding somewhere). All the CI versions seem to run ok for me as far as I can test. The plugin manager works using https://github.com/openstenoproject/plover_plugins_manager/pull/22 , but no plugins will work as they're all PyQt5 - I'm not aware of any non-graphical plugins.

I'm sure some work is still needed (reviews are welcome) but I'm going to mark this as ready.

[mkrnr]

This is really exciting!

Here are the results of my first tests:

Win11 64bit opened via ZIP

• I'm able to install plover-regenpfeifer via the plugin manager and it works (except a known bug within the plugin itself regarding file paths)
• In Configuration > Machine when I press the Port dropdown menu, I get ERROR: Qt GUI error AttributeError: type object 'Qt' has no attribute 'UserRole'
• Everthing else I tested works

Win11 64bit via installer

• The installation works but when starting plover I get the following stacktrace:

2023-07-13 20:49:49,386 [MainThread] WARNING: Qt: QGuiApplication::font(): no QGuiApplication instance and no application font set.
2023-07-13 20:49:49,389 [MainThread] ERROR: Qt GUI error
Traceback (most recent call last):
  File "build/windist/data\Lib\site-packages\plover\scripts\main.py", line 131, in main
  File "build/windist/data\Lib\site-packages\plover\gui_qt\main.py", line 126, in main
  File "build/windist/data\Lib\site-packages\plover\gui_qt\main.py", line 77, in __init__
  File "build/windist/data\Lib\site-packages\plover\gui_qt\main_window.py", line 96, in __init__
  File "build/windist/data\Lib\site-packages\plover\gui_qt\utils.py", line 59, in Icon
  File "importlib\resources.py", line 161, in path
  File "importlib\resources.py", line 49, in _get_package
  File "importlib\resources.py", line 40, in _resolve
  File "importlib\__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'plover_plugins_manager.gui_qt.resources'

MacOS m1 (Ventura 13.4.1) via DMG

• Everything looks to work well but the Plugins Manager is not shown in the GUI

Ubuntu 64bit via AppImage

• The plugin manager is shown and I'm able to install plugins
• Same error as with Windows above: In Configuration > Machine when I press the Port dropdown menu, I get ERROR: Qt GUI error AttributeError: type object 'Qt' has no attribute 'UserRole'
• Everything else I tested works

It would be awesome to have more people testing this.

[greghope667]

Thanks for the help testing, especially across multiple systems.

ERROR: Qt GUI error AttributeError: type object 'Qt' has no attribute 'UserRole'

This is a PyQt api change, I think it's meant to be Qt.ItemDataRole.UserRole in PyQt6. Possibly from here? I'll fix this.

The Windows installer error looks like a missing file in the bundle. Maybe it's not extracting correctly? This is literally just an icon failing to load however, so maybe we can catch + log instead of aborting.

Not sure why the plugin manager isn't working on MacOS though. Do you have multiple plover installs at all? Plover uses the default (global) python system import system so can get confused quite easily. Will take a look at the DMG contents as well.

[mkrnr]

One more thing I noticed now: when installing via DMG on mac (M1), the dark mode doesn't work (😱). Looking through the step Build distribution (macOS DMG) I saw this which might be related:

Warning:  This wheel needs a higher macOS version than is set in MACOSX_DEPLOYMENT_TARGET variable.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 10_14 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:  

In terms of general functionality it works though.

[Robberduckzilla]

@mkrnr looks like this gets passed in .github/workflows/ci/helpers.sh, line 80 I think we can just bump this number to 10.14? i.e. run_eval "echo MACOSX_DEPLOYMENT_TARGET=10.14 >>\$GITHUB_ENV"

It's just a warning though, so I don't think it's causing the issue of plover not picking up default theme. Does it work as expected on Linux/Windows machines?

[mkrnr]

@Robberduckzilla makes sense. Another thought: maybe it's because the DMG is compiled for intel Macs and Rosetta doesn't care about theming?

Anyhow, I wouldn't consider this a show stopper at all. Definitely something that we can take care of in a follow-up PR.

[Robberduckzilla]

I don't have write access to your repo, so a quick note on a small change you should make: /plover/__init__.py, line 15

- __version__ = '4.0.0.dev12'
+ __version__ = '5.0.0'

This will show version 5 in the about window, as well as (I think) install it into a plover 5.0.0 folder, rather than the misleading 4.0.0dev12