dsnote
dsnote copied to clipboard
mkiol
[Feature Enhacement] Use clipboard service on Wayland instead of the current implementation
Hello again!
Introduction
I want to explore using the dbus service org.kde.klipper on Plasma, and com.github.hluk.copyq anywhere else to implement the clipboard interaction instead of the current QClipboard implementation. And fall back to X11's clipboard through xclip's copy implementation.
Why
On Wayland, simulating copy-paste action doesn't work because we aren't focusing the dsnote window when we want to paste text into it. If we were, you wouldn't need to use the simulated pasting in the first place. However, I discovered that it is possible to set and get text from the clipboard by using the clipboard DBus service on your respective desktop environment.
Why not just default to X11
because it's going away in the future, but we can also use the XClipboard as a fallback on XWayland.
Why these dbus services?
org.kde.klipper is the Plasma clipboard service, so it's the default on Plasma.
com.github.hluk.copyq is CopyQ service bus; I believe it's usable.
@mkiol what do you think. Can I implement this? Simulate Copy paste is unusable in wayland environments at the moment.
Great idea! I didn't know that org.kde.klipper service existed. Yes, please try out the possibilities and implement them.
As I understand, this should work like this: On Wayland org.kde.klipper (or com.github.hluk.copyq) API is used if it is available, otherwise QClipboard. Which API is used should be transparent to the user.
Side note: Most likely, new Dbus service permissions has to be added to the Flatpak manifest so that Speech Note can access the new APIs.
Hey, I also think it should be transparent to the user!
I just found this documentation:
So, CopyQ would work on Wayland, as I understand it.
It looks like if you are not on KDE or don't have Klipper installed, you won't be able to use copy-paste on Wayland.... I'm still trying to find another solution.
For now, if we don't find any solution, I think we should warn the user about the functionality not working when Klipper isn't available.
Found a better solution. We can use QClipboard on X11 systems, and wl-clipboard everywhere else. I have tested this code:
sleep 3; wl-copy "Hi WiFi"
3 seconds for me to put the terminal in the background, and it still copied to my clipboard. That may be an exit.
It looks like if you are not on KDE or don't have Klipper installed, you won't be able to use copy-paste on Wayland.... I'm still trying to find another solution.
I am on Arch and didn't install Klipper service explicitly, but DBus service org.kde.klipper is running. It must comes with typical Plasma dependencies. I think you could start with support for Klipper and extend it for other APIs in the future.
Found a better solution. We can use QClipboard on X11 systems, and wl-clipboard everywhere else.
I didn't look deeply into the wl-clipboard code, but it seems to use the Wayland protocol for clipboard handling, so (in theory) it should work similarly to the QClipboard Qt wrapper. The same limitation regarding no access to the clipboard when the application is running in the background should also apply to wl-copy/wl-paste. Perhaps it does something more... Unfortunately, wl-clipboard is not available as a standalone library that you can simply linked and use. Personally, I don't like the idea of calling external executables in Speech Note.
So, copying in Wayland proved to be a way harder process than I thought it would be. 😄
Would spawning the wl-copy binary directly be problematic from your POV? My current plan is to copy their implementation, but it's quite extensive and complicated, whereas spawning the command would be:
wl-copy "Text to copy"
Clipboard_Contents=$(wl-paste)
AND QProcess seems to be an effortless library to use.
Would spawning the wl-copy binary directly be problematic from your POV?
It is acceptable, but not super elegant from an purely ascetic POV.
My current plan is to copy their implementation
Please do try. I have some doubts as to whether this method will work, but I want to be proven wrong :)
Moreover, I'm not sure if Flatpak sandboxed app can execute external binaries. Maybe yes, but I don't know. Just signaling potential problems.
Turns out:
- Flatpak can't execute programs in the host system (very secure indeed)
- Implementing a copy command in the Wayland protocol is unnecessarily difficult.
- I swear, I've read 300 lines of code, 5 different documentations, and the protocol definition itself, and I still couldn't make it work yet....... Very frustrating.
I'll take a closer look tomorow or the day after, it's already 22:22 here, and I have to sleep.
Note for future self:
wl-clipboard defines a static library. Explore using it instead of reimplementing the code itself.
I see in the CMakelists.txt:
unset(meson_bin CACHE)
find_program(meson_bin meson)
Does this mean we can use Meson build dependencies? How can I include a meson dependency in this project, @mkiol? Or please point me in the direction of how to do it.
What I need is:
git clone https://github.com/bugaevc/wl-clipboard
meson compile wl-clipboard
Then add a static link to the wl-clipboard that was generated from it. Ideally, also add it to the cache.
I swear, I've read 300 lines of code, 5 different documentations, and the protocol definition itself, and I still couldn't make it work yet...
I feel the same way. In fake_keyboard, I implemented the fetching of keymap data from Wayland composer. I thought it would be a simple task, but in the end, it was so complicated. Accessing the Wayland API is a nightmare.
wl-clipboard defines a static library. Explore using it instead of reimplementing the code itself.
Interesting, so perhaps it is possible to use it as a library.
All static (and shared) dependencies are included into the project with CMake's ExternalProject_Add. Yes, you can build meson libraries. Take a look on xkbcommon or rubberband.
Hey @mkiol, please tell me there is a way to accelerate the build process on Flatpak. It takes 16 minutes on my local machine every single time.
So, this is my wlclipboard.cmake:
set(wlclipboard_source_url "https://github.com/bugaevc/wl-clipboard/archive/refs/tags/v2.2.1.tar.gz")
set(wlclipboard_checksum "6eb8081207fb5581d1d82c4bcd9587205a31a3d47bea3ebeb7f41aa1143783eb")
if(${meson_bin} MATCHES "-NOTFOUND$")
message(FATAL_ERROR "meson not found but it is required to build wl-clipboard")
endif()
ExternalProject_Add(wlclipboard
SOURCE_DIR ${external_dir}/wlclipboard
BINARY_DIR ${PROJECT_BINARY_DIR}/external/wlclipboard
INSTALL_DIR ${PROJECT_BINARY_DIR}/external
URL ${wlclipboard_source_url}
URL_HASH SHA256=${wlclipboard_checksum}
# Meson configuration
CONFIGURE_COMMAND ${meson_bin} setup --prefix=<INSTALL_DIR> --buildtype=release --libdir=lib --datadir=<INSTALL_DIR>/data
-Dfishcompletiondir=<INSTALL_DIR>/completions
<BINARY_DIR> <SOURCE_DIR>
# Build the static library
BUILD_COMMAND ninja -C <BINARY_DIR>
BUILD_ALWAYS False
# Install the library and headers (manually copy the static library since it's not installed by default)
INSTALL_COMMAND echo "Installed to <INSTALL_DIR> from <BINARY_DIR>"
)
# Ensure the external project target exists as a build dependency
list(APPEND deps wlclipboard)
# Paths:
# - built library produced by meson in the external project's build dir (observed at <BINARY_DIR>/src/libwl-clipboard.a)
# - installed library path expected by other parts of the CMake build (external/lib/libwl-clipboard.a)
set(wlclipboard_built_lib "${PROJECT_BINARY_DIR}/external/wlclipboard/src/libwl-clipboard.a")
set(wlclipboard_installed_lib "${external_lib_dir}/libwl-clipboard.a")
# Create a custom command that copies the built library into the external lib dir.
# This generates a concrete file `${external_lib_dir}/libwl-clipboard.a` so Ninja has a rule
# to produce the file that other targets (which may reference the literal path) can use.
add_custom_command(
OUTPUT ${wlclipboard_installed_lib}
COMMAND ${CMAKE_COMMAND} -E make_directory ${external_lib_dir}
COMMAND ${CMAKE_COMMAND} -E copy ${wlclipboard_built_lib} ${wlclipboard_installed_lib}
DEPENDS ${wlclipboard_built_lib}
COMMENT "Copying wl-clipboard static library to ${wlclipboard_installed_lib}"
)
# A custom target that depends on that installed file and on the ExternalProject build step.
add_custom_target(wlclipboard_installed ALL DEPENDS ${wlclipboard_installed_lib})
add_dependencies(wlclipboard_installed wlclipboard)
# Provide an IMPORTED target that points to the installed library (so other targets can link to it)
add_library(wlclipboard_lib STATIC IMPORTED GLOBAL)
set_target_properties(wlclipboard_lib PROPERTIES
IMPORTED_LOCATION "${wlclipboard_installed_lib}"
INTERFACE_INCLUDE_DIRECTORIES "${external_include_dir}/wlclipboard/src"
)
add_dependencies(wlclipboard_lib wlclipboard_installed)
# Use the imported target for linking
list(APPEND deps_libs wlclipboard_lib)
# Keep the raw include directory available as well for older parts of the build that use 'includes' variable
list(APPEND includes "${external_dir}/wlclipboard/src")
For testing manually before implementing the cmake build steps I cloned the directory into /tmp/wl-clipboard
Then, I ran:
$ cd /tmp/wl-clipboard
$ meson setup --prefix=$PWD/prefix-dir --buildtype=release --libdir=$PWD/lib-dir --datadir=$PWD/data-dir -Dfishcompletiondir=$PWD/completions bin-dir .
$ cd /tmp/wl-clipboard/bin-dir
$ ninja -C .
$ ls /tmp/wl-clipboard/bin-dir
build.ninja compile_commands.json completions data meson-info meson-logs meson-private src subprojects
$ ls /tmp/wl-clipboard/bin-dir/src
config.h libwl-clipboard.a libwl-clipboard.a.p protocol wl-copy wl-copy.p wl-paste wl-paste.p
I know the library is being built, but when I try to build it with the Flatpak toolchain, this error happened:
..... Normal Logs ........
CMake Warning (dev) at /usr/share/cmake-3.31/Modules/ExternalProject/shared_internal_commands.cmake:1276 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.31/Modules/ExternalProject.cmake:3041 (_ep_add_download_command)
cmake/wlclipboard.cmake:8 (ExternalProject_Add)
CMakeLists.txt:557 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
..... more normal logs .........
CMake Error in CMakeLists.txt:
Imported target "wlclipboard_lib" includes non-existent path
"/run/build/dsnote/external/include/wlclipboard/src"
in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include:
* The path was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and references files it does not
provide.
CMake Error in CMakeLists.txt:
Imported target "wlclipboard_lib" includes non-existent path
"/run/build/dsnote/external/include/wlclipboard/src"
in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include:
* The path was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and references files it does not
provide.
-- Generating done (0.1s)
CMake Warning:
Manually-specified variables were not used by the project:
BUILD_UROMAN
CMake Generate step failed. Build files cannot be regenerated correctly.
Error: module dsnote: Child process exited with code 1
I can't find /run/build/dsnote/external/include/wlclipboard/src anywhere.....
I can't solve it. Any input is welcome!
Best,
I see. The build script is only for executable, so you can't use INSTALL_COMMAND ninja -C <BINARY_DIR> install because it won't install a library.
You might do just this:
ExternalProject_Add(wlclipboard
SOURCE_DIR ${external_dir}/wlclipboard
BINARY_DIR ${PROJECT_BINARY_DIR}/external/wlclipboard
INSTALL_DIR ${PROJECT_BINARY_DIR}/external
URL ${wlclipboard_source_url}
URL_HASH SHA256=${wlclipboard_checksum}
CONFIGURE_COMMAND ${meson_bin} setup --prefix=<INSTALL_DIR>
--buildtype=release --libdir=lib --datadir=<INSTALL_DIR>/data
-Dfishcompletiondir=no
-Dzshcompletiondir=no
<BINARY_DIR> <SOURCE_DIR>
BUILD_COMMAND ninja -C <BINARY_DIR>
BUILD_ALWAYS False
INSTALL_COMMAND cp ${external_dir}/wlclipboard/src/libwl-clipboard.a ${external_lib_dir}/
)
list(APPEND deps_lib "${external_lib_dir}/libwl-clipboard.a")
list(APPEND deps wlclipboard)
But you will observe linker errors. Even if this somehow works, you must satisfy all libwl-clipboard.a dependencies and also copy the files from the wayland-protocols directory because most likely they are needed in a runtime...
.
├── bin
│ ├── wl-copy
│ └── wl-paste
└── share
├── bash-completion
│ └── completions
│ ├── wl-copy
│ └── wl-paste
├── man
│ └── man1
│ ├── wl-clipboard.1
│ ├── wl-copy.1
│ └── wl-paste.1
├── pkgconfig
│ └── wayland-protocols.pc
├── wayland-protocols
│ ├── stable
│ │ ├── presentation-time
│ │ │ └── presentation-time.xml
│ │ ├── viewporter
│ │ │ └── viewporter.xml
│ │ └── xdg-shell
│ │ └── xdg-shell.xml
│ ├── staging
│ │ ├── drm-lease
│ │ │ └── drm-lease-v1.xml
│ │ └── xdg-activation
│ │ └── xdg-activation-v1.xml
│ └── unstable
│ ├── fullscreen-shell
│ │ └── fullscreen-shell-unstable-v1.xml
│ ├── idle-inhibit
│ │ └── idle-inhibit-unstable-v1.xml
│ ├── input-method
│ │ └── input-method-unstable-v1.xml
│ ├── input-timestamps
│ │ └── input-timestamps-unstable-v1.xml
│ ├── keyboard-shortcuts-inhibit
│ │ └── keyboard-shortcuts-inhibit-unstable-v1.xml
│ ├── linux-dmabuf
│ │ └── linux-dmabuf-unstable-v1.xml
│ ├── linux-explicit-synchronization
│ │ └── linux-explicit-synchronization-unstable-v1.xml
│ ├── pointer-constraints
│ │ └── pointer-constraints-unstable-v1.xml
│ ├── pointer-gestures
│ │ └── pointer-gestures-unstable-v1.xml
│ ├── primary-selection
│ │ └── primary-selection-unstable-v1.xml
│ ├── relative-pointer
│ │ └── relative-pointer-unstable-v1.xml
│ ├── tablet
│ │ ├── tablet-unstable-v1.xml
│ │ └── tablet-unstable-v2.xml
│ ├── text-input
│ │ ├── text-input-unstable-v1.xml
│ │ └── text-input-unstable-v3.xml
│ ├── xdg-decoration
│ │ └── xdg-decoration-unstable-v1.xml
│ ├── xdg-foreign
│ │ ├── xdg-foreign-unstable-v1.xml
│ │ └── xdg-foreign-unstable-v2.xml
│ ├── xdg-output
│ │ └── xdg-output-unstable-v1.xml
│ ├── xdg-shell
│ │ ├── xdg-shell-unstable-v5.xml
│ │ └── xdg-shell-unstable-v6.xml
│ └── xwayland-keyboard-grab
│ └── xwayland-keyboard-grab-unstable-v1.xml
└── zsh
└── site-functions
├── _wl-copy
└── _wl-paste
I don't want to discourage you, but maybe org.kde.klipper would be a better option. Just saying.
I will implement the Klipper implementation as the default for now (falling back to QClipboard).
And try to implement it via Wayland protocols or invoking the wl-clipboard copy directly.
Hey @mkiol, please tell me there is a way to accelerate the build process on Flatpak. It takes 16 minutes on my local machine every single time.
Is there a specific reason why you need to build Flatpak? Why not build it directly on your system? I suspect you don't want to download all the dependencies. That might not be such a bad thing, as CMake can download almost all dependencies for you. You could simply disable most of the "heavy" dependencies. Here is an example of a stripped-down configuration:
git clone [email protected]:mkiol/dsnote.git
cd dsnote
mkdir build
cd build
cmake ../ -DWITH_DESKTOP=ON \
-DWITH_PY=OFF \
-DDOWNLOAD_VOSK=ON \
-DBUILD_VOSK=OFF \
-DBUILD_WHISPERCPP=OFF \
-DBUILD_OPENBLAS=OFF \
-DBUILD_RHVOICE=OFF \
-DBUILD_RHVOICE_MODULE=OFF \
-DBUILD_BERGAMOT=OFF
make -j16
LD_LIBRARY_PATH=./external/lib ./dsnote --verbose
what if we bundle wl-copy and wl-paste into the flatpak binary?
Yes, this option is on a table as well, but we need to be sure that wl-copy/wl-paste, executed from the flatpak sandbox, does what we expect it to do. In my opinion, that's not so obvious.
Is there a specific reason why you need to build Flatpak? Why not build it directly on your system? I suspect you don't want to download all the dependencies.
I was following the documentation 👍
Yes, this option is on a table as well, but we need to be sure that wl-copy/wl-paste, executed from the flatpak sandbox, does what we expect it to do. In my opinion, that's not so obvious.
Understood, I will test and report back
git clone [email protected]:mkiol/dsnote.git
cd dsnote
mkdir build
cd build
cmake ../ -DWITH_DESKTOP=ON \
-DWITH_PY=OFF \
-DDOWNLOAD_VOSK=ON \
-DBUILD_VOSK=OFF \
-DBUILD_WHISPERCPP=OFF \
-DBUILD_OPENBLAS=OFF \
-DBUILD_RHVOICE=OFF \
-DBUILD_RHVOICE_MODULE=OFF \
-DBUILD_BERGAMOT=OFF
make -j16
LD_LIBRARY_PATH=./external/lib ./dsnote --verbose
Thank you so much for this!
@mkiol today I woke up to this error after I wrote the XML interface to the DBus directory:
[ 68%] generate dbus app adaptor sources
Got unknown type `a{sv}'
You should add <annotation name="org.qtproject.QtDBus.QtTypeName.In0" value="<type>"/> to the XML description
make[2]: *** [CMakeFiles/dsnote_lib_autogen_timestamp_deps.dir/build.make:76: dbus_dsnote_adaptor.h] Error 1
make[1]: *** [CMakeFiles/Makefile2:1371: CMakeFiles/dsnote_lib_autogen_timestamp_deps.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
I have tried removing the file, deleting the build folder, restarting the system, and I am close to cloning the repo again. Any idea what is happening here?
What changes did you make in XML file?
When you use not trivial type like a{sv} in DBus XML description, you need to specify to what Qt-type it should be mapped. You do it by providing <<annotation>. Here is an example from dsnote.xml:
<signal name="ActiveTtsModelPropertyChanged">
<annotation name="org.qtproject.QtDBus.QtTypeName.Out0" value="QVariantMap"/>
<arg name="activeTtsModel" type="a{sv}" direction="out" />
</signal>
It says, that first out argument (Out0) should be mapped to QVariantMap type.
What changes did you make in XML file?
None, but I did install qdbusxml2cpp and ran:
But I rm -rf dsnote and cloned it again. Now it doesn't build even in the new repo
Hmm, Flatpak doesn't work: When pasting with the desired method, this error is logged to the console:
/usr/include/c++/14.3.0/optional:475: constexpr _Tp& std::_Optional_base_impl<_Tp, _Dp>::_M_get() [with _Tp = fake_keyboard; _Dp = std::_Optional_base<fake_keyboard, false, false>]: Assertion 'this->_M_is_engaged()' failed.
Here are my Flatpak permissions: