Emscripten: Handling "external" clipboard

[ypujante]

Version/Branch of Dear ImGui:

Version 1.91.0

Back-ends:

imgui_impl_glfw.cpp

Compiler, OS:

emscripten / browser

Full config/build information:

No response

Details:

With the release of ImGui 1.91.0 and the support for emscripten-glfw, as you know, the clipboard is now supported in the browser environment. For example, you can now copy the build configuration (which can try for yourself with the demo I pushed with the PR)

If you copy to the clipboard by calling glfwSetClipboardString, then I store the string locally and push it to the external clipboard (which is why you can paste it outside the browser). If you then paste from the clipboard by calling glfwGetClipboardString, then the clipboard string that was stored previously is returned.

In other words, copy/paste from within the application works great.

The issue comes with pasting from the "external" clipboard: the Javascript API, which is the API that I need to use is asynchronous. For this purpose, in the latest version of emscripten-glfw, I added a new API which reflects this API:

typedef void (* emscripten_glfw_clipboard_string_fun)(void *userData, char const *clipboardString, char const *error);
void emscripten_glfw_get_clipboard_string(emscripten_glfw_clipboard_string_fun callback, void *userData);

I am using this API in my recently released WebGPU Shader Toy tool for the code editor class (TextEditor) because I am in control of this logic: when I detect a "Paste" event (CTRL+V), I call the asynchronous API and when I receive the result, I do the actual Paste action.

Of course this does not work with ImGui::InputText and I wanted to ask you if you could think of a way, or giving me pointers of whether this would be possible to do in ImGui (I tried to look at Shortcut routing but I don't know if that is even the right place to start...). I would be more than happy to do a PR for the work if this is something that is possible at all. Otherwise I could implement an entirely new control for my project.

Thank you

Screenshots/Video:

No response

Minimal, Complete and Verifiable Example code:

No response

[ocornut]

when I detect a "Paste" event (CTRL+V),

Who is detecting the event? Is that a paste event emitted from the browser? or it is your app querying key for ctrl+v press and then doing an asynchronous query for clipboard data?

Could you e.g. preemptively do this query every time an InputText is activated (via polling io.WantTextInput) and everytime the application is refocused while one is active? and then you carry that data locally in the emscripten data and it would be ready to provide whenever needed?

[ypujante]

Who is detecting the event? Is that a paste event emitted from the browser? or it is your app querying key for ctrl+v press and then doing an asynchronous query for clipboard data?

The detection is from the ImGui side, something like this

if (ImGui::GetIO().KeyCtrl && ImGui::IsKeyPressed(ImGuiKey_V))

Could you e.g. preemptively do this query every time an InputText is activated (via polling io.WantTextInput) and everytime the application is refocused while one is active? and then you carry that data locally in the emscripten data and it would be ready to provide whenever needed?

I didn't think about doing it preemptively. But I know that Firefox does a little popup when you try do do a paste from Javascript to allow/disallow pasting:

If you want to try it yourself, you can use Firefox and go there and click on Paste after copying something in the clipboard outside the browser...

So if I were to do it preemptively then I would end up with these popups even if you are not trying to copy the clipboard.

[ocornut]

In that case we may need to design a new API replacement for GetClipboardText() that would allow widgets for polling for readyness of a async request and make InputTextEx() uses that.

[ypujante]

Clearly somebody already noticed the issue raised by this ticket :)

In that case we may need to design a new API replacement for GetClipboardText() that would allow widgets for polling for readyness of a async request and make InputTextEx() uses that.

I don't mind looking into your suggestion. Any code I could look at that does something somewhat similar?

Thanks

[ocornut]

Nope I cannot think of an example. I believe the caller would probably need to get a unique id (eg a simple sequential integer) and use that when creating and polling the request. Request would naturally elapse if it stops being polled.

[ypujante]

I will spend some cycles on this and report back. Thank you!

[ypujante]

Just wanted to let you know that I am looking into this. I was able to build a prototype with the async API fairly easily (far easier than I thought). I am bumping into some issues with the "Paste" popup displayed by Safari and Firefox as a security measure because they kill event propagation, so the keys (ex: CTRL + V) remained stuck down forever because I never get the "up" event :(

You can try for yourself with this 100% pure HTML/javascript test case.

paste.html.zip

I am looking into this as well as other ways of doing this entirely...

[Maksons]

Sorry, Did you tried it similar way to https://github.com/Armchair-Software/emscripten-browser-clipboard ? This one worked for me, had to find correct place to register callbacks, it is after "ImGui::CreateContext();" in my case.

[ypujante]

@Maksons I did not know about this project but in the end I am looking at similar ways of going around the problem. Note that I tried his/her demo and it does not really work on macOS due to the "paste" event being generated by "Meta + V" (and so it appears in the log window, but it does not get pasted in the Clipboard demo window).

It is definitely NOT an easy problem to make it work cross browser and cross platform... That is what I am trying to achieve as best I can.

Note that the async API for paste I have right now works quite well with Chrome and Edge (this async paste API is not offered in the project you linked to). And I have found ways to make it work decently with Safari and Firefox.

[ypujante]

Hi @ocornut. I spent the last couple of weeks at this problem and I think I have a result that I am quite pleased with.

First of all, the asynchronous API that I implemented in the library only works decently in Chrome and Edge (although you still get a warning the first time you use it). In Firefox and Safari, you get this "Paste" popup window which swallows all keyboard events. Although I added code to detect this situation, the fact that any other code using this API now needs to handle this asynchronicity API is burdensome and hard to implement. In fact, I have even deprecated this API to simplify the code of the library as well...

I went a completely different route and instead "embrace" the browser/OS API: what this means is now I am relying on the "paste" (as well as "copy" and "cut") events raised by the browser. And it just works in pretty much any browser I have tried it in, with no (for Windows)/minimal (for macOS) changes to ImGui.

Of course, there is a "catch": in order to receive these events you must be using the proper keyboard shortcuts for the platform it is running on. As a result this means that

• on PC/Linux, it just works with 0 changes whatsoever
• on macOS, you must use io.ConfigMacOSXBehaviors=true

Of course my library now automatically detects the runtime platform and offer an API to access it: emscripten_glfw_is_apple_platform(). As you know the Meta/Super key has an issue (in the browser) and I have written a workaround for it. But this is just a workaround and the repeat is just too fast in ImGui resulting in way too easy to do multiple paste. I offer an API to tweak the workaround. This only applies for the macOS platform! (Side note: if ImGui was relying on the repeat events from GLFW this would not be an issue whatsoever and would also respect the settings for repeat/delay set at the OS level...)

Note that if on macOS, you decide to still keep the PC/Linux keyboard shortcuts, it means that paste from the OS clipboard won't work, but everything else works including copy TO clipboard (I have tested on Safari with this scenario).

I personally think this is the right way to go: on macOS, users are used to the macOS shortcuts (and I believe you mentioned it in another thread as something you wanted to do). On PC/Linux, there is no change so it just works transparently...

So I am just curious what you think of the solution.

I have a build done with a not-yet released version of emscripten-glfw so you can try it for yourself.

In this demo, the code added is this in ImGui_ImplGlfw_InstallEmscriptenCallbacks:

    if(emscripten_glfw_is_apple_platform())
    {
        ImGuiIO& io = ImGui::GetIO();
        io.ConfigMacOSXBehaviors = true;
        printf("Detected macOS platform: GetSuperPlusKeyTimeout [%d, %d]\n",
               emscripten::glfw3::GetSuperPlusKeyTimeout(),
               emscripten::glfw3::GetSuperPlusKeyRepeatTimeout());
        io.KeyRepeatDelay = static_cast<float>(emscripten::glfw3::GetSuperPlusKeyTimeout()) / 1000.f;
        io.KeyRepeatRate = static_cast<float>(emscripten::glfw3::GetSuperPlusKeyRepeatTimeout()) / 1000.f;
    }

The io.KeyRepeatDelay lines are just temporary to play around with what works best with the workaround. It might be better to simply disable repeat entirely (for Meta + Key) by setting the workaround timeout to something very short instead while not touching io.KeyRepeatDelay/io.KeyRepeatRate (emscripten::glfw3::SetSuperPlusKeyTimeouts(10, 10))

I also implemented another version of openURL which works without blocking popups which you can try by clicking any link in the demo...

void ImGui_ImplGlfw_EmscriptenOpenURL(char const* url)
{
    if(url)
        emscripten::glfw3::OpenURL(url);
}

[ypujante]

Following up on this, I have now released emscripten-glfw v3.4.0.20240817 with the changes mentioned. Once released with emscripten (unclear about timing), I will create a PR for changes in ImGui (very small PR).

As an FYI I have updated my real live application WebGPU Shader Toy to use this latest version. You can try it out and you will see that the main editor supports copy/paste with the desktop as well as all text fields (like the one in the "Rename" popup) and this one is an ImGui::InputText widget. If you use a Mac, then the shortcuts are now the macOS shortcuts and copy/paste work as well.

The only changes in my code is this:

  ImGuiIO& io = ImGui::GetIO();
  io.ConfigMacOSXBehaviors = emscripten::glfw3::IsRuntimePlatformApple();

  // essentially disable repeat on Meta + Key
  emscripten::glfw3::SetSuperPlusKeyTimeouts(10, 10);

[ocornut]

Hello, Is this a closable issue?

[ypujante]

I believe so. When using emscripten 3.1.65+ (current version is 3.1.66) it will automatically work for pc/linux and it will work for macos as soon as the changes you committed recently are released.

Of course this assumes using emscripten-glfw with ImGui instead of the built-in implementation