Add remote desktop control using the Java Robot class and Chrome Native messaging

[deleolajide]

Replicate a better version of ofmeet remote control (ver 0.3.x) . This time use the Java Robot class to control a remote mouse and keyboard instead of nodejs and take advantage of the together.js code already in Pade that does co-browsing and multiple user cursors

[deleolajide]

Decided to re-use the Jitsi-meet remote control UI and intercept the remote control commands being sent down the webrtc data channel. A Java application which uses the Robot Class has to be installed on each desktop that needs to be controlled by a participant from the meeting.

It is not required for the remote participant that intends to control. Chrome native messaging is used to pass signal messages between Pàdé and the Java application while the WebRTC data channel is used to pass the mouse/keyboard events between the remote controller and the controlled desktop

First, install the application. There is an installer for Windows here. For OSX and Linux, this has to be done manually following the Chrome native messaging guidelines.

Next, enable remote control in Pade

That will cause the remote control feature to appear for that user in a meeting

Click on the icon to request permission to remote control. That will cause a prompt to appear in the UI of the user whose desktop can be controlled.

That user can then allow or cancel.

Allow will trigger the webrtc screenshare for only screens. No support for applications yet. Choose a screen if you have multiple monitors and that should starts the process.

[deleolajide]

Now fully working in version 0.4.6. Needs documentation on how to setup before I can close Issue

[marclaporte]

My team can handle user documentation. Just need a working example, and we'll make nice screenshots and some explanations.

We document here? https://igniterealtime.github.io/Pade/

Thanks!

[deleolajide]

Yes, the actual location in GitHub is

https://github.com/igniterealtime/Pade/tree/master/docs

Just submit PRs to add more documentation. It is vanilla HTML

-dele

Sent from Mail for Windows 10

From: Marc Laporte Sent: 21 June 2018 18:58 To: igniterealtime/Pade Cc: Dele Olajide; Author Subject: Re: [igniterealtime/Pade] Add remote desktop control using the JavaRobot class and Chrome Native messaging (#24)

My team can handle user documentation. Just need a working example, and we'll make nice screenshots and some explanations. We document here? https://igniterealtime.github.io/Pade/ Thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[deleolajide]

There is also an oustanding task of packaging the Pade Java native messaging client that actually controls the mouse/keyboard via the native messages from the Pade chrome extension. The packaging has to install correctly for OSX, Linux and Windows.

I will probably start with Windows first and may require assistance with OSX and Linux afterwards

[deleolajide]

Pàdé setup/installer/packager ver 0.1 for Windows now done and uploaded.

See https://github.com/igniterealtime/Pade/releases/tag/v0.1

[deleolajide]

Please note that only screen 1 can be remote controlled for now. I am seeking a solution that will support multi-screen desktops with the Java Robot Class

[premanandmanimaran]

Hello

I am trying create Skype for Business Alternative for my business. I looking to built Unified Communication Platform where my users login and have following features

1. Audio / Video Calling ( Option to auto accept)
2. Request Desktop Access ( Option to auto accept)
3. Co-Browsing
4. SIP Calling
5. Conference
6. Text Chatting
7. Call Recording

Can you let me know if anybody can help us do this.

Prem

[marclaporte]

2 Windows computers with

• Both Pàdé 0.6.0
• Both installed Pàdé Remote Control (Windows) v 0.1
• Different XMPP accounts under same XMPP domain, with Openfire 4.2.3 I launched an audio/video call. Seems all normal. But I don't see the "icon to request permission to remote control" which "will cause a prompt to appear in the UI of the user whose desktop can be controlled"

[marclaporte]

@premanandmanimaran Those are all things that are done at various levels of quality. Join the community to make them even better!

Please see: http://wikisuite.org/Why-Openfire

[deleolajide]

First configure on PC that has installed the remote control native application

To debug, open the dev tools window for the chrome background page. Check log to see that extension is talking to binary app

Start Jitsi Meet and check for remote control

[marclaporte]

I confirm we had both activated the "Enable Remote Control" checkbox.

I will ask 2 devs to try again and share their findings.

Thanks!

[deleolajide]

Use only Pade@Work. Pade@Home is not yet on the white-list of the remote control desktop application.

[marclaporte]

ahhh, that must be it!

and 0.2.0 solves it https://github.com/igniterealtime/Pade/releases/tag/v0.2

[Skaylat]

Hello, the remote desktop option isn't showing between 2 participants

Openfire v4.5.2 Audio and video call working

Both Remote desktop : v0.2 and activate the "Enable remote control" 1- Opera Pade@Work v1.6.8 : _generated_background_page : remote control incoming {message: "hello too"}

2- Chrome Pade@Home v1.5.4 : _generated_background_page : "Access to the specified native messaging host is forbidden." The extension id is the same as in the pade.remote.control.json file however

[deleolajide]

This is currently broken and not working

[Skaylat]

Hello, thanks for your fast answer Do you have an ETA for this issue ?

[deleolajide]

Hello, thanks for your fast answer Do you have an ETA for this issue ?

It was broken by this issue Jitsi-Meet is no more included in the chrome extension. This means that remote control has to be implemented directly in ofmeet.

No, I dont have an ETA

[chuleta7]

Hi! Sorry but I don't understand, is it supposed that pade remote desktop control should still be working through openfire plugin ?

[deleolajide]

pade remote desktop control should still be working through openfire plugin ?

Pade no more supports remote desktop control and there are no plans to add it back

[chuleta7]

Upppps, don't tell me that :( I had pinned my hopes on that solution because it doesn't need or depends on a third party account service like teamviewer or anydesk wich you don't control and depends precisely on internet but instead a closed solution like Openfire wich works very well in our intranet only

Thanks anyway! is it difficult to develop and bring back the remote desktop service ? I mean, if we can cooperate :)

[deleolajide]

You don't need to use Teamviewer anymore. There is now an open source alternative.

https://rustdesk.com/