minicap

minicap copied to clipboard

Wouldn't it be nice to have a VNC server on the Android device with some small standalone app to not to have it connected to the computer at all times? (The connection can be optional and VNC port forward via ADB is good too). I have many problems communicating with Minicap socket directly and having a vnc server, that is very fast (like the node demo) and can be implemented into anything easily would be great.

You are making several assumptions that I feel are incorrect.

You say that the implementation is easy. The basic VNC protocol relies on RGB framebuffers. This means that instead of being able to just forward the JPG image to the client as-is, we need to decode it. That already drops the frame rate by around 50%. The framebuffers are extremely large as well, meaning that just transferring them over the network is bound to decrease the frame rate again. Why not just send the difference? Because you’d have to compare the whole image against the previous one, which would be difficult, and make the whole process even slower than just sending the entire frame.

Now, some servers and clients do support JPG images as-is. Did you know that that functionality is not documented anywhere? It’s proprietary. We could perhaps figure it out by doing packet captures between a compatible server and client, but that may or may not have patent issues.

I know these things because I wrote the built-in VNC server for STF. It uses the basic framebuffer method and is very slow.

Does it still sound like the implementation would be easy and the final result very fast?

IIRC though, Appium may have a VNC repo under their org that uses minicap.

@sorccu:

• The VNC implementation you refer to is here: https://github.com/saucelabs/droid-VNC-server
• JPG encoding is part of Tight encoding. It comes from TightVNC but is also supported by noVNC. It's documented here: https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#tight-encoding . TightVNC is licensed under GPL.
• Calculating the differences instead of sending the entire framebuffer can be performant. For example, if you do it on a scanline-by-scanline basis, you can functions like memcmp which can be very fast.

Please try making a PR, I have no particular need for this right now and it would take too much time, which I don't have. If you leave it to me it could be years before there's any progress.

@sorccu I'm in the same boat as you - it sounds interesting to me but I don't have the time to work on this now. Hence, I'm just sharing some of the stuff I found out as it may be of interest to anyone who does have the time to create a PR.

Appreciated 👍

@qmfrederik

@sorccu

I have tried the same its working fine, but when I change the screen orientation, vncviewer loses some part of screen. I believe minicap send the rotated image data once detects the screen roatation/full screen.

After I have integrated the device handling features. If device is in portrait mode, its working fine even I am able to handle device properly but When I changed the device orientation, in vncviewer loses some part of screen.

Can you help me how to handle this orientation stuff.

STFService exposes a service which allows you to monitor for device rotation events.

You can either change the VNC server to watch for that event & update the framebuffer size, or do it on the client side.