UnityRenderStreaming
UnityRenderStreaming copied to clipboard
Unity-Technologies
[BUG] Incorrect mouse position in full-screen web player sample
Hello again, hope you all are doing well. :)
There is an issue with the web video player sample upon entering full screen. When you maximize the stream, the mouse position no longer syncs correctly to the renderstreaming IInput.
This was not an issue in 2.x.
Steps to reproduce the behavior:
- Run the webserver and the unity webserver example scene,
- Connect to the stream via videoplayer/index.html and go into full screen mode,
- Mouse Input position is no longer correct.
I've a similar issue but even without entering fullscreen. I used the WebBrowserInput scene + Videoplayer Sample website.
Tested in Chrome, Firefox and Edge all produce the same offset in the non-fullscreen mode. In fullscreen the offset is exacerbated.
@krisrok Hi, we fixed the mouse position when controlling it on browser. https://github.com/Unity-Technologies/UnityRenderStreaming/pull/586
Hello, when I was running unity-demo, the same problem as you (fix: Fix position of mouse pointer on browser #586) appeared. Is there a solution?
@zhanghaifei1997
Hi, this issue has not been solved yet.
I noticed that we should set two resolutions same.
@karasusan Because I saw that you have provided a recorded video (mousepointer.mp4) at the top. The mouse click is normal in the video. How is that done?
Considering this issue is not fixed, we were thinking of a applying synchronization ourselves. We thought of sending aspect ratio of both devices to each other as a text message and synchronize the cursor position accordingly.
However, RenderStreamingInternal class is the only place we found that uses PeerConnection and RTCDataChannel but it is pretty much abstracted. So my question is is there a way we can use Unity.WebRtc and Unity.RenderStreaming in pair. Can we use Unity.WebRtc features such as sending aspect ratio of the device as a text message along with Unity.RenderStreaming streams and connections?
@alikanat001
Currently, we do not provide API for that issue.
We think it's better to expose com.unity.webrtc class on RenderstreamingInternal, and only plan it.
memo : URS-447
Could you make a new issue with your feature request?
Hi. Is there any news about this issue?
I used before old version 1.2 but I decided to upgrade the module and also found this problem.
The first connected client has correctly synchronized touches. But if I set fullscreen or change the screen orientation, the issue with incorrect touch position appears. It's not just a small offset but a large difference. Second connected client (from another device) has always the problem. I have to restart the application and connect only second client if i want synchronized touches for him...
I also tested template project from docs with broadcast example but the issue is still here. Is there any solution or I should use any earlier version?
@Rhanjie Thank you for sharing the detail. This issus is prioritized for us and we will fix it next version.
@karasusan We also have the same problem as @Rhanjie. When it be possible to fix it? This blocks the launch of our project.
@ZeoWorks @yslab1 @Rhanjie @ByeTragedy @gotanidea @zhanghaifei1997 @krisrok Fixed the issue in #800. We'll release the new version next month.
I cloned the repo and tested the fix in your branch but unfortunately it didn't help. I recorded a short movie to better demonstrate the problem. Maybe it will be useful for you: https://youtu.be/4D61q_Dd4GU
In short. Every screen modification do not update the touch area correctly. Additionally, the UI Controller component that displays the red dot doesn't work good with touch gestures as you can see in the movie. Y position is always 0 for some reason and X has weird offset but the UI elements respond normally. It's just a problem with UI Controller.
I tested new version with previous webserver because the new one from exp.4 has problem described here: #795
I tested the merged version but still I can see the problem, nothing changed. Is it possible to work incorrectly because I have the previous version of webserver? The current one that I downloaded has problem with missing js files. Could you please update it?
@Rhanjie I fixed the logic in web client so you need to use the latest version of webserver.
@karasusan Thank you for the information. I built new webserver and now the problem is fixed! Mouse and touch position are updated correctly even in mobile. 🎉
Unfortunately, with longer testing I found two other problems. There is still problem when the other client connects to active stream. Either it does not detect touch gestures at all or it is blocked for some reason. After refreshing the website the new client has restored ability to control the application but the first one loses it. The problem is even if I use two separated devices like PC and mobile phone.
In addition, there is a slight offset in the detected position, which is not very problematic, but makes it a little difficult to use the application. The problem occurs when a second client takes the application focus. Before this situation, the position is ok.
I presented the problem in the video below: https://www.youtube.com/watch?v=rFifZz6fXkI
@Rhanjie
I haven't assumed that the use-case which multiple users control one camera. The InputReceiver class is designed for controlling input by single player.
I would like to make another ticket for fixing your issue. That will be alright won't it?
Of course, I will wait patiently for the next updates. Thank you for fast reaction and I really appreciate your work. The most important thing is that the main problem with the incorrect position is now fixed so I can go forward with my application. 😄
Should I create the new issue for this problem?
We fixed this issue for the next version, but we still have an issue sometimes. We are checking it.