Is there any way to get content of left/right eyes?

[NukeBird]

trafficstars

I have a chinese friend and we want to watch anime together, but sadly it has only chinese subtitles. I came up with an idea of real-time translation and projecting translated text back to VR

I have no problems with detecting and translating text, also I think I can find the way how to properly project back translated texts

BUT, I have no idea how to get color buffers of left/right eyes. Is it even possible? If so, will I have any issues with guard systems? I want to use this util in the game called VRChat. Technically I'm not trying modify a game code or benefit in any way. I just want to be with my friend

[Rectus]

There isn't any official way that would let you both read from the frames and write content over them.

It is possible to hook the submit functions to modify the frames though, but it will likely trigger anticheat software. There are several ways of hooking them, one is using a custom version of openvr_api.dll, like this application does: https://github.com/fholger/openvr_fsr

[TheDeveloperGuy]

It depends on the driver mode that the headset is using. I've written something that does exactly this for a commercial client. The method relies on "driver direct" (driver compositor) driver mode. This is used by Oculus and WMR headsets (and some streaming headsets). It won't work for headsets that use the SteamVR compositor (eg. Vive, Index etc.). The "driver direct" method involves hooking some internal OpenVR driver APIs, but is undetectable by client apps.

[TheDeveloperGuy]

Needless to say this is very advanced OpenVR driver development, and if you're asking this question, you probably don't have the level of skill to do the driver intercept solution mentioned. At the very least you need the skill level to write a "driver direct" driver. Using this you would do your own composition, doing whatever modifications you wanted to, and then pass the modified image as a single layer to the original headset driver.