[AI Translation] Could not read viewport when video driver is d3d11 for PPSSPP.

[OsamaHeweitat]

Description

When attempting to use the AI translation service in the PPSSPP core while d3d11 is selected as the video driver, the following error is thrown and nothing occurs: [INFO] Could not read viewport for translation service...

Expected behavior

The AI translation service will work and output the translation in the desired mode.

Actual behavior

RetroArch throws an error.

Steps to reproduce the bug

1. Select d3d11 as the video driver.
2. Open any PSP game using the PPSSPP core.
3. Attempt translating using the translation hot-key, nothing will occur and an error will be thrown in the log.

Version/Commit

You can find this information under Information/System Information

• RetroArch: 102abb0

Environment information

• OS: Windows 10 64-bit

[neosonic2]

I am experiencing this same issue while attempting to use the AI service with games under any core that supports D3D11 (including Dolphin), though it also occurs with cores supporting D3D12. I am running RetroArch 1.16.0 on an ASUS ROG Zephyrus G14 2023 laptop running Windows 11 Pro x64 and the JAWS for Windows screen reading software for the visually impaired. The issue occcurs regardless of whether AMD integrated or NVIDIA GeForce discrete graphics are being used.

[PCdasala]

Mine is like this with Dreamcast and PS2 games only, were you able to solve this problem?

[neosonic2]

Unfortunately I haven’t found a solution or workaround yet. The issue appears for me on GameCube (Dolphin), Dreamcast, PS1, and PS2 cores. Switching to any other video backend besides Direct3D causes RetroArch to crash when I press the AI service hotkey (though sometimes it will actually work once but then crashes when I try a second time). I’ve tried all of this on a different Windows laptop that I have (also running Windows 11 but with an older graphics card) and the same issues occur. I am new to RetroArch and not sure of the best way to go about resolving or even diagnosing the issue.

On Feb 20, 2024, at 8:52 AM, PCdasala @.***> wrote:

Mine is like this with Dreamcast and PS2 games only, were you able to solve this problem?

— Reply to this email directly, view it on GitHub https://github.com/libretro/RetroArch/issues/14432#issuecomment-1954261353, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJDEDGNEHIPPTTLSB3CDQTYUSTCTAVCNFSM6AAAAAAQRFIQP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGI3DCMZVGM. You are receiving this because you commented.

[PCdasala]

issue.

So friend, this is exactly my problem, I realized that I can't take a screenshot from these emulators, so this is where the problem is, they aren't able to capture the screen from these emulators to do the translation, I just don't know how to solve it! I want to be able to play Grandia 2 in Portuguese

[BarryJRowe]

Can we get a retroarch log output for when RA crashes when the ai service is called for certain drivers? If the d3d drivers do not have screenshot capability, then it would be expected that they wouldn't work with the ai service. The other video drivers should be able to take screenshots however.

[neosonic2]

Can we get a retroarch log output for when RA crashes when the ai service is called for certain drivers? If the d3d drivers do not have screenshot capability, then it would be expected that they wouldn't work with the ai service. The other video drivers should be able to take screenshots however.

Here is log output from a recent run of RetroArch showing the error thrown by the AI translation service. Note that what is not mentioned here is that while the laptop I am using does include built-in AMD graphics, it also has (and RetroArch also uses) an NVIDIA GeForce RTX4060 dedicated graphics card. The AI translation service errors are thrown regardless of which graphics card and which D3D backend (11, 12, etc.) is selected in RetroArch.

[INFO] === Build ======================================= [INFO] CPU Model Name: AMD Ryzen 9 7940HS w/ Radeon 780M Graphics
[INFO] Capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4 SSE42 AES AVX AVX2 [INFO] Version: 1.16.0 [INFO] Git: fdc7fe5 [INFO] Built: Jan 17 2024 [INFO] ================================================= [INFO] === Build ======================================= [INFO] CPU Model Name: AMD Ryzen 9 7940HS w/ Radeon 780M Graphics
[INFO] Capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 SSE4 SSE42 AES AVX AVX2 [INFO] Version: 1.16.0 [INFO] Git: fdc7fe5 [INFO] Built: Jan 17 2024 [INFO] ================================================= [ERROR] [Environ]: SET_HW_RENDER: Dynamic verify HW context failed. [ERROR] [Environ]: SET_HW_RENDER: Dynamic verify HW context failed. [ERROR] Requesting OpenGLES3 context, but RetroArch is compiled against OpenGL. Cannot use HW context. [ERROR] [Environ]: SET_HW_RENDER - Dynamic request HW context failed. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport. [ERROR] [Translate] Could not read viewport.

[hota-hota]

I got the PROBLEM RESOLUTION. I also received the "Could not read viewport" message on the screen with PSP and PS2 games, but I did not receive this message in SNES games, that is, the translations worked in SNES games, but did not work in PSP and PS2 games . I only tested PSP, PS2 and SNES games. I'm using Windows 10. Let's get to the resolution:

1st - Follow the steps in this video, it has everything you need to do: https://www.youtube.com/watch?v=x9MxSYecNYw It's in Chinese, but you can find your way around.

2nd - You will have to make an account on ztranslate and get a key.

3rd - ONLY USE THE VERSIONS OF THE PROGRAMS THAT ARE IN THE VIDEO. They are: retroarch-1-9-3 (I used retroarch-1-9-6), vgtranslate_serve_v1.05, python-2.7.amd64.

4th - End.

Note: this text was translated using Google translator. Below is an attached image.

[neosonic2]

@hota-hota What solution do you have for those like myself who are visually impaired and thus unable to watch the video (and also unable to speak Chinese)?

[hota-hota]

@neosonic2, unfortunately I don't understand anything about JAWS for Windows screen reading, so I can't help you. I also don't understand the Chinese language at all. What can I tell you that installing a translation AI for PPSSPP is quite difficult. It is a process that involves all programs in the correct versions, registration on the ztranslate website and a server (vgtranslate) working separately from Retroarch.

[neosonic2]

I’ve already registered on the ZTranslate site and I have the VGTranslate client (version 1.05) and Python installed as well, but probably not in the versions you suggested in your post. If you can tell me what the video is asking users to do, then I should be able to complete those tasks on my computer even without being able to see the video. I’d like to think I am a relatively tech savvy person and I have a degree in computer science so I should be able to get this working without issue, as long as I know what steps the video requests users take.

On Mar 8, 2024, at 6:11 PM, hota-hota @.***> wrote:

@neosonic2 https://github.com/neosonic2, unfortunately I don't understand anything about JAWS for Windows screen reading, so I can't help you. I also don't understand the Chinese language at all. What can I tell you that installing a translation AI for PPSSPP is quite difficult. It is a process that involves all programs in the correct versions, registration on the ztranslate website and a server (vgtranslate) working separately from Retroarch.

— Reply to this email directly, view it on GitHub https://github.com/libretro/RetroArch/issues/14432#issuecomment-1986549130, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJDEDAMS37N4S2ERFU3SKLYXJAQTAVCNFSM6AAAAAAQRFIQP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWGU2DSMJTGA. You are receiving this because you were mentioned.

[hota-hota]

@neosonic2, I'll try to make a tutorial:

1 – Install retroarch-1-9-6 (it can only be this version). https://retro-arch.br.uptodown.com/windows/download/3781096

2 – Register with ztranslate and obtain an access key. https://ztranslate.net/signup

3 – Open Retroarch and then: Setting → Input → HotKeys → AI Service = press a button on the keyboard or control to activate the translation AI. Close Retroarch.

4 – Open Retroarch again and then: Setting →AI Service: AI Service Enable = ON AI Service Output = Image Mode AI Service URL = http://ztranslate.net/service?api_key=YOUR_KEY_FROM_ZTRANSLATE Note: in this url put the key you got after registering with ztranslate. Source Language = choose the language of your game, or leave it at default for the AI to detect. Target Language: choose the language you know how to speak.

Now close Retroarch.

5 – Install python-2.7.amd64 https://www.python.org/download/releases/2.7/

6 – Unzip the vgtranslate_serve_v1.05 file https://gitlab.com/spherebeaker/vgtranslate

7 – Inside the vgtranslate_serve_v1.05 directory look for the config.json file Now open this file with Windows Notepad and it should look like this:

{ "default_target": "En", "server_host": "ztranslate.net", "server_port": 443, "user_api_key": "", "local_server_enabled": true, "local_server_api_key_type": "ztranslate", "local_server_host": "localhost", "local_server_port": 4404, "local_server_ocr_key": "", "local_server_ocr_processor": {}, "local_server_translation_key": "" }

Now change it to:

{ "default_target": "En", "server_host": "ztranslate.net", "server_port": 443, "user_api_key": "YOUR_KEY_FROM_ZTRANSLATE", "local_server_enabled": true, "local_server_api_key_type": "ztranslate", "local_server_host": "localhost", "local_server_port": 4404, "local_server_ocr_key": "", "local_server_ocr_processor": {}, "local_server_translation_key": "" }

Note that the only change was your Ztranslate key.

8 – Go to the vgtranslate_serve_v1.05 directory and run serve.exe

9 – Open Retroarch → look for your PPSSPP game and run it (I won't teach you how to make PSP games work on Retroarch, but you can find it at: https://www.youtube.com/watch?v=kW4Ft01vMI0).

10 – Be happy :D

Note: This AI translation may work for other emulators besides PPSSPP within retroarch.

[BlindGuyNW]

Is there any movement on this? I depend on the AI service for game accessibility, and it's frustrating that some drivers seem to just not work with it lately. Dolphin used to work on an earlier version of RA but I don't remember which video driver it used.

[BlindGuyNW]

SO it seems like the fundamental issue is that the D3D backend doesn't support screenshots? I guess? Any insights from folks would be appreciated, this is crippling for my use case.

[neosonic2]

Agreed. I also need to rely on the AI service to play most games but am unable to do so reliably with the current crop of backends/options, and continue getting the error I’ve posted about in previous comments to this thread.

On May 4, 2024, at 4:09 PM, Zachary Kline @.***> wrote:

SO it seems like the fundamental issue is that the D3D backend doesn't support screenshots? I guess? Any insights from folks would be appreciated, this is crippling for game accessibility.

— Reply to this email directly, view it on GitHub https://github.com/libretro/RetroArch/issues/14432#issuecomment-2094365422, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJDEDGS6USLUEQ4FDKVEPLZAU57HAVCNFSM6AAAAAAQRFIQP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGM3DKNBSGI. You are receiving this because you were mentioned.

[BlindGuyNW]

After doing a little more investigation, I'm able to get output of a sort on other drivers such as Vulcan. However, the output appears corrupted, that is, the results have no baring on what is actually shown on screen. It's also very much unstable.

I would love to know why this works reliably on some cores, such as MGBA, and not at all on others, such as PPSPP and Dolphin.

[gouchi]

I reproduced the issue using d3d11 video driver and PPSSPP libretro core which needs hardware acceleration.

[ERROR] [Translate] Could not read viewport.

I guess this issue is only happening with cores which need hardware acceleration hw_render = "true" ?

EDIT : I tested with latest commit for RA et PPSSPP core.

[BarryJRowe]

@BlindGuyNW @neosonic2 @gouchi Do we have confirmation that this is happening with the latest version of RA, and that when using the D3D11 driver, screenshots are working? The AI service uses the core's screenshot functionality, so if the driver is unable to take a screenshot, then we know it's a problem in the driver, and not in the AI service. Otherwise, it's an issue with the AI service.

[BlindGuyNW]

@BarryJRowe On RetroArch 1.17.0, screenshots with D3D11 do not appear to work. They do with the Vulcan driver, but the AI service failure mode is different in that case.

[BarryJRowe]

@BarryJRowe On RetroArch 1.17.0, screenshots with D3D11 do not appear to work. They do with the Vulcan driver, but the AI service failure mode is different in that case.

There was a revert of a breaking change from an earlier PR. Try 1.18.0 for both if you can.

[BlindGuyNW]

Alright. D3D11 still appears to be broken, I still can't take screenshots, but I don't get that viewport error either. It just acts as if the AI service hotkey wasn't pressed.

Vulcan on the other hand works as expected so far. The OCR and translation don't appear to crash and are comprehensible.

I'd like to know what the underlying issue with d3D is however, seeing as it is default at least on my Windows machine.

[gouchi]

I confirm using latest RA commit and d3d11 video driver, it seems not possible to take screenshot for hardware accelerated cores.

I thought it was linked to this issue but the GPU screenshot option is not present when using d3d11 video driver.

[gouchi]

I had the confirmation there is no support for GPU screenshot for d3dx, if you look for GFX_CTX_FLAGS_SCREENSHOTS_SUPPORTED, it is not enabled for d3dx.

So it must be the issue why it is not working for hardware accelerated cores.

[gouchi]

I confirm it is working using #16760 with D3D11 video driver and hardware accelerated cores.

I tested it with PPSSPP core.

@OsamaHeweitat @BlindGuyNW @neosonic2 May you try to make a test with latest nightly ? So that, if it is working for you, we can close this issue.

Thank you.

[neosonic2]

I tested the nightly release linked to in the comment and had success with both the Nintendo GameCube and Sega Dreamcast cores. Text was read out after a delay of a few seconds once I pressed the AI service hotkey, and no crashes were observed even after calling up the AI service a few times during the course of game play.

However, one thing I noticed is that the hotkey has to be pressed every time the user wants a new translation, even though the service documentation https://docs.libretro.com/guides/ai-service/#automatic-mode seems to imply that an “automatic mode” is present to allow for continuous translation. What must be done to enable this mode?

My sincere gratitude to everyone who worked on fixing this AI service bug. It makes games much more playable for me now that the service works as intended (minus the automatic mode) regardless of the core I use. I don’t know which video driver was in place at the time, but I assume since I was running on a Windows 11 laptop that D3D was probably the selected driver.

On Jul 8, 2024, at 4:48 PM, gouchi @.***> wrote:

I confirm it is working using #6427 https://github.com/libretro/RetroArch/issues/6427 with D3D11 video driver and hardware accelerated cores.

I tested it with PPSSPP core.

@OsamaHeweitat https://github.com/OsamaHeweitat @BlindGuyNW https://github.com/BlindGuyNW @neosonic2 https://github.com/neosonic2 May you try to make a test with latest nightly https://buildbot.libretro.com/nightly/windows/x86_64/RetroArch.7z ? So that, if it is working for you, we can close this issue.

Thank you.

— Reply to this email directly, view it on GitHub https://github.com/libretro/RetroArch/issues/14432#issuecomment-2215257754, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJDEDFYBT72MLFF2Q6XOHLZLL3IVAVCNFSM6AAAAAAQRFIQP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJVGI2TONZVGQ. You are receiving this because you were mentioned.

[gouchi]

@neosonic2 Thank you for the test and feedback.

Thanks to @miguedevel for the fix.

@OsamaHeweitat Please close this issue as it has been fixed. Thank you.