Android Vukan renderer doesn't respect Landscape

[AntTheAlchemist]

Everything's drawn in portrait. I haven't looked very deep into this, but here's the following Landscape config I'm using:

SDL_SetHint(SDL_HINT_ORIENTATIONS, "LandscapeLeft LandscapeRight");
&
<uses-feature android:name="android.hardware.screen.landscape" android:required="true" />
&
android:screenOrientation="landscape"

[slouken]

I'm pretty sure Vulkan bypasses the OS orientation and that has to be handled in the renderer. Does that square with your understanding, @danginsburg?

[danginsburg]

Yes, that's correct. We need to add swapchain prerotation support to the renderer.

[slime73]

Making it use VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR instead would also work (just with lower theoretical performance), right?

[slouken]

Yes, that's correct. We need to add swapchain prerotation support to the renderer.

Any hints on how to do that, or do you want to throw together a quick PR for this?

[danginsburg]

Any hints on how to do that, or do you want to throw together a quick PR for this?

I will try to find time to take a look at this. I think there is code we can hoist from the D3D12 renderer which is already handling orientation.

[slouken]

@danginsburg, any luck with that? I'd like to either have something working for 3.2.0 release, or disable the Vulkan renderer by default on Android.

[danginsburg]

@danginsburg, any luck with that? I'd like to either have something working for 3.2.0 release, or disable the Vulkan renderer by default on Android.

I'm not sure precisely how to reproduce the issue. I got SDL3 setup to build/run on Android, but when I try for example testsprite it renders landscape and handles flipping 180 the same as the GLES2 backend. Does one of the SDL test examples demonstrate the issue so I can debug it further?

[AntTheAlchemist]

Allow me to dive back into this and come up with a simple repro. Stand by.

[danginsburg]

So actually, I was not getting the Vulkan renderer when I tested this earlier. I am now, but I am hitting this issue https://github.com/libsdl-org/SDL/issues/10279 which it looks like you were hitting too @AntTheAlchemist. That is, vkCreateAndroidSurfaceKHR is returning VK_ERROR_NATIVE_WINDOW_IN_USE_KHR, so that needs to be fixed before I can even look at this.

[danginsburg]

If I locally hack SDL_DefaultGraphicsBackends to not return SDL_WINDOW_OPENGL on Android then I get surface creation to work, and if I specify the window is SDL_WINDOW_RESIZABLE I am seeing it correctly handle landscape/portrait orientations on Google Pixel XL3.

[AntTheAlchemist]

VK_ERROR_NATIVE_WINDOW_IN_USE_KHR is something new. It got further than that before, so I can't get passed that to look into this again.

@danginsburg , did you try the settings mentioned in the OP? They set the app to use landscape only, which is where everything was drawn at 90 degrees in a portrait.

[danginsburg]

@danginsburg , did you try the settings mentioned in the OP? They set the app to use landscape only, which is where everything was drawn at 90 degrees in a portrait.

I did not try that, but I did modify testsprite to support SDL_WINDOW_RESIZABLE and got rotation to work and it did the right thing in landscape and portrait. So it sounds like the bug is actually with explicitly setting landscape it will still render in portrait? I need to figure out how to repro that (I know you mentioned it above, but if you could modify one of the test programs to do it and post a diff or a branch somewhere that would make it easier for me to investigate).

[AntTheAlchemist]

I can probably repro with a basic main() example, but, I can't get passed the "vkCreateAndroidSurfaceKHR failed: VK_ERROR_NATIVE_WINDOW_IN_USE_KHR" error, which is new. What code should I change to side-step this bug?

[slouken]

I can probably repro with a basic main() example, but, I can't get passed the "vkCreateAndroidSurfaceKHR failed: VK_ERROR_NATIVE_WINDOW_IN_USE_KHR" error, which is new. What code should I change to side-step this bug?

Is there a separate issue with repro case tracking this?

[danginsburg]

Is there a separate issue with repro case tracking this?

This is #10279. The patch in this comment should fix it https://github.com/libsdl-org/SDL/issues/10279#issuecomment-2397748531

We were waiting for @captain0xff to confirm it fixes his issue. I tested it locally and it fixed the issue for me. The problem is that by default it's initializing with EGL which makes the surface not usable by Vulkan.

[slouken]

Okay, that change is pushed. @AntTheAlchemist, you should be able to get past that now.

[captain0xff]

Sorry, for the wrong comment (now deleted). I wrongly assumed that the vulkan renderer is now default on android. But it appears that isn't the case. Anyways, so after realising that, I tried the vulkan driver and both https://github.com/libsdl-org/SDL/issues/10279 and this issue are still reproducible on my end. And yeah my SDL build is latest and I confirmed that it has the patch. The only thing this patch seemed to fix is that the SDL_WINDOW_VULKAN flag is not needed anymore (which iirc was needed earlier to use the vulkan backend on android).

[AntTheAlchemist]

I have a simple repro. No explicit settings needed. It just happens every time in Landscape, no matter what configuration I choose. So, it could be specific to my hardware? Nokia 8.

This code simply draws a green box at the top left of the window. But, it's being drawn bottom left, because all drawing is rotated 90 degrees left. Rotate the phone portrait and it works ok. (screenshort catches me working at 3am again - I know I said I'd stop that 😪 )

#include <SDL3/SDL_main.h>
#include <SDL3/SDL.h>
int main(int argc, char** argv) {
    SDL_Init(SDL_INIT_VIDEO);
    SDL_Window *w = SDL_CreateWindow("Test", 640, 480, SDL_WINDOW_RESIZABLE);
    SDL_Renderer *r = SDL_CreateRenderer(w, "vulkan");
    bool running = true;
    while (running) {
        SDL_PumpEvents();
        SDL_Event event;
        while (SDL_PeepEvents(&event, 1, SDL_GETEVENT, SDL_EVENT_FIRST, SDL_EVENT_LAST) == 1) {
            switch(event.type) {
                case SDL_EVENT_KEY_UP:
                case SDL_EVENT_QUIT:
                    running = false;
                    break;
            }
        }
        SDL_SetRenderDrawColor(r, 64, 64, 64, SDL_ALPHA_OPAQUE);
        SDL_RenderClear(r);
        SDL_FRect box = { 5, 5, 300, 300};
        SDL_SetRenderDrawColor(r, 0, 255, 0, SDL_ALPHA_OPAQUE);
        SDL_RenderFillRect(r, &box);
        SDL_RenderPresent(r);
    }
    SDL_DestroyRenderer(r);
    SDL_DestroyWindow(w);
    SDL_Quit();
    return 0;
}

[AntTheAlchemist]

Notice how the box is stretched, because renderer coordinates, and all input coordinates are working on landscape coordinates.

Just tested on a Samsun S20 and it does exactly the same. So that's an old and new phone with Vulkan, so everyone should be able to repro this.

[AntTheAlchemist]

It's not hard to reproduce this; it's happening everywhere.

[slouken]

Diving into SDL, GPU is the default renderer! GPUs selected backend chooses Vulkan. backends[] doesn't contain opengles2.

So effectively, Vulkan is still the default renderer.

I think my phone doesn't have capabilities that SDL GPU needs, so I get opengles2 instead. If you explicitly set the vulkan renderer with SDL_SetHint(SDL_HINT_RENDER_DRIVER, "vulkan"), do you have the same problem?

[AntTheAlchemist]

Yes, and SDL_CreateRenderer(window, "vulkan");. I initially found the problem while using SDL_GPU_DISABLED, because I wanted to test vulkan. So it's 100% a vulkan issue, not GPU, or opengl.

[slouken]

It's interesting that I'm not seeing it here. What devices are you able to reproduce this on?

[AntTheAlchemist]

Nokia 8, and a Samsung S20. Old and newer phones. I have no other Vulkan devices to test on. I did wonder if it was my Android Studio project, until I tried your apk. Now about to try emulator...

[AntTheAlchemist]

So you implicitly request "vulkan" and it works for you?

[slouken]

So you implicitly request "vulkan" and it works for you?

Yep!

[AntTheAlchemist]

I guess it works on some devices and not others because of: "The Android OS can use the device's Display Processing Unit (DPU), which can efficiently handle surface rotation in hardware. Available on supported devices only." is the current implementation? How do we know which devices are supported?

A wholeistic solution is the compositor pass, which is what @slime73 mentioned, which would take a potential performance hit?

Or SDL could just rotate the whole surface accordingly? "The application itself can handle the surface rotation by rendering a rotated image onto a render surface that matches the current orientation of the display."

Is this good info on this problem? It mentions doing something different on Android 10+ https://android-developers.googleblog.com/2020/02/handling-device-orientation-efficiently.html

[slouken]

@AntTheAlchemist, can you retest with the latest code that fixes the Vulkan viewport?

[AntTheAlchemist]

@slouken, that didn't fix it, unfortunately. Didn't appear to change anything.

[rabbit-ecl]

I'm also seeing this when using SDL_GPU directly. It seems the issue is that SDL_GPU & SDL_Render always set preTransform = currentTransform on swapchain creation but then don't actually do any pre-rotation when currentTransform requires it. Forcing swapchainCreateInfo.preTransform = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR; at least works for fixing the rotation mismatch on SDL_GPU. (with possible perf impact that I haven't looked into)