Adding support for RTL Custom TitleBar

[faogustavo]

We had an issue reported on Jewel that the custom titlebar/decorated window did not support RTL alignment (https://github.com/JetBrains/intellij-community/pull/3087).

The initial solution involved hiding the controls and drawing them manually on the correct side. But as this is an issue with the title bar, we thought it would be better to fix inside the runtime.

Solutions

While debugging the JBR, I found three possible solutions:

Fixing the button orders

The first solution was simply usin the WS_EX_LAYOUTRTL and WS_EX_RTLREADING on the client side. However, the button orders did not match the expectations and hover/clicks were all bugged out. (video)

Even though this is a simple solution, it would still require the client app (or the Jewel library) to perform system calls to update the window. Here is a small snippet to perform this action from Kotlin:

public fun Window.setWindowsRtlLayout() {
    if (!hostOs.isWindows) return

    val isRtl = !this.componentOrientation.isLeftToRight
    // Obtain HWND from AWT component
    val hwnd = WinDef.HWND(Native.getComponentPointer(this))

    val current = User32.INSTANCE.GetWindowLongPtr(hwnd, GWL_EXSTYLE).toLong()
    val newStyle = if (isRtl) {
        current or WS_EX_LAYOUTRTL.toLong() or WS_EX_RTLREADING.toLong()
    } else {
        current and WS_EX_LAYOUTRTL.inv().toLong() and WS_EX_RTLREADING.inv().toLong()
    }

    if (newStyle != current) {
        User32.INSTANCE.SetWindowLongPtr(hwnd, GWL_EXSTYLE, LONG_PTR(newStyle))
        // Tell the window manager to re-evaluate styles
        User32.INSTANCE.SetWindowPos(
            hwnd, null, 0, 0, 0, 0,
            SWP_NOMOVE or SWP_NOSIZE or SWP_NOZORDER or SWP_FRAMECHANGED
        )
    }
}

Add RTL flag as titlebar property (Current PR)

This solution is similar to the previous one. It still requires the "client" to pass the flag informing the RTL orientation. With this solution, we are able to achieve a solution that support RTL, without impacting the existing behavior and the IDEs.

In this implementations, even though the user forces the WS_EX_LAYOUTRTL flag, it will keep the buttons in the expected places (making the correction to placement and everything).

Matching Swing

Swing and Compose use the user Locale to determine if it should render LTR or RTL. On Compose, we use the GlobalLayoutDirection (See GlobalLayoutDirection).

We could use the same logic on C++ with the following snippet:

DWORD readingLayout; // Buffer for orientationAdd commentMore actions
int cchData = GetLocaleInfoEx(
  LOCALE_NAME_USER_DEFAULT,   // Use the user's default locale name
  LOCALE_IREADINGLAYOUT | LOCALE_RETURN_NUMBER,      // Get the reading layout for the locale
  (LPWSTR) &readingLayout,
  sizeof(readingLayout) / sizeof(WCHAR)
);

// cchData > 0 is a success result; Layout 1 or 3 means RTLAdd commentMore actions
this->rtl = cchData > 0 && (readingLayout == 1 || readingLayout == 3); Add comment

Even though this is the solution that achieves the best result, it would be the more invasive, changing the behavior on the IDE too.

Evidences

[!NOTE] This solution is only applied to the custom titlebar implementation. Default/AWT window controls were not changed

LTR (en-us)	RTL (fa-ir)

Button hover state

Before	After

[faogustavo]

Created a ticket to track it: JBR-9035 Support RTL on Decorated Window Title Bar

[kdroidFilter]

In the latest version of JBR 21.0.9, I can no longer activate RTL support. It' s work fine on 21.0.8

[vprovodin]

In the latest version of JBR 21.0.9, I can no longer activate RTL support. It' s work fine on 21.0.8

Could you please create a ticket in https://youtrack.jetbrains.com/issues/JBR?

Please provide us with more details there: JBR version, IDE version, OS.