Even winecfg doesn't launch on plucky hangover-wine DEBs

[JamiKettunen]

After installing all debs from https://github.com/AndreRH/hangover/releases/download/hangover-10.10/hangover_10.10_ubuntu2504_plucky_arm64.tar upon running anything including winecfg only wine: created the configuration directory '/home/user/.wine' is output with wineboot.exe --init pinning one core at 100% indefinitely. With WINEDEBUG=+all winecfg it crashes instead with the following log: hangover-x1e-crash.txt

I'm testing on an Asus Vivobook S 15 (X1E) running Ubuntu 25.04 (upgraded from original concept 24.10 install) with kernel 6.14.0-35-qcom-x1e. I've setup box64/32 from source (ae0c91dc) and installed FEX through https://github.com/FEX-Emu/FEX#quick-start-guide with 24.04 RootFS (extracted) and made FEX the default binfmt handler for x86* for now.

I should note that this is an AArch64-only CPU with no native AArch32 support. Let me know if I can help somehow else or if I missed something obvious :)

[AndreRH]

In your crash log you get EXCEPTION_ILLEGAL_INSTRUCTION, and I wonder why that would be the case... You mentioned you installed the Linux versions of Box64 and FEX, be aware that this is not necessary and not how hangover works, see the README. I hope that's not the cause though. Aarch64-only is a good hint, but also should not be the cause I think. I'm afraid I'm a bit lost here, you could try to run with gdb to try to find which "illegal instruction" might cause this.

[AndreRH]

One more note: It looks like it is executing on the stack, which would be harder to debug. Could you try compiling original Wine from source and run that?

[akhilpo]

I see the same issue on an X elite laptop running debian Trixie. I have tried "gdb --args wine iexplorer", but it doesn't seem to catch anything. Probably because I am using a stripped version wine from your release deb packages. For the unstripped version of wine binaries, do I have to compile the entire hangover-wine myself or is there an easier way?

[akhilpo]

BTW, I could run bylaw's version of wine with arm64ec support a few months ago: https://github.com/bylaws/wine/tree/arm64ec

[AndreRH]

I see the same issue on an X elite laptop running debian Trixie. I have tried "gdb --args wine iexplorer", but it doesn't seem to catch anything. Probably because I am using a stripped version wine from your release deb packages. For the unstripped version of wine binaries, do I have to compile the entire hangover-wine myself or is there an easier way?

Read my other comments again

[JamiKettunen]

Sorry for the late reply, I've dropped Box64 (/etc/binfmt.d/box* & /usr/local/bin/box64*) and FEX (apt remove fex-emu-armv8.4) followed by a reboot for now just to isolate them out from potentially being related to this issue. Following that I can still reproduce the issue with a fresh ~/.wine and all debs setup from hangover_10.11_ubuntu2504_plucky_arm64.tar

Following https://gitlab.winehq.org/wine/wine/-/wikis/Building-Wine I was able to get both https://gitlab.winehq.org/wine/wine/-/tree/wine-10.11 and https://github.com/AndreRH/wine/tree/hangover-10.11 running built with configure --enable-win64 and only missing (irrelevant) libpcsclite/OSS sound system support, though I probably screwed up the WoW64 stuff since I wasn't able to run https://github.com/zufuliu/notepad4/releases/download/v25.06r5696/Notepad4_en_ARM_v25.06r5696.zip for example but I'm ignoring that for now.

I'll now try getting hangover built from source and perhaps better isolate exactly what could be causing the breakage unless you have some further ideas already based on this.

[AndreRH]

Following https://gitlab.winehq.org/wine/wine/-/wikis/Building-Wine I was able to get both https://gitlab.winehq.org/wine/wine/-/tree/wine-10.11 and https://github.com/AndreRH/wine/tree/hangover-10.11 running built with configure --enable-win64

But it was built for ARM64?

I'll now try getting hangover built from source and perhaps better isolate exactly what could be causing the breakage unless you have some further ideas already based on this.

Still not much of an idea here, sorry. Just make sure you get the PE build working (--with-mingw=clang --enable-archs=arm64ec,aarch64,i386)

[JamiKettunen]

Yes those were all native ARM64 builds done on the laptop itself, it's basically as powerful as my Ryzen 9 5950X desktop after all :D but now the plot seemingly thickens: things were still just working as expected on my hangover wine build from this repo following https://github.com/AndreRH/hangover/blob/hangover-10.11/docs/COMPILE.md including --enable-archs=arm64ec,aarch64,i386,arm (yes, 32-bit ARM doesn't appear to be a problem there).

So the final bosses remain:

1. libarm64ecfex.dll: appears to work as expected after built with error pointing to copying it under wineprefix C:\windows\system32\ if not found in host /usr/local/lib/wine/aarch64-windows/ during wineprefix creation (placing it in the latter path afterward doesn't work); an example of the missing library message:

0024:err:module:load_arm64ec_module could not load L"C:\\windows\\system32\\libarm64ecfex.dll", status c0000135

2. wowbox64.dll: appears to work as expected after built, BUT only if copied under /usr/local/lib/wine/aarch64-windows/ BEFORE wineprefix creation; trying to place it in C:\windows\system32\ on an already created wineprefix causes a wine: could not load kernel32.dll, status c0000135 following the usual BOX64 header print, also no message about it missing from system32 compared to libarm64ecfex.dl is printed at all
3. libwow64fex.dll: similar to above proper error messages aren't necessarily shown when trying to use it via HODLL=libwow64fex.dll and it's missing and it also must exist at /usr/local/lib/wine/aarch64-windows/ when creating a wineprefix; provided that using it for 32-bit x86 software including https://github.com/wine-mono/wine-mono/releases/download/wine-mono-10.1.0/wine-mono-10.1.0-x86.msi it seems to love mostly just hanging while pinning one core to 100% on "random stuff" for now (out of scope here I'd presume but worth noting perhaps still)

Are the https://github.com/AndreRH/hangover/releases/download/hangover-10.11/hangover_10.11_ubuntu2504_plucky_arm64.tar builds perhaps cross-built instead of natively compiled? https://github.blog/changelog/2025-01-16-linux-arm64-hosted-runners-now-available-for-free-in-public-repositories-public-preview/ + https://github.blog/changelog/2025-04-14-windows-arm64-hosted-runners-now-available-in-public-preview/ should make life easier here if yes; I guess now the difference between these builds has to be found. At least now I have something semi-functional locally even if I can make other especially networked even simple x86/x86_64 programs hang pretty easily :)

[JamiKettunen]

I've now retried with https://github.com/AndreRH/hangover/releases/tag/hangover-10.14 and it appears this issue is still happening, but now I realized it's specific to the 25.04 (plucky) hangover-wine_10.14~plucky_arm64.deb; even if I install the 24.04 (noble) one on 25.04 all is good and then I can no longer reproduce this

[AndreRH]

winecfg and i386 apps work for me on Plucky

I have issues with the FEX based dlls on Plucky though, including ARM64EC

[icecream95]

The issue is because of BTI, which started being enabled by default in object files since Ubuntu Plucky, but which has no effect (and hence cannot cause crashes) on CPUs that do not support the feature.

The following bpftrace script shows the target and origin of the call that causes the problem:

fentry:vmlinux:do_el0_bti {
  printf("%lx %lx\n", args->regs->pc, args->regs->regs[30]);
}

So the source, from ntdll.dll, indeed does an indirect call...

Dump of assembler code for function NtAllocateVirtualMemory:
   0x0000000180064450 <+0>:     mov     x8, #0x18                       // #24
   0x0000000180064454 <+4>:     mov     x9, x30
   0x0000000180064458 <+8>:     ldr     x16, 0x180064468 <NtAllocateVirtualMemory+24>
   0x000000018006445c <+12>:    ldr     x16, [x16]
   0x0000000180064460 <+16>:    blr     x16
   0x0000000180064464 <+20>:    ret
   0x0000000180064468 <+24>:    .inst   0x80151700 ; undefined
   0x000000018006446c <+28>:    udf     #1

But the target, which looks to be __wine_syscall_dispatcher, is buggy, because it does not have a bti c instruction to indicate that it is okay to indirect call to it, and yet the ntdll.so shared object is marked as supporting the BTI feature, as seen by readelf.

   0x0000000000038400:  ldr     x10, [x18, #888]
   0x0000000000038404:  stp     x18, x19, [x10, #144]
   0x0000000000038408:  stp     x20, x21, [x10, #160]
   0x000000000003840c:  stp     x22, x23, [x10, #176]
   0x0000000000038410:  stp     x24, x25, [x10, #192]
   0x0000000000038414:  stp     x26, x27, [x10, #208]
   0x0000000000038418:  stp     x28, x29, [x10, #224]
   0x000000000003841c:  mov     x19, sp
   0x0000000000038420:  stp     x9, x19, [x10, #240]
   0x0000000000038424:  mrs     x9, nzcv
   0x0000000000038428:  stp     x30, x9, [x10, #256]
...

$ readelf -n /usr/lib/wine/aarch64-unix/ntdll.so

Displaying notes found in: .note.gnu.property
  Owner                Data size        Description
  GNU                  0x00000010       NT_GNU_PROPERTY_TYPE_0
      Properties: AArch64 feature: BTI, PAC

...

What's the fix?

A quicker fix might be to set -mbranch-protection=pac-ret to override -mbranch-protection=standard in the CFLAGS.

To actually fix Wine, it might be enough to include bti c before the code in __ASM_DEFINE_FUNC which is defined in include/wine/asm.h. (The instruction is a NOP on older CPUs, so it's okay to include unconditionally when building for AArch64.)

Finally, this is a way to strip out the BTI information with an existing install:

sudo objcopy -R .note.gnu.property /usr/lib/wine/aarch64-unix/ntdll.so

It is a bit frustrating that the Linux kernel reuses SIGILL for BTI failures and throws away the extra information that it has; otherwise it would have been easier to find what the problem is.

[AndreRH]

Excellent clarification, thanks a lot I'll look into upstreaming it

[AndreRH]

@JamiKettunen Could you please test with 10.18 as soon as the packages are built? I included https://github.com/AndreRH/wine/commit/8dbb985689702c40c5bde8cc050978d01789ce95 but can't test, cause I don't have the right hardware...

[JamiKettunen]

still appears to be reproducible on hangover-wine_10.18~plucky_arm64.deb of https://github.com/AndreRH/hangover/releases/download/hangover-10.18/hangover_10.18_ubuntu2504_plucky_arm64.tar

[icecream95]

The hex for the bti jc instruction isn't right, the instruction would actually be 0xd50324df.

But looking at the assembly GCC generates for a simple function, using bti c through the hint mnemonic should work: (The j bit is unnecessary; the architecture already accounts for tail calls using jump instructions.)

func:
        .cfi_startproc
        hint    34 // bti c
        add     w0, w0, 1
        ret
        .cfi_endproc

[AndreRH]

not sure how I messed that up, anyway, here is a test build, so... please test :) https://github.com/AndreRH/hangover/releases/tag/20251103

[AndreRH]

@JamiKettunen could you please try again?

[JamiKettunen]

seems good now, thanks :) feel free to close this or with the next proper release

[AndreRH]

upstream MR: https://gitlab.winehq.org/wine/wine/-/merge_requests/9671

[AndreRH]

Hangover 11.0-rc1 includes a new version of a fix as discussed upstream. Please retest

[JamiKettunen]

still good on my side 👍

[AndreRH]

Awesome, thanks for testing