Automate Compatibility Database Testing

[jet082]

This is a bit of an odd request, but I noticed you had a compatibility database. It seems to me that, in this early stage of development at least, it might be worth automating some of it. What I mean can be better explained in a use case.

For instance, I do the following in WSL:

for i in `ls ./ios2/*.ipa`; do ./touchHLE.exe "$i"; done

This outputs the following for each crashing app:

App bundle info:
- Display name: 101 Games
- Version: 1.2
- Identifier: com.nordcurrent.101-in-1Games
- Internal name (canonical): 101-in-1Games.app
- Minimum OS version: 2.2.1

No options found for this app in touchHLE_default_options.txt
No options found for this app in touchHLE_options.txt

touchHLE::gles: Creating an OpenGL ES 1.1 context:
touchHLE::gles: Trying: Native OpenGL ES 1.1
touchHLE::gles: => Failed: Could not initialize OpenGL / GLES library.
touchHLE::gles: Trying: OpenGL ES 1.1 via touchHLE GLES1-on-GL2 layer
touchHLE::gles: => Success!
touchHLE::window: Driver info: OpenGL 4.6.0 NVIDIA 551.23 / NVIDIA Corporation / NVIDIA GeForce RTX 3080/PCIe/SSE2
touchHLE::dyld: Warning: unhandled non-lazy symbol "_exit" at 0xb00b0 in "101-in-1Games"
touchHLE::dyld: Warning: unhandled non-lazy symbol "___mb_cur_max" at 0x3800c8e0 in "libstdc++.6.dylib"
CPU emulation begins now.
touchHLE::libc::cxxabi: TODO: __cxa_atexit(GuestFunction(0x3000ca90), 0xfffffe58, 0xfffffe58) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_finalize(0xfffffe58) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_finalize(0xfffffe58) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_atexit(GuestFunction(0x3000cf44), 0x3000b570, 0x3000c000) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_atexit(GuestFunction(0x3000cf44), 0x3000b580, 0x3000c000) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_atexit(GuestFunction(0x4a64), 0x3000b590, 0x1000) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_atexit(GuestFunction(0x4a64), 0x3000b5a0, 0x1000) (unimplemented)
touchHLE::libc::cxxabi: TODO: __cxa_atexit(GuestFunction(0x4a64), 0x3000b5b0, 0x1000) (unimplemented)
touchHLE::frameworks::uikit::ui_nib: TODO: UIProxyObject replacement for IBFirstResponder, instance 0x3000b6c0 left unreplaced
thread 'main' panicked at 'Object 0x3000b8a0 (class "EAGLView", 0xb62d8) does not respond to selector "setExclusiveTouch:"!', src\objc\messages.rs:60:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Register state immediately after panic:
         R0: 0x3000b8a0  R1: 0x000ab91c  R2: 0x00000001  R3: 0x43700000
         R4: 0x3000b8a0  R5: 0x3000b5c0  R6: 0xffffff24  R7: 0xfffffefc
         R8: 0x00000000  R9: 0x00000000 R10: 0x00000000 R11: 0x00000000
        R12: 0x00004b78  SP: 0xfffffed4  LR: 0x00004ec4  PC: 0x000aadf0
Attempting to produce stack trace for main thread:
 0. 0xaadf0 (PC)
 1. 0x4ec4 (LR)
 2. [host function]
 3. 0x48ac
Next FP (0x0) is outside the stack.

It seems to me that the App Bundle information could be automatically inserted into the compatibility database alongside the names of the unimplemented functions. Moreover, as functions get implemented, perhaps this database could be updated automatically and when an app becomes free of 'unimplemented functions' it could be automatically tested again to see if it crashes.

This might help people find games that unexpectedly work and allows devs to pinpont which functions are the most widely used and thus would be the most effective to implement early. I would imagine as devs you have a whole archive of ipa files ripped from old devices that you're using for testing things.

I could be barking up the wrong tree with this and, if so, I apologize. You get a lot of weird issues and I would prefer not to be one of them. I just noticed that so much information was being provided on a crash that is also used in the compatibility database form that it might be plausible to not rely as much on human input.

[nighto]

Hello @jet082, I think so too :) eventually it would be interesting to mass-add it to the appdb and have it showing some sort of percentage support, similar to other emulators.

I did a mass test (very similar to your idea) with a certain package of iOS 2.x apps, towards the 0.2.1 release. Here's a link to a Google spreadsheet: https://docs.google.com/spreadsheets/d/1ggu8l5rrELldwdmsIF6SuvY_18zUMk1gyxIFikDs4iI/edit#gid=418490059

Cheers

[nighto]

And to document how I did it, it was with:

for i in path/to/*.ipa ; do path/to/touchHLE.exe "$i" 2> "$i.txt" ; done

And then, I got the panic reason with:

cat path/to/*.txt | grep "panicked" > panicked.txt

[nighto]

In the long run, it would be even nicer to have a CLI option, opt-in of course, in which you would log in with your GitHub account and it would create / update the appdb entry automatically with the log.