bug(android): internal engine unable to use keyboards

[sentry-io[bot]]

@jahorton here.

Sentry Issue: KEYMAN-WEB-5K

UnhandledRejection: Non-Error promise rejection captured with value: Cannot find the Warang Citi keyboard for Ho (Warang Citi) at /data/user/0/com.tavultesoft.kmapro/app_data/packages/warang_citi/warang_citi.js?v=1664969917797.

I've gone through and merged a number of related Sentry events that all resolve to the same point of failure - KMW's installKeyboard function, which loads a keyboard's script within the context of its host page, is unable to load the keyboard from the specified file.

https://github.com/keymanapp/keyman/blob/0191d9a3ac81c5da9a3fa44eeb617fdcdb4381e3/web/source/keyboards/kmwkeyboards.ts#L735-L749

The message actually just indicates that the loading <script> tag generated an error, not load. Technically, we aren't currently checking that the error's due to a 404 (file not found)... but the net result is the same. Though... I suppose if we could verify exactly what sort of errors we're getting internally, that might help?

Just to make things extra explicit: we've (naturally) been assuming that Android code can simply pass a file URL off to the internal KMW engine, which KMW can then load here via script tag. For whatever reason, KMW is unable to load the provided file URL. As KMW is confined within a WebView used by the Android Keyman engine, it's Android-code's responsibility to ensure that the file is ready and available for KMW.

A quick brainstorm of possible causes, some of which can probably be eliminated very quickly:

• Might there be file permissions at play?
• Cross-origin security issues, despite being file-system local?
• Have the installed files straight-up gone missing somehow?

It's worth noting that this happens across the full range of supported Android devices, so it shouldn't be due to something specific to new versions of the OS:

[sentry-io[bot]]

This also happens with FirstVoices for Android:

Sentry issue: KEYMAN-WEB-39

[jahorton]

A general view of any related error I could find, after having merged a lot of very clear duplicates:

Interestingly, the third from the bottom is actually from iOS - it's pretty clear from the URL structure. But it's only got a count of 8, as opposed to a sum total of 1000+, and it apparently hasn't been seen for iOS in over two months - hence why the focus on Android.

[jahorton]

Just to be sure, since this issue's bug has a longer Sentry issue than #7412, here's the three-month filter for this issue:

Yep, that user count's still way higher.

[jahorton]

The message actually just indicates that the loading <script> tag generated an error, not load. Technically, we aren't currently checking that the error's due to a 404 (file not found)... but the net result is the same. Though... I suppose if we could verify exactly what sort of errors we're getting internally, that might help?

... and the error object doesn't provide any information about HTML header error codes or the like. It's easy enough to see what's available through KMW's keyboard-errors test page:

I'd hoped that'd be a good angle for investigation, but it doesn't look promising now that I look a bit more into it.

[darcywong00]

I was really confused how KMEA could end up with a filename like /data/user/0/com.tavultesoft.kmapro/app_data/packages/warang_citi/warang_citi.js?v=1664969917797

Then I saw KeymanWeb is appending a timestamp for "cache busting".

// Get KMEI, KMEA keyboard path (overrides default function, allows direct app control of paths) keymanweb.getKeyboardPath = function(Lfilename, packageID) { return Lfilename + "?v=" + (new Date()).getTime(); /cache buster/ };

Are you saying KMEA has a valid file /data/user/0/com.tavultesoft.kmapro/app_data/packages/warang_citi/warang_citi.js but needs to flag if the webview fails to load it?

[jahorton]

I was really confused how KMEA could end up with a filename like /data/user/0/com.tavultesoft.kmapro/app_data/packages/warang_citi/warang_citi.js?v=1664969917797

Then I saw KeymanWeb is appending a timestamp for "cache busting".

// Get KMEI, KMEA keyboard path (overrides default function, allows direct app control of paths) keymanweb.getKeyboardPath = function(Lfilename, packageID) { return Lfilename + "?v=" + (new Date()).getTime(); /cache buster/ };

Are you saying KMEA has a valid file /data/user/0/com.tavultesoft.kmapro/app_data/packages/warang_citi/warang_citi.js but needs to flag if the webview fails to load it?

All I can currently say is that as far as KMW can tell, the keyboard file it's expecting does not exist.

This could be because it actually doesn't exist, it's not at the expected location, or that it's not accessible to the WebView for some reason. The exact reason isn't currently clear.

I believe that appending ?v=... doesn't actually affect the URL used to retrieve the file; those are just 'query' parameters that can be parsed on page load. That said, the presence of query parameters does help with cache busting.

[darcywong00]

I'm starting to wonder if this is fixed with #7472

I noticed several of the Sentry logs involved the globe key and dismissing a key preview.

[sentry-io[bot]]

Sentry issue: KEYMAN-WEB-4H

[jahorton]

I'm starting to wonder if this is fixed with #7472

I noticed several of the Sentry logs involved the globe key and dismissing a key preview.

The PR mentioned here was merged in 16.0.85-alpha.

In retrospect, I don't think we can say it was fixed by that PR.

[darcywong00]

Hm, I just noticed another weird thing from Sentry is ~90% of the instances are from the

url:"file:///data/user/0/com.firstvoices.keyboards/app_data/keyboard.html" url:"file:///data/data/com.firstvoices.keyboards/app_data/keyboard.html"

~Maybe the Keyman and FV system keyboards are trying to access each other's keyboards?~ Nope, it's just a ton of instances where Keyman can't access kmapro keyboards and FV can't access fv keyboards.. Still very confusing

[darcywong00]

One instance was

Cannot find the Hǝn̓q̓ǝmin̓ǝm keyboard for Hǝn̓q̓ǝmin̓ǝm at /data/data/com.firstvoices.keyboards/app_data/packages/fv_all/fv_henqeminem_kmw.js?v=1704913742733

Where fv_all.kmp doesn't even bundle fv_henqeminem_kmw.js (expecting fv_henqeminem). Is that from a pre-migration file?

https://github.com/keymanapp/keyman/blob/6b6284fd7932ca6392c47d37cc4c85db77ad685c/oem/firstvoices/keyboards.csv?plain=1#L17

[darcywong00]

I believe #10907 fixes most of the Sentry instances

I'm still investigating why KeymanWeb can't find the following keyboards: kmapro/app_data/packages/chatino/chatino.js - not in the Keyman repo kmapro/app_data/packages/mon_anonta/mon_anonta.js

[darcywong00]

Gonna close this as mostly fixed by #10907. We can see how prevalent this issue still is when 17.0 goes to stable.

[darcywong00]

For future reference, Sentry had an instance of kmapro not finding chatino.kmp.

It installed and ran fine for me on 17.0.298-beta

[mcdurdin]

Is this really fixed? I am hesitant to close this issue if we are not clear on the cause and the resolution.

[darcywong00]

Is this really fixed? I am hesitant to close this issue if we are not clear on the cause and the resolution.

I'd say mostly fixed at this point.

Nearly all of the Sentry reports involve attempting to fallback to to EuroLatin (SIL) (when it's not currently installed).

85% of the Sentry reports are in the FV app, which will be fixed in 17.0 with #10907 Most of the Keyman app reports are fixed in 17.0 with #10794.

There's a handful of keyboard errors that remain where we don't have more visibility on why. Chatino Mon Anonta

[mcdurdin]

OK, so that looks like keyboards installed from local file rather than from keyman.com?