PKHeX icon indicating copy to clipboard operation
PKHeX copied to clipboard

Check nicknames/OTs against characters in font

Open abcboy101 opened this issue 1 year ago • 4 comments

This adds data on which Unicode codepoints are in the games' fonts, so that nicknames/OTs can be validated against the font and warnings can be shown if these names include characters that won't be displayed properly. The fonts contain a superset of the enterable characters in a given game/language, so it won't catch everything that's illegal, but everything it does catch should be.

  • For the warning: If a nickname/OT contains a character that is not in the font for the current game/language, a button with a "?" is displayed next to the text box. Clicking on it will display a warning such as The nickname "하양이" will be displayed as "???" in-game.
  • For legality: A nickname/OT is checked against the font in the origin game/language, with a special case for unnicknamed transfers that could have been nicknamed in Gen 8/9. If it has an invalid character, it adds a message like Nickname has character not in font.

The warning and the legality check are separate so that the warning can still appear for affected legal Pokémon (such as an ENG-tagged Pokémon hatched in a Korean game in Gen 5-7, whose nickname/OT would display as question marks in Gen 8/9), since even the same nickname/OT can be displayed differently in-game depending on the Pokémon's language.

abcboy101 avatar Jan 02 '24 17:01 abcboy101

This PR is causing an exception whenever the blank save is changed to gen 1-4. image

Lusamine avatar Jan 04 '24 03:01 Lusamine

Looks like this is also causing errors when a past-gen Pokemon is in a future generation, e.g. gen 3 being legality checked in gen 5.

image image

Lusamine avatar Jan 04 '24 04:01 Lusamine

With all the potential combinations (and future permutations), it may be wise to abstract a KeyboardKeySet (aka keyset) class/interface for each game & language (if different), and to have the logic call into all implementations accessible to the player. Each implementation would just be a singleton, essentially how I handled the Learnset iteration validation (implementations for each game). Name could be KeyboardEntry8 for SW/SH or whatever, just to brainstorm names.

Naturally, OT Names can only have the OT keyset, which should be fairly straightforward to check against a singular keyset. For nicknames, would have to iterate through all contexts & associated keysets (if multiple for a context). Likely would need an API of sorts to determine what keysets are available, and maybe a (EntityContext, Language) return tuple to richly indicate which keyset validated the nickname.

Essentially a codified abstraction of Bulbapedia's Text entry documentation :D

The text entry validation would ideally use the SearchValues<char> from an incrementally-ordered ReadOnlySpan<char>.

kwsch avatar Jan 04 '24 06:01 kwsch

Two additional issues: The Switch keyboard check is missing some symbols, so a CHS Pokémon that has them in the name or OT is incorrectly flagged. 2024010415053300-50E2A11CE4BDDC72EF99DF78315D4938 0625 - Bisharp - Gentle - 10.7.14.24.26.23 - Anubis - 832328 - Master - 543F00919A1C.zip Here's an example Bisharp captured on a CHS game and nicknamed with ingame keyboard.

I found some of my files from 2021 where I caught a Pokémon in gens 3-7, nicknamed them symbols in ENG games, and transferred them upward into each game and finally into SWSH. Some of these are getting flagged incorrectly. symboltransfer.zip

Keyboard verification has been an unsolved issue for years now because of lack of documentation, i.e. we need to know which glyphs are possible in every game and every language, what happens when they are transferred between the games, and if there are any language quirks that affect the data (such as transferring a JPN Mew up from an ENG Emerald). While what you have is a great start for validating specific games, in the absence of full knowledge of all transfer remappings, it's better to relax the check for transferred Pokémon to eliminate false positives. Eventually as knowledge becomes more complete, the checks can be tightened up again.

Lusamine avatar Jan 04 '24 21:01 Lusamine