spoken-to-signed-translation icon indicating copy to clipboard operation
spoken-to-signed-translation copied to clipboard

Fingerspelling

Open AmitMY opened this issue 1 year ago • 0 comments

I would love to add fingerspelling to this repo.

Basic:

  • [x] Collect fingerspelling data for various signed languages. One video per letter
    • [x] Upload, process, pose estimate etc.
  • [x] Curate a fingerspelling list per language. For example, "SCH" is "one letter" in swiss-german
  • [x] Implement a fallback strategy to combine the letters

Advanced

Combining letters naively is a little silly, as the transitions aren't necessarily "human".

  • [ ] For repeating letters, in supported languages (for example, no idea what they do in BSL), move the hand slightly, then, return to original position
  • [ ] Allow specifying letter transitions: annotated as "SCH-B" for example, is the transition between "SCH" and "B". "B-∅" is the transition between "B" and nothing.
  • [ ] Combine letters based on their transition (SCHULE = ∅-SCH + SCH-U + U-L + L-E + E-∅) - https://fingerspell.net/#/spell

AmitMY avatar Oct 07 '24 15:10 AmitMY