spoken-to-signed-translation
spoken-to-signed-translation copied to clipboard
Fingerspelling
I would love to add fingerspelling to this repo.
Basic:
- [x] Collect fingerspelling data for various signed languages. One video per letter
- [x] Upload, process, pose estimate etc.
- [x] Curate a fingerspelling list per language. For example, "SCH" is "one letter" in swiss-german
- [x] Implement a fallback strategy to combine the letters
Advanced
Combining letters naively is a little silly, as the transitions aren't necessarily "human".
- [ ] For repeating letters, in supported languages (for example, no idea what they do in BSL), move the hand slightly, then, return to original position
- [ ] Allow specifying letter transitions: annotated as "SCH-B" for example, is the transition between "SCH" and "B". "B-∅" is the transition between "B" and nothing.
- [ ] Combine letters based on their transition (SCHULE = ∅-SCH + SCH-U + U-L + L-E + E-∅) - https://fingerspell.net/#/spell