big_vision
big_vision copied to clipboard
Text lowering issue
I found an issue here https://github.com/google-research/big_vision/blob/main/big_vision/pp/ops_text.py#L165
When lowering UTF-8 non-latin text encoding ='utf-8'
should be used as mentioned here https://www.tensorflow.org/api_docs/python/tf/strings/lower .
This at least can influence at i18n model. But due to models already trained, i'm not sure if this issue should be fixed.