big_vision icon indicating copy to clipboard operation
big_vision copied to clipboard

Text lowering issue

Open shkarupa-alex opened this issue 8 months ago • 0 comments

I found an issue here https://github.com/google-research/big_vision/blob/main/big_vision/pp/ops_text.py#L165 When lowering UTF-8 non-latin text encoding ='utf-8' should be used as mentioned here https://www.tensorflow.org/api_docs/python/tf/strings/lower .

This at least can influence at i18n model. But due to models already trained, i'm not sure if this issue should be fixed.

shkarupa-alex avatar Nov 27 '23 05:11 shkarupa-alex