NeMo-text-processing

NeMo-text-processing copied to clipboard

Received warning message when normalizing text. Could you pls provide what the message indicates?

Reproduciple code:

from nemo_text_processing.text_normalization.normalize import Normalizer
text_normalizer = Normalizer(lang="zh", input_case="cased", overwrite_cache=True, cache_dir=str("cache_dir"))
text_normalizer_call_kwargs = {"punct_pre_process": True, "punct_post_process": True}
normalizer_call = lambda x: text_normalizer.normalize(x, **text_normalizer_call_kwargs)

text = "免除GOOGLE在一桩诽谤官司中的法律责任。"
normed_text = normalizer_call(text)
print(normed_text)

Output

NeMo-text-processing :: INFO     :: Created cache_dir[/zh_tn_True_deterministic__tokenize.far](http://localhost:8889/zh_tn_True_deterministic__tokenize.far)
INFO:NeMo-text-processing:Created cache_dir[/zh_tn_True_deterministic__tokenize.far](http://localhost:8889/zh_tn_True_deterministic__tokenize.far)
 NeMo-text-processing :: WARNING  :: Failed text: 免除GOOGLE在一桩诽谤官司中的法律责任。Key: integer_part Value: None
WARNING:NeMo-text-processing:Failed text: 免除GOOGLE在一桩诽谤官司中的法律责任。Key: integer_part Value: None
免除GOOGLE在一桩诽谤官司中的法律责任。

@BuyuanCui, could you please take a look?

Investigating.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

@BuyuanCui was this issue fixed?

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

This issue was closed because it has been inactive for 7 days since being marked as stale.