Korean Phonemizer

[harmlessman]

This PR implements Korean phonemizer that using Hangeul.

You can use it by adding Korean to the character. More information can be found in ko_kr_phonemizer.py.

[CLAassistant]

All committers have signed the CLA.

[harmlessman]

I added only Korean phonemizer to the phonemizer. The error is Error: Bad zip file, is there a problem with the sauce I added? i just sync fork this project and add korean phonemizer.

[p0p4k]

Hi, install black, pylint and run it for correcting code format. That will fix style tests. For zoo-tests, sometimes the server will not respond, so it gives badzip error. Just run the test again and it will fix.

[harmlessman]

Okay. i will try.

[p0p4k]

"B": "씨",
"PPropertyPositionPowerPrisonP" "S": "에스",

이개 안맞아요. dictionary를 촘 도 확인을 해야 될것같습니다.

[harmlessman]

Run make lint pylint tests TTS notebooks recipes

---

Your code has been rated at 10.00/10

black tests TTS notebooks recipes --check Skipping .ipynb files as Jupyter dependencies are not installed. You can fix this by running pip install black[jupyter] All done! ✨ 🍰 ✨ 304 files would be left unchanged. isort tests TTS notebooks recipes --check-only ERROR: /home/runner/work/TTS/TTS/TTS/tts/utils/text/korean/korean.py Imports are incorrectly sorted and/or formatted. ERROR: /home/runner/work/TTS/TTS/TTS/tts/utils/text/korean/phonemizer.py Imports are incorrectly sorted and/or formatted. ERROR: /home/runner/work/TTS/TTS/TTS/tts/utils/text/phonemizers/ko_kr_phonemizer.py Imports are incorrectly sorted and/or formatted. ERROR: /home/runner/work/TTS/TTS/TTS/tts/utils/text/phonemizers/init.py Imports are incorrectly sorted and/or formatted. make: *** [Makefile:48: lint] Error 1 Error: Process completed with exit code 2.

I got this problem in style test. How can I solve this problem?

[p0p4k]

1. Install isort on your local system. pip install isort
2. run isort on the file isort {source_file_or_directory} In our case, isort <path_to_korean.py> and so on. isort reference

[harmlessman]

thank you! I will try it.

[p0p4k]

thank you! I will try it.

Ok, I think all checks are successful too. Now maybe, if possible, we write some test cases for verification, so that this does not break in future versions.

[erogol]

Thanks for the ✨PR✨. Before merging these, we need some testing. You can check here for examples.

[harmlessman]

Should I make a test_korean_phonemizer.py for TTS/tests/text_tests and upload it again?

[p0p4k]

Hello, while I was making test cases for this PR, I noticed that it is not doing "phonemization" yet. It is just splitting sentences into basic characters or converting numbers to text (without context). I would suggest to switch to a better library built on top of jamo , called g2pk. See -> https://github.com/Kyubyong/g2pK We want words like "신림" -> "실림" (see picture) Also, numbers must convert into their respective Korean context. eg. In the sentence, "지금 시각은 12시 12분입니다", 12 is pronounced 열두, whereas the second 12 is pronounced 십이. Another issue is g2pk, requires python-mecab-ko which does not install easily on Windows (need to build from source). Also, jamo uses nltk library and we should add it to requirements.txt. So, I think you might want to look into this first and let us improve this PR. Thanks.

TODO (08/11/2022):

• [add g2pk support]
• [fix requirements.txt]
• [write test cases]

[harmlessman]

Thank you for your feedback. I will reflect the feedback and upload it again.

[erogol]

Sorry for the late reply. Yes adding test_korean_phonemizer.py would be nice.

[harmlessman]

TODO (08/11/2022):

• [add g2pk support]
• [fix requirements.txt]
• [write test cases]

Hello, we are supplementing the code based on the contents of TODO. There is no problem with the 2nd and 3rd, but the problem occurs with the g2pk window compatibility of the 1st.

<<Another issue is g2pk, requires python-mecab-ko which does not install easily on Windows (need to build from source).>>

I looked up how to install g2pk in the Windosw for about a week. (Easy to install on linux)

1. Use eunjeon, Korean-mecab that is compatible with Windows -> not installed in linux.
2. Use konlpy's other morpheme analyzer instead of python-mecab-ko in the g2pk source -> g2pk should be put into the TTS folder, but this increases the capacity of TTS.
3. Make python-mecab-ko compatible in Windows -> I can't find a way.

How should we solve this problem? as far as I'm concerned Referring to espake's method, I am thinking about using g2pk if g2pk is installed, then using jamo, and using jamo if it is not installed.

Is there no problem if I configure the source in the above method?

. . .

As a result of checking, the number will be converted to Korean context. :)

[p0p4k]

TODO (08/11/2022):

• [add g2pk support]
• [fix requirements.txt]
• [write test cases]

Hello, we are supplementing the code based on the contents of TODO. There is no problem with the 2nd and 3rd, but the problem occurs with the g2pk window compatibility of the 1st.

<<Another issue is g2pk, requires python-mecab-ko which does not install easily on Windows (need to build from source).>>

I looked up how to install g2pk in the Windosw for about a week. (Easy to install on linux)

1. Use eunjeon, Korean-mecab that is compatible with Windows -> not installed in linux.
2. Use konlpy's other morpheme analyzer instead of python-mecab-ko in the g2pk source -> g2pk should be put into the TTS folder, but this increases the capacity of TTS.
3. Make python-mecab-ko compatible in Windows -> I can't find a way.

How should we solve this problem? as far as I'm concerned Referring to espake's method, I am thinking about using g2pk if g2pk is installed, then using jamo, and using jamo if it is not installed.

Is there no problem if I configure the source in the above method?

. . .

As a result of checking, the number will be converted to Korean context. :)

Okay. Let me look into this matter and let you know my thoughts next week. Thanks for the detailed information.

[harmlessman]

Oh, thank you. I will also consider whether there is a solution.

[erogol]

Are these libraries interchangeable (producing compatible outputs for the same input text)? If not just using the installed lib would probably break models that are trained with a different backend.

Then I'd suggest having different Korean phonemizer for each backend and letting the user define which in the config explicitly. We can raise an error when the required backend is not installed.

[p0p4k]

Is it ok to add sys_platform == "linux" in requirements.txt for this PR? Also, we can then add a tip that Korean phonemizers are supported only in Linux systems.

[harmlessman]

Thanks for your replies!

With reference to both of you, I was going to make sure that the koean-phonemizer only works on linux. However, I'm mostly working in a windows , and it was kind of funny that I couldn't use what I made. So I thought about it and found a solution!

The reason why the G2pk library is not built in the Windows is because of the Korean Morphology Analyzer called python-mecab-ko. Instead of python-mecab-ko, I modified the source to use a different morpheme analyzer that can be built in windows and linux, and it works well after running it. (Konlpy's Morpheme Analyzer)

I will check more about this and reply when it is confirmed.

(What I'm thinking about right now is distributing the source that changed the morpheme analyzer from the source of g2pk to the library, or getting the necessary part of the source of g2pk and putting it in the Korean folder.)

[p0p4k]

https://wonhwa.tistory.com/49 Does this work on windows? @harmlessman

[harmlessman]

I tried that method before, but it was possible to execute mecab. However, I did not choose this method because the installation process was very cumbersome. (And the same error occurred when I installed mecab of the link because of python-mecab-ko in install_requires of setup.py in g2pk.)

[erogol]

Is it ok to add sys_platform == "linux" in requirements.txt for this PR? Also, we can then add a tip that Korean phonemizers are supported only in Linux systems.

This is not a good idea as it relies on the users pre-knowledge. But people don't read :). It is better to handle it in the code.

[erogol]

@harmlessman In the worse case maybe you can try Win WSL

[harmlessman]

@erogol I will refer to it when testing source on Linux :).

[harmlessman]

I made a cross-platform G2p for Korean! I will upload a phonemizer with g2p with test_korean_phonemizer.py soon. :)

[harmlessman]

I'm done!

[p0p4k]

I think the PR is good to go!

[harmlessman]

The above articles seem to be the code before pull today. Is it a review of the codes before I pull?

[harmlessman]

There are no words in the code posted today on dictionary.py that have been corrected, and 'TTS/ts/utils/text/korean/korean.py' The function get_mode of also does not exist. The codes you kindly reviewed above seem to be the old codes.

Can you look at the code that I pulled today?