Error during step 2 of "02_prepare_labels.sh"

[HandsomeDevilv112]

While trying to make my own voice, I've managed to complete step 1 of "02_prepare_labels.sh" (after properly installing the other tools) But then during step 2, it fails automatically, and my sincere apologies, but I'm pretty out of my element with the error messaging.

Labels are ready in: /home/al/merlin/egs/build_your_own_voice/s1/label_no_align !! Step 2: forced-alignment using HTK tools... ---preparing enverionment ---checking data ---extracting features Traceback (most recent call last): File "forced_alignment.py", line 379, in aligner.prepare_training(file_id_list_name, wav_dir, lab_dir, work_dir, multiple_speaker) File "forced_alignment.py", line 253, in prepare_training self._HCopy() File "forced_alignment.py", line 190, in _HCopy check_call([HCopy, '-C', self.cfg, '-S', self.copy_scp]) File "/usr/lib/python2.7/subprocess.py", line 536, in check_call retcode = call(*popenargs, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 523, in call return Popen(*popenargs, **kwargs).wait() File "/usr/lib/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

[seblemaguer]

Hello,

he seems to not find a file. are you sure HCopy (the command) is available: HTK/HTS is installed ?

[HandsomeDevilv112]

Heya new friend, I ran the compile_htk.sh in merlin/tools, but it's very possible that I made an error during the process. I'll try running that again to see if it'll give me a different result

[HandsomeDevilv112]

it sure looks like it installed correctly at the right location, but it has not successfully been able to "make the donuts" How would you install htk?

[ronanki]

If you are facing troubles with HTK, then try other alternative:

• change the Labels=state_align to Labels=phone_align in conf/global_settings.cfg
• run both steps of 02_prepare_labels.sh

[HandsomeDevilv112]

I must be doing something wrong. Changing state_align to phone_align in global settings and running 02_prepare_labels.sh, I'm encountering the exact same error with references to forced_alignment.py from state_align. Just trying to run run_aligner.sh in phone_align runs the slt demo, and appears very tied to the demo. My apologies, but I'm feeling rather clueless

[r9y9]

OSError: [Errno 2] No such file or directory

I believe this is because HCopy cannot be found by the forced_alignment.py. Did you check that paths in https://github.com/CSTR-Edinburgh/merlin/blob/8ff864e22f6c2561b4877193c0145feb29471442/misc/scripts/alignment/state_align/forced_alignment.py#L19-L24 are correct?

[HandsomeDevilv112]

I hadn't, but it looks like the place to be... Right? HTKDIR = "/home/al/merlin/tools/htk" HCompV = os.path.join(HTKDIR, 'HCompV') HCopy = os.path.join(HTKDIR, 'HCopy' ) HERest = os.path.join(HTKDIR, 'HERest') HHEd = os.path.join(HTKDIR, 'HHEd' ) HVite = os.path.join(HTKDIR, 'HVite' )

[HandsomeDevilv112]

doing further research, I'm back to HTK is not properly installed. I found a reference saying "Hvite -V" should produce basic version information. Mine says "no command found". I'm not sure where my installation of HTK (through "tools/compile_htk.sh") failed, as it certainly didn't produce a "failed to build" message.

[r9y9]

https://github.com/CSTR-Edinburgh/merlin/blob/8ff864e22f6c2561b4877193c0145feb29471442/tools/compile_htk.sh#L52

Ah, I may find the problem. Can you try HTKDIR = "/home/al/merlin/tools/bin/htk" instead of HTKDIR = "/home/al/merlin/tools/htk"? Also, please make sure that you have htk binaries inside HTKDIR.

[HandsomeDevilv112]

New errors = Progress! Question: I don't see a folder labeled binaries, do you just mean the contents of the htk folder?

python forced_alignment.py ---preparing enverionment ---checking data ---extracting features Sat Sep 2 03:34:15 2017 ---feature_normalisation Sat Sep 2 03:34:36 2017 ---making proto Sat Sep 2 03:34:43 2017 ---training HMM models ERROR [+7390] StepAlpha: Alpha prune failed sq(3541) > qHi(3540) at time 81482 FATAL ERROR - Terminating program /home/al/merlin/tools/bin/htk/HERest Traceback (most recent call last): File "forced_alignment.py", line 381, in aligner.train_hmm(7, 32) File "forced_alignment.py", line 285, in train_hmm stdout=PIPE) File "/usr/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home/al/merlin/tools/bin/htk/HERest', '-C', '/home/al/merlin/egs/build_your_own_voice/s1/labels/config/cfg', '-S', '/home/al/merlin/egs/build_your_own_voice/s1/labels/config/train.scp', '-I', '/home/al/merlin/egs/build_your_own_voice/s1/labels/config/mono_phone.mlf', '-M', '/home/al/merlin/egs/build_your_own_voice/s1/labels/model/hmm_mix_1_iter_1', '-H', '/home/al/merlin/egs/build_your_own_voice/s1/labels/model/hmm0/macros', '-H', '/home/al/merlin/egs/build_your_own_voice/s1/labels/model/hmm0/hmmdefs', '-t', '250.0', '150.0', '2000.0', '/home/al/merlin/egs/build_your_own_voice/s1/labels/mono_phone.list']' returned non-zero exit status 222

[r9y9]

I meant check that htk binarires (i.e., HCompV, HCopy) actually exist.

[HandsomeDevilv112]

o~h. Okay. Thank you. Yes. merlin/tools/bin/htk has 16 items. All of them are HSomething.

[HandsomeDevilv112]

I'm going to go ahead and close this, as the problem was htk pathing related. I will enter a new report if I encounter a new problem. Thanks!

[r9y9]

The original problem hasn't been solved in Merlin. Probably needs fix for build_your_own_voice. I think we should keep the issue open.

[HandsomeDevilv112]

reopening per above

[abhigarg]

I got the ERROR [+2019] HCompV: Training data file name expected
while running "02_prepare_labels.sh" and I have checked all the paths seems correct. The error is thrown at check_call([HCompV, '-f', F, '-C', self.cfg, '-S', self.train_scp, '-M', self.cur_dir, self.proto]) One thing I noticed is that if I give my own path for labels_dir then it throws an error while executing "forced_alignment.py" that labels path not found because it messes up the path by adding path of merlin to the path I have already provided.

[abhigarg]

I copied the wave and text folders to the /merlin/egs/build_your_own_voice/s1/ directory and that seems to have resolved the issue

[abhigarg]

I also ended up with same error as @HandsomeDevilv112
ERROR [+7390] StepAlpha: Alpha prune failed sq(9) > qHi(8) at time 60116 FATAL ERROR - Terminating program ../merlin/tools/bin/htk/HERest Could it be due to some config parameter values?

[abhigarg]

Found some hints to resolve this issue:
https://firojalam.wordpress.com/2014/01/23/herest-error-7390-stepalpha-alpha-prune-failed/ https://stackoverflow.com/questions/5463645/htk-herest-issue

[HandsomeDevilv112]

My apologies for not following up. Yeah, I got past the alpha prune error by turning up the targetrate as suggested by the firojalam wordpress. I forget if I doubled the default rate, or added an extra zero. I've kinda experienced an unrelated issue that resulted in me having to reformat, so I don't have the information handy, but if memory serves, the next issue I faced was that the files needed to be in mono configuration. This can be solved easily with audacity.

[abhigarg]

I am stuck at hmm training step since more than half a day now. Any ideas how can I get the status of the training like usually some kind of error rates or accuracy values. Its not showing me anything to know the status of the training.

On Mon 18 Sep, 2017, 08:18 HandsomeDevilv112 [email protected] wrote:

My apologies for not following up. Yeah, I got past the alpha prune error by turning up the targetrate as suggested by the firojalam wordpress. I forget if I doubled the default rate, or added an extra zero. I've kinda experienced an unrelated issue that resulted in me having to reformat, so I don't have the information handy, but if memory serves, the next issue I faced was that the files needed to be in mono configuration. This can be solved easily with audacity.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/CSTR-Edinburgh/merlin/issues/223#issuecomment-330116737, or mute the thread https://github.com/notifications/unsubscribe-auth/ACiH_J10SYxcC6HUYY9LQOgO8TekDYx-ks5sjdntgaJpZM4PJvnz .

[HandsomeDevilv112]

I remember having a similar thought. I forget the last message printed, but mine eventually finished somewhere in the "half a day" period. I do agree that some additional messaging wouldn't go unappreciated.

[ZofiaMalisz]

I encountered an identical error as the original poster while trying to build a new voice.

I checked the paths (forced_alignment.py, global_settings.cfg), checked the installation of htk (provided compile_htk.sh was not successful, I copied a functional installation into tools/htk), to no avail.

Does it need a fix in Merlin or does anybody have a hint what I could try? Thanks.

[HandsomeDevilv112]

just to ask, did you do the whole login/password dance to install HTK? It's a bit of a pain.

[raghdr]

Just wanted to note that I got stuck at hmm training step too. No progress after the log message: Step 2: forced-alignment using HTK tools... but the process was running for more than a day.

I am using Python 2.7 and the subprocess.check_call documentation says "Do not use stdout=PIPE or stderr=PIPE with this function as that can deadlock based on the child process output volume". But the train_hmm() function in forced_alignment.py runs HERest using check_call with stdout=PIPE It progressed for me after explicitly opening a file and changing stdout=stdoutlog.

[Sangramsingkayte]

I need some help in this error Step 2: Preparing labels... mkdir: cmu_us_IIIT: File exists Copying labels to duration and acoustic data directories... sed: 1: "experiments/IIIT/durati ...": invalid command code e sed: 1: "experiments/IIIT/acoust ...": invalid command code e done...!

[raghdr]

@Sangramsingkayte

1. Are the labels correctly created in your $lab_dir? I guess not since your log does not print "You should have your labels ready in: <>". Maybe you could try deleting cmu_us_IIIT and retry?

(These label filenames are written to $FileIDList and the sed command just removes the .lab extension, you could go through the 02_prepare_labels and run_aligner scripts line by line)

2. If (1) is not the case, are you running on Mac OS? It has a lot of idiosyncrasies when compared to Unix/Linux (https://markhneedham.com/blog/2011/01/14/sed-sed-1-invalid-command-code-r-on-mac-os-x/)

[Sangramsingkayte]

DrSangramsings-MacBook-Air:s1 sangramsing$ bash ./02_prepare_labels.sh ./database/wav ./database/txt.txt ./database/labels Step 2: Preparing labels... mkdir: cmu_us_sing_iiit: File exists Copying labels to duration and acoustic data directories... sed: 1: "experiments/sing_iiit/d ...": invalid command code e sed: 1: "experiments/sing_iiit/a ...": invalid command code e done...!

[mirfan899]

got the same issue related to hmm training. Here is the output of error

95.lab
96.lab
97.lab
98.lab
99.lab
forced-alignment using HTK tools...
HTK step 1 success
HTK step 2 success
HTK step 3 success
---preparing enverionment
---checking data
---extracting features
Fri Jun  7 11:16:18 2019
---feature_normalisation
Fri Jun  7 11:16:25 2019
---making proto
Fri Jun  7 11:16:25 2019
---training HMM models
Traceback (most recent call last):
  File "/home/virtuoso_irfan/merlin/misc/scripts/alignment/state_align/forced_alignment.py", line 381, in <module>
    aligner.train_hmm(7, 32)
  File "/home/virtuoso_irfan/merlin/misc/scripts/alignment/state_align/forced_alignment.py", line 285, in train_hmm
    stdout=PIPE)
  File "/root/anaconda3/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/virtuoso_irfan/merlin/tools/bin/htk/HERest', '-C', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/config/cfg', '-S', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/config/train.scp', '-I', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/config/mono_phone.mlf', '-M', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/model/hmm_mix_1_iter_1', '-H', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/model/hmm0/macros', '-H', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/model/hmm0/hmmdefs', '-t', '250.0', '150.0', '2000.0', '/home/virtuoso_irfan/merlin/egs/build_your_own_voice/s1/database/labels/mono_phone.list']' returned non-zero exit status -11
Force-alignment unsucessful!! Please check /home/virtuoso_irfan/merlin/misc/scripts/alignment/state_align/forced_alignment.py
Copying labels to duration and acoustic data directories...
done...!

[RasmusD]

Hi @mirfan899 If you try and run forced_alignment.py by itself you can try and debug from the output of that. It'll give you more helpful errors.