kaldi
kaldi copied to clipboard
kaldi-asr
WIP: Speed up of compute-wer
Changed LevenshteinEditDistance to use algorithm which much faster than O(NM) while still using linear amount of memory. I needed this change because I was scoring large chunks of text (thousands of phonems). The speed up was ~4x. The idea behind the algorithm is as follows: For each diagonal of the NxM matrix I maintain at most single "particle". Initially there is only a single particle located in (0,0). All particles are in the same edit distance from (0,0) - initially 0. Each round progresses them one unit ("error") further. The number of rounds equals to edit distance. Each particle can go along the diagonal at zero cost if letters of both strings match. It can also move one to neigboring cell (right or down or diagonal right-down) by paying +1 cost.
seems reasonable. Please run doc/cpplint.py on the file to find additional style errors.
Not to nitpick, but I'm somewhat sceptical about this algorithm being faster than O(MN), because it seems that would be a major algorithmic breakthrough ..
Also, while I'm not sure I completely understand the algorithm I ran a quick test to compare the old and the new implementation. The total distance calculated by the new code is OK, but it has an overly strong preference for substitutions:
./lev <(echo 6 6 6 5 2 1 1 1 ) <(echo 5 2 3 3 1 1 1)
5 (ins: 0; del: 1; sub: 4)
Old:
5 (ins: 2; del: 3; sub: 0)
Maybe that's fixable, but I don't want to spend more time on this at the moment.
Hm. I would have thought that even if the general case can't be sped up, the normal case (in which the strings are quite similar) could be sped up substantially. Obviously the submitted code is incorrect, or it would give the same result. It's disappointing that this wasn't caught by the automated testing. Dan
On Wed, Apr 27, 2016 at 6:08 AM, Vassil Panayotov [email protected] wrote:
Not to nitpick, but I'm somewhat sceptical about this algorithm being faster than O(MN), because it seems that would be a major algorithmic breakthrough https://en.wikipedia.org/wiki/Levenshtein_distance#Computational_hardness ..
Also, while I'm not sure I completely understand the algorithm I ran a quick test to compare the old and the new implementation. The total distance calculated by the new code is OK, but it has an overly strong preference for substitutions:
./lev <(echo 6 6 6 5 2 1 1 1 ) <(echo 5 2 3 3 1 1 1) 5 (ins: 0; del: 1; sub: 4)
Old: 5 (ins: 2; del: 3; sub: 0)
Maybe that's fixable, but I don't want to spend more time on this at the moment.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215038932
Vassil, i just noticed that in your example the total number of errors is the same, so which is correct is actually undefined (since they both give the same edit distance). Actually I'd tend to prefer an algorithm that favored substitutions over insertions and deletions, since such output is easier to display. Dan
On Wed, Apr 27, 2016 at 1:10 PM, Daniel Povey [email protected] wrote:
Hm. I would have thought that even if the general case can't be sped up, the normal case (in which the strings are quite similar) could be sped up substantially. Obviously the submitted code is incorrect, or it would give the same result. It's disappointing that this wasn't caught by the automated testing. Dan
On Wed, Apr 27, 2016 at 6:08 AM, Vassil Panayotov < [email protected]> wrote:
Not to nitpick, but I'm somewhat sceptical about this algorithm being faster than O(MN), because it seems that would be a major algorithmic breakthrough https://en.wikipedia.org/wiki/Levenshtein_distance#Computational_hardness ..
Also, while I'm not sure I completely understand the algorithm I ran a quick test to compare the old and the new implementation. The total distance calculated by the new code is OK, but it has an overly strong preference for substitutions:
./lev <(echo 6 6 6 5 2 1 1 1 ) <(echo 5 2 3 3 1 1 1) 5 (ins: 0; del: 1; sub: 4)
Old: 5 (ins: 2; del: 3; sub: 0)
Maybe that's fixable, but I don't want to spend more time on this at the moment.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215038932
Thanks for review! Sorry for a late reply, I was a bit busy. As already pointed out, I was careless when writing "much faster than O(NM)". On the plus side, at least I did not write "o(NM)" :) What I had in mind was, that my solution is O(edit_distance(a,b)*(|a|+|b|)) which tends to be smaller than O(|a|+|b|) when the difference is small (which is to be expected). I must admit that in case where WER is a lower-bounded by constant, one should expect edit_distance(a,b) to be Omega(|a|+|b|) and thus my solution is Theta((|a|+|b|)^2) which is not impressive at all. But...in practice: we have WER ~ 30% in our project, and the my algorithm is still 4x faster than the older, which is important for me, as verification took several hours before my patch.
My algorithm is not really mine - it's a simplification of O(edit_distance(a,b)^2) algorithm, which is explained during String Algorithms lecture (I forgot the author, perhaps Ukkonen?) - the more complicated version performs the "diagonal scan" in O(1) by using LCA in a Suffix Tree if I recall correctly, which seems discouraging to me, so I've chosen a simple while loop which is one of the reasons why my algorithm has (|a|+|b|) factor in the complexity.
The other reason is that, for clarity I've used vector assignment to copy diagonal into future_diagonal and back - this takes Theta(|a|+|b|) but could be optimized to O(edit_distance(a,b)) by manually copying only the entries in use.
There are at most edit_distance(a,b) iterations of the outer loop.
Each iteration of the main loop ads at most two new active diagonals, so there are at most O(edit_distance(a,b)) entries in use.
The other reason for not copying entries manually was that would probably cause more random accesses, than simply coping vectors.
Probably the best of both worlds would be to use swap(diagonal,future_diagonal) which is O(1) and elegant.
I'll fix that.
As for the code style - you are right - I wasn't aware that there is a tool for checking it. I'll commit all the fixes as soon as get my Cygwin to compile this project (no luck so far) - hopefully by the end of the week.
As for the discrepancies between numbers of del,ins,sub... I'm not sure what is the supposed "good" answer here - the problem is underspecified, the name of the function and comments in the code do not clarify what should be optimized except for the number of errors, so I didn't think much about which path in the edit graph to favor. The way the code is written now, indeed favors subs a bit (locally), as this is the first case being considered and ties are broken by "the first path to reach a node wins", but I could not prove that this local property of the algorithm generalizes to the whole execution, that is: I'm not sure if my algorithm always returns not-less subs than the old one. But does it really matter? I could not find any place in the project that really depends on this aspect of the function - it's used in the stdout report, but if somebody based their decisions on this particular part of the output then they might have a larger trouble, as this is an unspecified detail. If we really want to optimize for subs, or dels, or ins, then perhaps they should have different weights, or we should perform a more complicated algorithm which uses additional dimension for number of subs O(NM*edit_distance(N,M)) - I'm not aware of a simpler solution :(
I'm all in favor for not doing premature optimizations, but this one was not that much premature in my case.
Since we are looking at this, I'd say it would be more ideal to have the algorithm favor substitutions 'if it's all the same', e.g. have a cost pair (num_errs, num_ins_del_errs) and use lexicographical comparison. This is probably usually what we want.
Dan
On Wed, Apr 27, 2016 at 4:04 PM, qbolec [email protected] wrote:
Thanks for review! Sorry for a late reply, I was a bit busy. As already pointed out, I was careless when writing "much faster than O(NM)". On the plus side, at least I did not write "o(NM)" :) What I had in mind was, that my solution is O(edit_distance(a,b)*(|a|+|b|)) which tends to be smaller than O(|a|+|b|) when the difference is small (which is to be expected). I must admit that in case where WER is a lower-bounded by constant, one should expect edit_distance(a,b) to be Omega(|a|+|b|) and thus my solution is Theta((|a|+|b|)^2) which is not impressive at all. But...in practice: we have WER ~ 30% in our project, and the my algorithm is still 4x faster than the older, which is important for me, as verification took several hours before my patch.
My algorithm is not really mine - it's a simplification of O(edit_distance(a,b)^2) algorithm, which is explained during String Algorithms lecture (I forgot the author, perhaps Ukkonen?) - the more complicated version performs the "diagonal scan" in O(1) by using LCA in a Suffix Tree if I recall correctly, which seems discouraging to me, so I've chosen a simple while loop which is one of the reasons why my algorithm has (|a|+|b|) factor in the complexity. The other reason is that, for clarity I've used vector assignment to copy diagonal into future_diagonal and back - this takes Theta(|a|+|b|) but could be optimized to O(edit_distance(a,b)) by manually copying only the entries in use. There are at most edit_distance(a,b) iterations of the outer loop. Each iteration of the main loop ads at most two new active diagonals, so there are at most O(edit_distance(a,b)) entries in use. The other reason for not copying entries manually was that would probably cause more random accesses, than simply coping vectors. Probably the best of both worlds would be to use swap(diagonal,future_diagonal) which is O(1) and elegant. I'll fix that.
As for the code style - you are right - I wasn't aware that there is a tool for checking it. I'll commit all the fixes as soon as get my Cygwin to compile this project (no luck so far) - hopefully by the end of the week.
As for the discrepancies between numbers of del,ins,sub... I'm not sure what is the supposed "good" answer here - the problem is underspecified, the name of the function and comments in the code do not clarify what should be optimized except for the number of errors, so I didn't think much about which path in the edit graph to favor. The way the code is written now, indeed favors subs a bit (locally), as this is the first case being considered and ties are broken by "the first path to reach a node wins", but I could not prove that this local property of the algorithm generalizes to the whole execution, that is: I'm not sure if my algorithm always returns not-less subs than the old one. But does it really matter? I could not find any place in the project that really depends on this aspect of the function - it's used in the stdout report, but if somebody based their decisions on this particular part of the output then they might have a larger trouble, as this is an unspecified detail. If we really want to optimize for subs, or dels, or ins, then perhaps they should have different weights, or we should perform a more complicated algorithm which uses additional dimension for number of subs O(NM*edit_distance(N,M)) - I'm not aware of a simpler solution :(
I'm all in favor for not doing premature optimizations, but this one was not that much premature in my case.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215211318
In my opinion preference for substitutions is a plus. I have some real life scenarios where I would need to create an alignment, and from the nature of the date we know that there would be a region where accuracy will be very low, and another region where accuracy will be very high. I was wanting to have a few different alignment algorithms (step before WER), so that I could compare. It would be nice if both algorithms could co-exist and be available.
On Wed, Apr 27, 2016 at 4:17 PM, Daniel Povey [email protected] wrote:
Since we are looking at this, I'd say it would be more ideal to have the algorithm favor substitutions 'if it's all the same', e.g. have a cost pair (num_errs, num_ins_del_errs) and use lexicographical comparison. This is probably usually what we want.
Dan
On Wed, Apr 27, 2016 at 4:04 PM, qbolec [email protected] wrote:
Thanks for review! Sorry for a late reply, I was a bit busy. As already pointed out, I was careless when writing "much faster than O(NM)". On the plus side, at least I did not write "o(NM)" :) What I had in mind was, that my solution is O(edit_distance(a,b)*(|a|+|b|)) which tends to be smaller than O(|a|+|b|) when the difference is small (which is to be expected). I must admit that in case where WER is a lower-bounded by constant, one should expect edit_distance(a,b) to be Omega(|a|+|b|) and thus my solution is Theta((|a|+|b|)^2) which is not impressive at all. But...in practice: we have WER ~ 30% in our project, and the my algorithm is still 4x faster than the older, which is important for me, as verification took several hours before my patch.
My algorithm is not really mine - it's a simplification of O(edit_distance(a,b)^2) algorithm, which is explained during String Algorithms lecture (I forgot the author, perhaps Ukkonen?) - the more complicated version performs the "diagonal scan" in O(1) by using LCA in a Suffix Tree if I recall correctly, which seems discouraging to me, so I've chosen a simple while loop which is one of the reasons why my algorithm has (|a|+|b|) factor in the complexity. The other reason is that, for clarity I've used vector assignment to copy diagonal into future_diagonal and back - this takes Theta(|a|+|b|) but could be optimized to O(edit_distance(a,b)) by manually copying only the entries in use. There are at most edit_distance(a,b) iterations of the outer loop. Each iteration of the main loop ads at most two new active diagonals, so there are at most O(edit_distance(a,b)) entries in use. The other reason for not copying entries manually was that would probably cause more random accesses, than simply coping vectors. Probably the best of both worlds would be to use swap(diagonal,future_diagonal) which is O(1) and elegant. I'll fix that.
As for the code style - you are right - I wasn't aware that there is a tool for checking it. I'll commit all the fixes as soon as get my Cygwin to compile this project (no luck so far) - hopefully by the end of the week.
As for the discrepancies between numbers of del,ins,sub... I'm not sure what is the supposed "good" answer here - the problem is underspecified, the name of the function and comments in the code do not clarify what should be optimized except for the number of errors, so I didn't think much about which path in the edit graph to favor. The way the code is written now, indeed favors subs a bit (locally), as this is the first case being considered and ties are broken by "the first path to reach a node wins", but I could not prove that this local property of the algorithm generalizes to the whole execution, that is: I'm not sure if my algorithm always returns not-less subs than the old one. But does it really matter? I could not find any place in the project that really depends on this aspect of the function - it's used in the stdout report, but if somebody based their decisions on this particular part of the output then they might have a larger trouble, as this is an unspecified detail. If we really want to optimize for subs, or dels, or ins, then perhaps they should have different weights, or we should perform a more complicated algorithm which uses additional dimension for number of subs O(NM*edit_distance(N,M)) - I'm not aware of a simpler solution :(
I'm all in favor for not doing premature optimizations, but this one was not that much premature in my case.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215211318
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215214938
Sorry Nagendra, I don't think it's super practical to have 2 different versions of Levenshtein alignment in the Kaldi codebase. But we can make sure it prefers substitutions, all other things being equal.
On Wed, Apr 27, 2016 at 8:57 PM, Nagendra Goel [email protected] wrote:
In my opinion preference for substitutions is a plus. I have some real life scenarios where I would need to create an alignment, and from the nature of the date we know that there would be a region where accuracy will be very low, and another region where accuracy will be very high. I was wanting to have a few different alignment algorithms (step before WER), so that I could compare. It would be nice if both algorithms could co-exist and be available.
On Wed, Apr 27, 2016 at 4:17 PM, Daniel Povey [email protected] wrote:
Since we are looking at this, I'd say it would be more ideal to have the algorithm favor substitutions 'if it's all the same', e.g. have a cost pair (num_errs, num_ins_del_errs) and use lexicographical comparison. This is probably usually what we want.
Dan
On Wed, Apr 27, 2016 at 4:04 PM, qbolec [email protected] wrote:
Thanks for review! Sorry for a late reply, I was a bit busy. As already pointed out, I was careless when writing "much faster than O(NM)". On the plus side, at least I did not write "o(NM)" :) What I had in mind was, that my solution is O(edit_distance(a,b)*(|a|+|b|)) which tends to be smaller than O(|a|+|b|) when the difference is small (which is to be expected). I must admit that in case where WER is a lower-bounded by constant, one should expect edit_distance(a,b) to be Omega(|a|+|b|) and thus my solution is Theta((|a|+|b|)^2) which is not impressive at all. But...in practice: we have WER ~ 30% in our project, and the my algorithm is still 4x faster than the older, which is important for me, as verification took several hours before my patch.
My algorithm is not really mine - it's a simplification of O(edit_distance(a,b)^2) algorithm, which is explained during String Algorithms lecture (I forgot the author, perhaps Ukkonen?) - the more complicated version performs the "diagonal scan" in O(1) by using LCA in a Suffix Tree if I recall correctly, which seems discouraging to me, so I've chosen a simple while loop which is one of the reasons why my algorithm has (|a|+|b|) factor in the complexity. The other reason is that, for clarity I've used vector assignment to copy diagonal into future_diagonal and back - this takes Theta(|a|+|b|) but could be optimized to O(edit_distance(a,b)) by manually copying only the entries in use. There are at most edit_distance(a,b) iterations of the outer loop. Each iteration of the main loop ads at most two new active diagonals, so there are at most O(edit_distance(a,b)) entries in use. The other reason for not copying entries manually was that would probably cause more random accesses, than simply coping vectors. Probably the best of both worlds would be to use swap(diagonal,future_diagonal) which is O(1) and elegant. I'll fix that.
As for the code style - you are right - I wasn't aware that there is a tool for checking it. I'll commit all the fixes as soon as get my Cygwin to compile this project (no luck so far) - hopefully by the end of the week.
As for the discrepancies between numbers of del,ins,sub... I'm not sure what is the supposed "good" answer here - the problem is underspecified, the name of the function and comments in the code do not clarify what should be optimized except for the number of errors, so I didn't think much about which path in the edit graph to favor. The way the code is written now, indeed favors subs a bit (locally), as this is the first case being considered and ties are broken by "the first path to reach a node wins", but I could not prove that this local property of the algorithm generalizes to the whole execution, that is: I'm not sure if my algorithm always returns not-less subs than the old one. But does it really matter? I could not find any place in the project that really depends on this aspect of the function - it's used in the stdout report, but if somebody based their decisions on this particular part of the output then they might have a larger trouble, as this is an unspecified detail. If we really want to optimize for subs, or dels, or ins, then perhaps they should have different weights, or we should perform a more complicated algorithm which uses additional dimension for number of subs O(NM*edit_distance(N,M)) - I'm not aware of a simpler solution :(
I'm all in favor for not doing premature optimizations, but this one was not that much premature in my case.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215211318
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215214938
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215276836
Kaldi supports sever different acoustic models and that part is super nice.
On Wed, Apr 27, 2016 at 8:59 PM, Daniel Povey [email protected] wrote:
Sorry Nagendra, I don't think it's super practical to have 2 different versions of Levenshtein alignment in the Kaldi codebase. But we can make sure it prefers substitutions, all other things being equal.
On Wed, Apr 27, 2016 at 8:57 PM, Nagendra Goel [email protected] wrote:
In my opinion preference for substitutions is a plus. I have some real life scenarios where I would need to create an alignment, and from the nature of the date we know that there would be a region where accuracy will be very low, and another region where accuracy will be very high. I was wanting to have a few different alignment algorithms (step before WER), so that I could compare. It would be nice if both algorithms could co-exist and be available.
On Wed, Apr 27, 2016 at 4:17 PM, Daniel Povey [email protected] wrote:
Since we are looking at this, I'd say it would be more ideal to have the algorithm favor substitutions 'if it's all the same', e.g. have a cost pair (num_errs, num_ins_del_errs) and use lexicographical comparison. This is probably usually what we want.
Dan
On Wed, Apr 27, 2016 at 4:04 PM, qbolec [email protected] wrote:
Thanks for review! Sorry for a late reply, I was a bit busy. As already pointed out, I was careless when writing "much faster than O(NM)". On the plus side, at least I did not write "o(NM)" :) What I had in mind was, that my solution is O(edit_distance(a,b)*(|a|+|b|)) which tends to be smaller than O(|a|+|b|) when the difference is small (which is to be expected). I must admit that in case where WER is a lower-bounded by constant, one should expect edit_distance(a,b) to be Omega(|a|+|b|) and thus my solution is Theta((|a|+|b|)^2) which is not impressive at all. But...in practice: we have WER ~ 30% in our project, and the my algorithm is still 4x faster than the older, which is important for me, as verification took several hours before my patch.
My algorithm is not really mine - it's a simplification of O(edit_distance(a,b)^2) algorithm, which is explained during String Algorithms lecture (I forgot the author, perhaps Ukkonen?) - the more complicated version performs the "diagonal scan" in O(1) by using LCA in a Suffix Tree if I recall correctly, which seems discouraging to me, so I've chosen a simple while loop which is one of the reasons why my algorithm has (|a|+|b|) factor in the complexity. The other reason is that, for clarity I've used vector assignment to copy diagonal into future_diagonal and back - this takes Theta(|a|+|b|) but could be optimized to O(edit_distance(a,b)) by manually copying only the entries in use. There are at most edit_distance(a,b) iterations of the outer loop. Each iteration of the main loop ads at most two new active diagonals, so there are at most O(edit_distance(a,b)) entries in use. The other reason for not copying entries manually was that would probably cause more random accesses, than simply coping vectors. Probably the best of both worlds would be to use swap(diagonal,future_diagonal) which is O(1) and elegant. I'll fix that.
As for the code style - you are right - I wasn't aware that there is a tool for checking it. I'll commit all the fixes as soon as get my Cygwin to compile this project (no luck so far) - hopefully by the end of the week.
As for the discrepancies between numbers of del,ins,sub... I'm not sure what is the supposed "good" answer here - the problem is underspecified, the name of the function and comments in the code do not clarify what should be optimized except for the number of errors, so I didn't think much about which path in the edit graph to favor. The way the code is written now, indeed favors subs a bit (locally), as this is the first case being considered and ties are broken by "the first path to reach a node wins", but I could not prove that this local property of the algorithm generalizes to the whole execution, that is: I'm not sure if my algorithm always returns not-less subs than the old one. But does it really matter? I could not find any place in the project that really depends on this aspect of the function - it's used in the stdout report, but if somebody based their decisions on this particular part of the output then they might have a larger trouble, as this is an unspecified detail. If we really want to optimize for subs, or dels, or ins, then perhaps they should have different weights, or we should perform a more complicated algorithm which uses additional dimension for number of subs O(NM*edit_distance(N,M)) - I'm not aware of a simpler solution :(
I'm all in favor for not doing premature optimizations, but this one was not that much premature in my case.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215211318
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215214938
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215276836
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-215277082
I didn't have chance to work on it. I want to get it done today. The only difficulty for me is to get Kaldi to compile with Cygwin. So far, I found this hint https://sourceforge.net/p/kaldi/discussion/1355349/thread/7e088b0b/ helpful. I still get bazillion warnings about
/home/qbolec/kaldi/kaldi/tools/openfst/include/fst/compat.h:70:16: warning: typedef ‘VerifySizesAreEqual’ locally defined but not used [-Wunused-local-typedefs]
and similar (Label, StateId, RevWeight, Weight), and errors like:
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836)
/tmp/ccTggEkL.s: Assembler messages:
/tmp/ccTggEkL.s: Fatal error: can't write kws-functions.o: File too big
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836)
/tmp/ccTggEkL.s: Fatal error: can't close kws-functions.o: File too big
I found https://sourceforge.net/p/kaldi/mailman/message/32357729/ where they suggest to replace O0 with at least O1 in kaldi.mk. Maybe that will help.
Perhaps -Os might work better. Not sure, though. Also, you can add "-Wunused-local-typedefs" to your cygwin flags, hopefully it stops the warnings.
MSVC has /bigobj switch that can help in the "File too big" cases, not sure if there is an equivalent switch for gcc/clang y.
On Mon, May 2, 2016 at 3:09 AM, qbolec [email protected] wrote:
I didn't have chance to work on it. I want to get it done today. The only difficulty for me is to get Kaldi to compile with Cygwin. So far, I found this hint https://sourceforge.net/p/kaldi/discussion/1355349/thread/7e088b0b/ helpful. I still get bazillion warnings about
/home/qbolec/kaldi/kaldi/tools/openfst/include/fst/compat.h:70:16: warning: typedef ‘VerifySizesAreEqual’ locally defined but not used [-Wunused-local-typedefs]
and similar (Label, StateId, RevWeight, Weight), and errors like:
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836) /tmp/ccTggEkL.s: Assembler messages: /tmp/ccTggEkL.s: Fatal error: can't write kws-functions.o: File too big /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836) /tmp/ccTggEkL.s: Fatal error: can't close kws-functions.o: File too big
I found https://sourceforge.net/p/kaldi/mailman/message/32357729/ where they suggest to replace O0 with at least O1 in kaldi.mk. Maybe that will help.
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216120582
yenda, I think you mean -Wno-unused-local-typedefs. I'll add it to the cygwin.mk file soon. In the past we have split up object files into multiple pieces to work around Windows limitations like this; I'm surprised it came up again. The source file is fairly small (brings in many templates though). Let us know if -O1 helps, I can add it to the flags. Dan
On Mon, May 2, 2016 at 10:54 AM, jtrmal [email protected] wrote:
Perhaps -Os might work better. Not sure, though. Also, you can add "-Wunused-local-typedefs" to your cygwin flags, hopefully it stops the warnings.
MSVC has /bigobj switch that can help in the "File too big" cases, not sure if there is an equivalent switch for gcc/clang y.
On Mon, May 2, 2016 at 3:09 AM, qbolec [email protected] wrote:
I didn't have chance to work on it. I want to get it done today. The only difficulty for me is to get Kaldi to compile with Cygwin. So far, I found this hint https://sourceforge.net/p/kaldi/discussion/1355349/thread/7e088b0b/ helpful. I still get bazillion warnings about
/home/qbolec/kaldi/kaldi/tools/openfst/include/fst/compat.h:70:16: warning: typedef ‘VerifySizesAreEqual’ locally defined but not used [-Wunused-local-typedefs]
and similar (Label, StateId, RevWeight, Weight), and errors like:
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836) /tmp/ccTggEkL.s: Assembler messages: /tmp/ccTggEkL.s: Fatal error: can't write kws-functions.o: File too big /usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836) /tmp/ccTggEkL.s: Fatal error: can't close kws-functions.o: File too big
I found https://sourceforge.net/p/kaldi/mailman/message/32357729/ where they suggest to replace O0 with at least O1 in kaldi.mk. Maybe that will help.
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216120582
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216257080
I made a pull request to try to fix these compilation issues: https://github.com/kaldi-asr/kaldi/pull/743 please see if it works.
On Mon, May 2, 2016 at 1:24 PM, Daniel Povey [email protected] wrote:
yenda, I think you mean -Wno-unused-local-typedefs. I'll add it to the cygwin.mk file soon. In the past we have split up object files into multiple pieces to work around Windows limitations like this; I'm surprised it came up again. The source file is fairly small (brings in many templates though). Let us know if -O1 helps, I can add it to the flags. Dan
On Mon, May 2, 2016 at 10:54 AM, jtrmal [email protected] wrote:
Perhaps -Os might work better. Not sure, though. Also, you can add "-Wunused-local-typedefs" to your cygwin flags, hopefully it stops the warnings.
MSVC has /bigobj switch that can help in the "File too big" cases, not sure if there is an equivalent switch for gcc/clang y.
On Mon, May 2, 2016 at 3:09 AM, qbolec [email protected] wrote:
I didn't have chance to work on it. I want to get it done today. The only difficulty for me is to get Kaldi to compile with Cygwin. So far, I found this hint https://sourceforge.net/p/kaldi/discussion/1355349/thread/7e088b0b/ helpful. I still get bazillion warnings about
/home/qbolec/kaldi/kaldi/tools/openfst/include/fst/compat.h:70:16: warning: typedef ‘VerifySizesAreEqual’ locally defined but not used [-Wunused-local-typedefs]
and similar (Label, StateId, RevWeight, Weight), and errors like:
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836) /tmp/ccTggEkL.s: Assembler messages: /tmp/ccTggEkL.s: Fatal error: can't write kws-functions.o: File too big
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: kws-functions.o: too many sections (35836) /tmp/ccTggEkL.s: Fatal error: can't close kws-functions.o: File too big
I found https://sourceforge.net/p/kaldi/mailman/message/32357729/ where they suggest to replace O0 with at least O1 in kaldi.mk. Maybe that will help.
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216120582
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216257080
I haven't tried -O1 nor your pull request yet. I did however successfully compile everything using -O2. I'll look into testing your pull request, and/or -O1 later this week. Is it usual that the kaldi directory after successful compilation weights 12,6 GB ??
Yes, it's possible, as you are compiling everything statically (which, I think, is needed in cygwin) y.
On Tue, May 3, 2016 at 5:41 AM, qbolec [email protected] wrote:
I haven't tried -O1 nor your pull request yet. I did however successfully compile everything using -O2. I'll look into testing your pull request, and/or -O1 later this week. Is it usual that the kaldi directory after successful compilation weights 12,6 GB ??
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216481254
marking this WIP. @qbolec, any progress? I've merged my changes to enable hassle-free cygwin compilation, to master-- please test.
I'm working on it right now.
I'm new to git, not really good in cygwin, and totally fresh to kaldi :)
Right now, I'm waiting for git push origin master to complete after I did git merge kaldi-asr/master. It's already more than 30 minutes, and there is no progress indicator, so I'm not sure what's going on.
Meanwhile, I've tried to perform INSTALL steps on this merged working copy, and have some issue. The first one is the same issue I had before, but didn't report on:
$ ./extras/check_dependencies.sh
./extras/check_dependencies.sh: linia 2: $'\r': nie znaleziono polecenia
./extras/check_dependencies.sh: linia 8: $'\r': nie znaleziono polecenia
./extras/check_dependencies.sh: linia 9: błąd składni przy nieoczekiwanym znaczniku `$'{\r''
'/extras/check_dependencies.sh: linia 9: `function add_packages {
The solution is to simply run dos2unix ./extras/check_dependencies.sh on it.
Another one is some error during performing make in tools/ directory:
$ make
extras/check_dependencies.sh
extras/check_dependencies.sh: all OK.
tar xojf sctk-2.4.10-20151007-1312Z.tar.bz2
rm -rf sctk && ln -s sctk-2.4.10 sctk
cd sctk; make config
make[1]: Wejście do katalogu '/cygdrive/e/dev/kaldi/kaldi/tools/sctk-2.4.10'
make[1]: makefile: Permission denied
make[1]: *** Brak reguł do wykonania obiektu 'config'. Stop.
make[1]: Opuszczenie katalogu '/cygdrive/e/dev/kaldi/kaldi/tools/sctk-2.4.10'
Makefile:110: polecenia dla obiektu 'sctk/.configured' nie powiodły się
make: *** [sctk/.configured] Błąd 2
Interestingly if I run it as make -j4 the error is also displayed, but it does not stop the build process (perhaps the three other threads just keep going?). echo $? prints out 2 after make -j4, but it seems that libtools is installing a lot of things successfully.
When performing ./configure in /src directory, similar problem occurs:
$ ./configure
./configure: linia 9: $'\r': nie znaleziono polecenia
./configure: linia 10: $'\r': nie znaleziono polecenia
./configure: linia 23: $'\r': nie znaleziono polecenia
./configure: linia 27: $'\r': nie znaleziono polecenia
./configure: linia 28: błąd składni przy nieoczekiwanym znaczniku `$'{\r''
'/configure: linia 28: `function rel2abs {
The dos2unix ./configure helps a little, but then there is another problem:
$ ./configure
Configuring ...
Checking OpenFST library in /cygdrive/e/dev/kaldi/kaldi/tools/openfst ...
Checking OpenFst library was patched.
Backing up kaldi.mk to kaldi.mk.bak
Doing OS specific configurations ...
On Cygwin: checking for linear algebra libraries ...
Configuration succeeded for platform cygwin
cc1plus: error: unknown pass auto-import specified in -fenable
cc1plus: error: unknown pass auto-import specified in -fenable
make: *** [exp-test] Błąd 1
./configure: linia 221: ./exp-test: No such file or directory
SUCCESS
$ echo $?
0
The error word in the messages is in bright red, so I think it's something worth fixing, but the "SUCCESS" and 0 return code are puzzling. I have gcc version 5.3.0 (GCC).
The ./exp-test mentioned in the output indeed is missing (find -name exp-test does not report it anywhere in /src).
I assume the "auto-import" part is referring to --enable-auto-import for LDFLAGS defined in kaldi.mk and makefiles/cygwin.mk
Despite these error messages make depend (in /src) runs smoothly.
The make (in /src) so far runs without problems (it didn't finish yet though - it takes ages, as I ran it in a single threaded mode).
The make failed with:
make[1]: Wejście do katalogu '/cygdrive/e/dev/kaldi/kaldi/src/chain'
g++ -msse -msse2 -Wall -I.. -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -DHAVE_CLAPACK -I ../../tools/CLAPACK/ -Wno-sign-compare -Wno-unused-local-typedefs -Winit-self -I ../../tools/CLAPACK/ -I /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include -g -c -o chain-supervision.o chain-supervision.cc
In file included from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/fstlib.h:54:0,
from ../fstext/fstext-lib.h:22,
from ../lat/kaldi-lattice.h:24,
from ../chain/chain-supervision.h:30,
from chain-supervision.cc:20:
/cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/compact-fst.h: In instantiation of ‘static bool fst::CompactFst<A, C, U>::WriteFst(const F&, const C&, std::ostream&, const fst::FstWriteOptions&) [with F = fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >; A = fst::ArcTpl<fst::TropicalWeightTpl<float> >; C = fst::AcceptorCompactor<fst::ArcTpl<fst::TropicalWeightTpl<float> > >; U = unsigned int; std::ostream = std::basic_ostream<char>]’:
chain-supervision.cc:532:22: required from here
/cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/compact-fst.h:937:42: warning: variable ‘num_compacts’ set but not used [-Wunused-but-set-variable]
size_t num_arcs = -1, num_states = -1, num_compacts = -1;
^
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: chain-supervision.o: too many sections (34434)
/tmp/ccxSbEtn.s: Assembler messages:
/tmp/ccxSbEtn.s: Fatal error: can't write chain-supervision.o: File too big
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: chain-supervision.o: too many sections (34434)
/tmp/ccxSbEtn.s: Fatal error: can't close chain-supervision.o: File too big
<wbudowane>: polecenia dla obiektu 'chain-supervision.o' nie powiodły się
make[1]: *** [chain-supervision.o] Błąd 1
make[1]: Opuszczenie katalogu '/cygdrive/e/dev/kaldi/kaldi/src/chain'
Makefile:137: polecenia dla obiektu 'chain' nie powiodły się
make: *** [chain] Błąd 2
Hm. It's a shame you don't have access to a Linux machine or a mac. Cygwin is very glitchy, and generally causes a lot of different types of hassle.
Regarding the problems with "git push", these can be caused by network problems or by misconfiguration of git. But when there is Windows in the loop, who knows what it could be.
The problems with \r are likely due to you having edited those files with some type of Windows editor. There are no linefeeds in the checked-in versions of those files in master. dos2unix may fix it. Regarding the red "success", the color is not something we did-- it's probably a hold-over from an earlier error. I'll consider doing something about chain-supervision.o, i.e. splitting it up. As a short-term fix you may be able to edit kaldi.mk to add the flag that Yenda suggested earlier (was it -bigobj? )
Dan
On Sun, May 8, 2016 at 1:34 PM, qbolec [email protected] wrote:
I'm working on it right now. I'm new to git, not really good in cygwin, and totally fresh to kaldi :) Right now, I'm waiting for git push origin master to complete after I did git merge kaldi-asr/master. It's already more than 30 minutes, and there is no progress indicator, so I'm not sure what's going on.
Meanwhile, I've tried to perform INSTALL steps on this merged working copy, and have some issue. The first one is the same issue I had before, but didn't report on:
$ ./extras/check_dependencies.sh ./extras/check_dependencies.sh: linia 2: $'\r': nie znaleziono polecenia ./extras/check_dependencies.sh: linia 8: $'\r': nie znaleziono polecenia ./extras/check_dependencies.sh: linia 9: błąd składni przy nieoczekiwanym znaczniku
$'{\r'' '/extras/check_dependencies.sh: linia 9:function add_packages {The solution is to simply run dos2unix ./extras/check_dependencies.sh on it.
Another one is some error during performing make in tools/ directory:
$ make extras/check_dependencies.sh extras/check_dependencies.sh: all OK. tar xojf sctk-2.4.10-20151007-1312Z.tar.bz2 rm -rf sctk && ln -s sctk-2.4.10 sctk cd sctk; make config make[1]: Wejście do katalogu '/cygdrive/e/dev/kaldi/kaldi/tools/sctk-2.4.10' make[1]: makefile: Permission denied make[1]: *** Brak reguł do wykonania obiektu 'config'. Stop. make[1]: Opuszczenie katalogu '/cygdrive/e/dev/kaldi/kaldi/tools/sctk-2.4.10' Makefile:110: polecenia dla obiektu 'sctk/.configured' nie powiodły się make: *** [sctk/.configured] Błąd 2
Interestingly if I run it as make -j4 the error is also displayed, but it does not stop the build process (perhaps the three other threads just keep going?). echo $? prints out 2 after make -j4, but it seems that libtools is installing a lot of things successfully.
When performing ./configure in /src directory, similar problem occurs:
$ ./configure ./configure: linia 9: $'\r': nie znaleziono polecenia ./configure: linia 10: $'\r': nie znaleziono polecenia ./configure: linia 23: $'\r': nie znaleziono polecenia ./configure: linia 27: $'\r': nie znaleziono polecenia ./configure: linia 28: błąd składni przy nieoczekiwanym znaczniku
$'{\r'' '/configure: linia 28:function rel2abs {The dos2unix ./configure helps a little, but then there is another problem:
$ ./configure Configuring ... Checking OpenFST library in /cygdrive/e/dev/kaldi/kaldi/tools/openfst ... Checking OpenFst library was patched. Backing up kaldi.mk to kaldi.mk.bak Doing OS specific configurations ... On Cygwin: checking for linear algebra libraries ... Configuration succeeded for platform cygwin cc1plus: error: unknown pass auto-import specified in -fenable cc1plus: error: unknown pass auto-import specified in -fenable make: *** [exp-test] Błąd 1 ./configure: linia 221: ./exp-test: No such file or directory SUCCESS $ echo $? 0
The error word in the messages is in bright red, so I think it's something worth fixing, but the "SUCCESS" and 0 return code are puzzling. I have gcc version 5.3.0 (GCC).
The ./exp-test mentioned in the output indeed is missing (find -name exp-test does not report it anywhere in /src). I assume the "auto-import" part is referring to --enable-auto-import for LDFLAGS defined in kaldi.mk and makefiles/cygwin.mk
Despite these error messages make depend (in /src) runs smoothly.
The make (in /src) so far runs without problems (it didn't finish yet though - it takes ages, as I ran it in a single threaded mode).
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-217735275
After performing
git checkout -- .
make clean
dos2unix configure
find . -name "Makefile" | xargs sed -i "s/CXXFLAGS =/CXXFLAGS = -Os/g"
make -j4 depend
make
it fails :(
If I run it with make -j4 it also fails, but manages to compile more files, and emits a lot of warnings similar to this:
n file included from arpa-lm-compiler.cc:26:0:
../base/kaldi-math.h:130:17: warning: ‘kaldi::kLogZeroBaseFloat’ defined but not used [-Wunused-variable]
const BaseFloat kLogZeroBaseFloat = -std::numeric_limits<BaseFloat>::infinity();
I'm now repeating the same procedure with -O1 instead of -Os , and let you know when it finishes.
git checkout -- . make clean dos2unix configure find . -name "Makefile" | xargs sed -i "s/CXXFLAGS =/CXXFLAGS = -Os/g" make -j4 depend make
it fails :(
It would be nice to know how it fails.
I assume that what you are checking out is a branch that contains your changes-- probably this differs from master, in which configure does not have linefeeds.
If I run it with make -j4 it also fails, but manages to compile more files, and emits a lot of warnings similar to this:
n file included from arpa-lm-compiler.cc:26:0: ../base/kaldi-math.h:130:17: warning: ‘kaldi::kLogZeroBaseFloat’ defined but not used [-Wunused-variable] const BaseFloat kLogZeroBaseFloat = -std::numeric_limits<BaseFloat>::infinity();
Usually that warning is something that you see after the real error. Dan
I'm now repeating the same procedure with -O1 instead of -Os , and let you know when it finishes.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-217736354
I have not edited the configure nor check_dependencies.sh files - the error is emitted fresh after performing git checkout -- ..
I think the reason could be in the git client itself - perhaps it chooses line endings depending on platform?
I think that it may be the case that the git client is for Windows, not for Cygwin and thus tries to add \r to the endings?
I want to clarify that the red color is used just for the word "error", and this is probably a regular feature of g++. What I am puzzled about is not the color, but the fact, that despite these errors ("unknown pass auto-import" and missing "exp-test") the final line states a "SUCCESS" and the return code is 0.
BTW. The build with -O1 just failed with the same error. I'll try with -O2.
-O2 failed as well. At this point, I'm wondering how I got it to compile a week ago. Perhaps the chain-supervision was smaller back then.
I apologize for copy-pasting only a fragment of the warning. Each warning looks like this:
make[1]: Wejście do katalogu '/cygdrive/e/dev/kaldi/kaldi/src/decoder'
g++ -msse -msse2 -Wall -I.. -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -DHAVE_CLAPACK -I ../../tools/CLAPACK/ -Wno-sign-compare -Wno-unused-local-typedefs -Winit-self -I ../../tools/CLAPACK/ -I /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include -O2 -Wno-sign-compare -g -c -o training-graph-compiler.o training-graph-compiler.cc
In file included from ../base/kaldi-common.h:38:0,
from ../decoder/training-graph-compiler.h:22,
from training-graph-compiler.cc:18:
../base/kaldi-math.h:130:17: warning: ‘kaldi::kLogZeroBaseFloat’ defined but not used [-Wunused-variable]
const BaseFloat kLogZeroBaseFloat = -std::numeric_limits<BaseFloat>::infinity();
^
and there are many of them but all related to kaldi::kLogZeroBaseFloat and do not terminate the make.
As for the error which terminates the make it is always:
g++ -msse -msse2 -Wall -I.. -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -DHAVE_CLAPACK -I ../../tools/CLAPACK/ -Wno-sign-compare -Wno-unused-local-typedefs -Winit-self -I ../../tools/CLAPACK/ -I /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include -g -c -o chain-supervision.o chain-supervision.cc
In file included from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/fstlib.h:54:0,
from ../fstext/fstext-lib.h:22,
from ../lat/kaldi-lattice.h:24,
from ../chain/chain-supervision.h:30,
from chain-supervision.cc:20:
/cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/compact-fst.h: In instantiation of ‘static bool fst::CompactFst<A, C, U>::WriteFst(const F&, const C&, std::ostream&, const fst::FstWriteOptions&) [with F = fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >; A = fst::ArcTpl<fst::TropicalWeightTpl<float> >; C = fst::AcceptorCompactor<fst::ArcTpl<fst::TropicalWeightTpl<float> > >; U = unsigned int; std::ostream = std::basic_ostream<char>]’:
chain-supervision.cc:532:22: required from here
/cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/compact-fst.h:937:42: warning: variable ‘num_compacts’ set but not used [-Wunused-but-set-variable]
size_t num_arcs = -1, num_states = -1, num_compacts = -1;
^
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: chain-supervision.o: too many sections (34434)
/tmp/cc68biKg.s: Assembler messages:
/tmp/cc68biKg.s: Fatal error: can't write chain-supervision.o: File too big
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/as: chain-supervision.o: too many sections (34434)
/tmp/cc68biKg.s: Fatal error: can't close chain-supervision.o: File too big
<wbudowane>: polecenia dla obiektu 'chain-supervision.o' nie powiodły się
make[1]: *** [chain-supervision.o] Błąd 1
make[1]: Opuszczenie katalogu '/cygdrive/e/dev/kaldi/kaldi/src/chain'
Makefile:137: polecenia dla obiektu 'chain' nie powiodły się
make: *** [chain] Błąd 2
I have not edited the configure nor check_dependencies.sh files - the error is emitted fresh after performing git checkout -- ..
The behavior of that command will change depending which branch you are on -- you mentioned that you had merged your changes. "git diff master..HEAD" may show whether
I think the reason could be in the git client itself - perhaps it chooses line endings depending on platform? I think that it may be the case that the git client is for Windows, not for Cygwin and thus tries to add \r to the endings?
You can figure this out with
which git. If your git client is indeed for Windows, this would be bad news. You'd have to reinstall git inside cygwin.
I want to clarify that the red color is used just for the word "error", and
this is probably a regular feature of g++. What I am puzzled about is not the color, but the fact, that despite these errors ("unknown pass auto-import" and missing "exp-test") the final line states a "SUCCESS" and the return code is 0.
Don't worry about that part, it just checks whether expf() is slower than exp(), and the result isn't important.
Try adding the flags -Wa,-mbig-obj to the CXXFLAGS by manually editing kaldi.mk and see if this helps. The "object file too big" errors are a Windows limitation. I don't think I can easily fix it by breaking up the code because I think there are too many files like this. chain-supervision.o is not even the biggest one-- kws-search.o is bigger. It's getting a bit ugly.
Dan
BTW. The build with -O1 just failed with the same error. I'll try with -O2.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-217736692
The build has not finished yet, but I see a new kind of warning in the mix:
g++ -Wa,-mbig-obj -msse -msse2 -Wall -I.. -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -DHAVE_CLAPACK -I ../../tools/CLAPACK/ -Wno-sign-compare -Wno-unused-local-typedefs -Winit-self -I ../../tools/CLAPACK/ -I /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include -O2 -Wno-sign-compare -g -c -o gmm-latgen-biglm-faster.o gmm-latgen-biglm-faster.cc
In file included from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/arc.h:29:0,
from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/fst.h:34,
from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/fstlib.h:49,
from ../fstext/fstext-lib.h:22,
from gmm-latgen-biglm-faster.cc:28:
/cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/float-weight.h: In function ‘void fst::ReadFstKaldi(std::istream&, bool, fst::VectorFst<A>*) [with Arc = fst::ArcTpl<fst::TropicalWeightTpl<float> >; std::istream = std::basic_istream<char>]’:
/cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/float-weight.h:65:63: warning: ‘w.fst::FloatWeightTpl<float>::value_’ may be used uninitialized in this function [-Wmaybe-uninitialized]
FloatWeightTpl(const FloatWeightTpl<T> &w) : value_(w.value_) {}
^
You can ignore the warnings.
On Sun, May 8, 2016 at 3:08 PM, qbolec [email protected] wrote:
The build has not finished yet, but I see a new kind of warning in the mix:
g++ -Wa,-mbig-obj -msse -msse2 -Wall -I.. -DKALDI_DOUBLEPRECISION=0 -DHAVE_POSIX_MEMALIGN -DHAVE_CLAPACK -I ../../tools/CLAPACK/ -Wno-sign-compare -Wno-unused-local-typedefs -Winit-self -I ../../tools/CLAPACK/ -I /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include -O2 -Wno-sign-compare -g -c -o gmm-latgen-biglm-faster.o gmm-latgen-biglm-faster.cc In file included from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/arc.h:29:0, from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/fst.h:34, from /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/fstlib.h:49, from ../fstext/fstext-lib.h:22, from gmm-latgen-biglm-faster.cc:28: /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/float-weight.h: In function ‘void fst::ReadFstKaldi(std::istream&, bool, fst::VectorFst<A>*) [with Arc = fst::ArcTplfst::TropicalWeightTpl
; std::istream = std::basic_istream ]’: /cygdrive/e/dev/kaldi/kaldi/tools/openfst/include/fst/float-weight.h:65:63: warning: ‘w.fst::FloatWeightTpl ::value_’ may be used uninitialized in this function [-Wmaybe-uninitialized] FloatWeightTpl(const FloatWeightTpl<T> &w) : value_(w.value_) {} ^ — You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-217740036
@qbolec: are you still planning to finish this? Or should I find someone else?
On Tue, May 3, 2016 at 8:45 AM, jtrmal [email protected] wrote:
Yes, it's possible, as you are compiling everything statically (which, I think, is needed in cygwin) y.
On Tue, May 3, 2016 at 5:41 AM, qbolec [email protected] wrote:
I haven't tried -O1 nor your pull request yet. I did however successfully compile everything using -O2. I'll look into testing your pull request, and/or -O1 later this week. Is it usual that the kaldi directory after successful compilation weights 12,6 GB ??
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216481254
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/kaldi-asr/kaldi/pull/732#issuecomment-216516017