peoples-speech icon indicating copy to clipboard operation
peoples-speech copied to clipboard
verified publisher icon mlcommons

Truncating keys to length 100 sometimes causes duplication

Open galv opened this issue 4 years ago • 1 comments

I found this by diffing utt2spk and spk2utt in kaldi:

Can I write tar files with no maximum file name length?

-Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_87.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_87.flac Talented\
_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_87.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_87.flac Talented_People_w\
ith_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_87.flac                                                                                                                             
-Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_88.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_88.flac Talented\
_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_88.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_88.flac Talented_People_w\
ith_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_88.flac                                                                                                                             
-Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_89.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_89.flac Talented\
_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_89.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_89.flac Talented_People_w\
ith_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_89.flac                                                                                                                             
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_0.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_0.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_0.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_0.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_1.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_1.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_1.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_1.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_2.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_2.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_2.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_2.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_3.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_3.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_3.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_3.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_4.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_4.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_4.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_4.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_5.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_5.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_5.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_5.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_6.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_6.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_6.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_6.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_7.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_7.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_7.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_7.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_8.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_8.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_8.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_8.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_9.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_9.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_9.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_9.flac            
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_10.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_10.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_10.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_10.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_11.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_11.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_11.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_11.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_12.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_12.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_12.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_12.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_13.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_13.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_13.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_13.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_14.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_14.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_14.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_14.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_15.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_15.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_15.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_15.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_16.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_16.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_16.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_16.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_17.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_17.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_17.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_17.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_18.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_18.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_18.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_18.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_19.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_19.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_19.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_19.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_20.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_20.flac          
+Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_20.flac Talented_People_with_Jeanne_Murphy_10_16-Talented_People_with_Jeanne_Murphy_10_16_HD_mp3_20.flac

galv avatar Sep 08 '21 18:09 galv

Solution is to set longFileMode= LONGFILE_POSIX as described here: http://commons.apache.org/proper/commons-compress/tar.html

galv avatar Sep 08 '21 18:09 galv