Two undefined behaviors by bitshifting of negative values
Clang's sanitizers complain about bitshifting of negative values on 64-bit Ubuntu Linux:
1:
libvorbis/src/floor1.c:840:24: runtime error: left shift of negative value -2 #0 0x7f34f64c3eb1 in floor1_encode libvorbis/src/floor1.c #1 0x7f34f650acd0 in mapping0_forward libvorbis/src/mapping0.c:625:20 #2 0x7f34f64b0035 in vorbis_analysis libvorbis/src/analysis.c:47:11
Fix:
libvorbis/src/floor1.c:
- val=-1-(val<<1); + val=-1-(val*2);
2:
libvorbis/src/psy.c:320:24: runtime error: left shift of negative value -11 #0 0x7fdfedcec8b5 in _vp_psy_init libvorbis/src/psy.c:320:24 #1 0x7fdfedcfc87b in _vds_shared_init libvorbis/src/block.c:225:7 #2 0x7fdfedcfbf6d in vorbis_analysis_init libvorbis/src/block.c:298:6
Fix:
libvorbis/src/psy.c: - p->bark[i]=((lo-1)<<16)+(hi-1); + p->bark[i]=((lo-1)*65536)+(hi-1);