audiofile.js icon indicating copy to clipboard operation
audiofile.js copied to clipboard

Sample scaling is incorrect

Open Barabas5532 opened this issue 3 years ago • 1 comments

The code for unsigned to signed integer conversion is incorrect.

Paste this into console, and the result is wrong.

function scale(value, bitDepth)
{
    var range = 1 << bitDepth - 1;
    if (value >= range) {
        value |= ~(range - 1);
    }
    return value;
}

[0, 1, 2, 3].forEach(function(n) {console.log(n + ", " + scale(n, 2))})
Output:
0, 0
1, 1
2, -2
3, -1

Moreover, according to stack overflow, greater than 8 bits per sample is already signed, so there is no need to do this conversion.

Barabas5532 avatar Apr 11 '21 13:04 Barabas5532

Hmm, it seems this actually does sign extension on the data, so only the comment is wrong. 8-bits per sample (always unsigned) will be broken after sign extension though.

Barabas5532 avatar Apr 11 '21 13:04 Barabas5532