midimonster icon indicating copy to clipboard operation
midimonster copied to clipboard

Rounding when two-bytes data is converted to one-byte data?

Open cjunekim opened this issue 3 years ago • 1 comments

It seems to truncate the value when converting two-bytes to one-byte data.

For example, pitchbend to cc. Suppose the pitchbend data incoming is 0x60 0x68 (LSB and MSB, in order). Its normalized value is(tested on Lua console):

> ((0x68 << 7) | 0x60&0x7f)/16383
0.81840932674113

When it is handed to CC, it is converted as,

> ((0x68 << 7) | 0x60&0x7f)/16383 * 127.0
103.93798449612

This value is handed to an unsigned int and the fractional part is discarded becoming 103, which, in turn, in hex is 0x67. It isn't 0x68, even though it was closer to 0x68, mathematically. It's because there is no rounding algorithm when a conversion from higher resolution to lower resolution occurs. I think this can cause some confusion and issues.

What do you think?

cjunekim avatar Dec 02 '22 19:12 cjunekim

Hm yes, not optimal in this case but hard to fix. Will think about it some more..

cbdevnet avatar Jun 24 '24 15:06 cbdevnet