Binary states - avoid buffer.toString() / console.log() - write buffer contents directly to process.stdout

[klein0r]

Tested with js-controller 4.0.23

Test Script (PNG file header):

await this.setForeignObjectNotExistsAsync('0_userdata.0.testBinary', {
    type: 'state',
    common: {
        name: 'Test Binary',
        type: 'file',
        role: 'value',
        read: true,
        write: false,
    },
    native: {},
});

const uint8 = new Uint8Array([0x50, 0x89, 0x47, 0x4e, 0x0a, 0x0d, 0x0a, 0x1a, 0x00, 0x00, 0x0d, 0x00, 0x48, 0x49, 0x52, 0x44]);
const testData = Buffer.from(uint8);

this.setForeignBinaryState('0_userdata.0.testBinary', testData, () => {
    this.log.debug('saved');
});

read with hex option

./iobroker state getBinary 0_userdata.0.testBinary --encoding hex
5089474e0a0d0a1a00000d0048495244

✅ ok

read with base64 option

./iobroker state getBinary 0_userdata.0.testBinary --encoding base64 | base64 -d | hexdump -C
00000000  50 89 47 4e 0a 0d 0a 1a  00 00 0d 00 48 49 52 44  |P.GN........HIRD|

✅ ok

read with binary option

./iobroker state getBinary 0_userdata.0.testBinary --encoding binary | hexdump -C
00000000  50 c2 89 47 4e 0a 0d 0a  1a 00 00 0d 00 48 49 52  |P..GN........HIR|
00000010  44 0a                                             |D.|

⚠️ ???

[AlCalzone]

Is it possible that js-controller uses the "binary" encoding for this? AFAIK this is some weird text encoding and does NOT result in the raw buffer

[foxriver76]

Yes, this is exactly the case, encoding is passed as given https://github.com/ioBroker/ioBroker.js-controller/blob/1fe5a1e66dc343f146a54a3b5476add09aa97416/packages/cli/src/lib/cli/cliStates.js#L142

[klein0r]

Okay, seems like that option is really strange:

https://stackoverflow.com/questions/52564196/buffer-tostringbinary-adds-new-characters

https://nodejs.org/docs/latest-v16.x/api/buffer.html#buffers-and-character-encodings

'binary': Alias for 'latin1'. See binary strings for more background on this topic. The name of this encoding can be very misleading, as all of the encodings listed here convert between strings and binary data. For converting between strings and Buffers, typically 'utf8' is the right choice.

[klein0r]

Why is toString used there? Maybe it would be a better option to write the buffer contents to stdout?

const uint8 = new Uint8Array([0x50, 0x89, 0x47, 0x4e, 0x0a, 0x0d, 0x0a, 0x1a, 0x00, 0x00, 0x0d, 0x00, 0x48, 0x49, 0x52, 0x44]);
const testData = Buffer.from(uint8);

console.log(testData.toString('binary'));

⏬

node testBuffer.js | hexdump -C
00000000  50 c2 89 47 4e 0a 0d 0a  1a 00 00 0d 00 48 49 52  |P..GN........HIR|
00000010  44 0a                                             |D.|
00000012

---

const uint8 = new Uint8Array([0x50, 0x89, 0x47, 0x4e, 0x0a, 0x0d, 0x0a, 0x1a, 0x00, 0x00, 0x0d, 0x00, 0x48, 0x49, 0x52, 0x44]);
const testData = Buffer.from(uint8);

process.stdout.write(testData);

⏬

node testBuffer.js | hexdump -C
00000000  50 89 47 4e 0a 0d 0a 1a  00 00 0d 00 48 49 52 44  |P.GN........HIRD|
00000010

[Apollon77]

Ok, then the "fix" is easy ... when "necoding" is set we use "toString" and if no encoding is set we putput the buffer ...

[Apollon77]

As we implemetned this is controller 4 we were sure that binary means binary :-((