How can I pass --raw command to avoid unicode characters inside keys?

[danitseitlin]

I want to pass use the --raw flag when using ioredis, is there a way to achieve that?

[danitseitlin]

@luin What can you say about this issue?

[TysonAndre]

I'm not sure what you mean by avoid unicode characters inside keys. Are you asking about https://en.wikipedia.org/wiki/Character_encoding (such as https://en.wikipedia.org/wiki/UTF-8 or single-byte character encodings such as https://en.wikipedia.org/wiki/ISO/IEC_8859-1)

https://redis.io/topics/rediscli seems to mean something entirely different in redis-cli from what you're asking (avoid unicode characters)?

If you mean raw Buffers (byte arrays) as keys, that partly works, but doesn't seem like it'd work with some features, such as autopipelining in Redis.Cluster https://github.com/luin/ioredis/blob/v4.27.9/lib/autoPipelining.ts#L145-L148 - which use strings for keyPrefix, etc. to hash right now

And hgetall assumes keys can be converted to utf-8 strings to use as object keys

(Redis also provides *Buffer alternates for many commands if you need to fetch raw bytes of values)

const Redis=require('ioredis');
const redis = new Redis();
const binKey = new Buffer([0x80]);
console.log(await redis.set(binKey, binKey));  // OK
console.log(await r.get(binKey)); // �
console.log(await r.getBuffer(binKey)); // <Buffer 80>

[TysonAndre]

When using

await this.client.set('api:cities', '.', '{"Mingá": {"id": "18", "name": "Mingá"}}'); in Redis I got this

{"Ming\u00c3\u00a1":{"id":"18","name":"Ming\u00c3\u00a1"}}

One possibility is the original issue had a terminal using a different encoding from utf-8 but they don't include enough information to say what client was used to get that string or whether something json encoded it, etc. https://nodejs.org/api/buffer.html#buffer_buffer may help. E.g. if a user has their terminal using latin-1 (and the string can be represented in latin1)

// Creates a Buffer containing the Latin-1 bytes [0x74, 0xe9, 0x73, 0x74].
const buf7 = Buffer.from('tést', 'latin1');

I also don't know why the spaces were in the redis.set call and not in the value they are showing as the resulting example - the steps to reproduce are inexact

[TysonAndre]

I don't even think the original linked question is a question related to ioredis. It may or may not be about encodings. If you pass --raw to redis-cli that changes the way redis-cli displays the data. But it's properly encoded as UTF-8 in the redis server storage, if you have a terminal that supports utf-8 you can see the raw utf-8 value displayed properly

» redis-cli 
127.0.0.1:6379> get foo
"{\"Ming\xc3\xa1\": {\"id\": \"18\", \"name\": \"Ming\xc3\xa1\"}}"
» redis-cli --raw
127.0.0.1:6379> get foo
{"Mingá": {"id": "18", "name": "Mingá"}}

[danitseitlin]

@TysonAndre The question originally asks how can I use the --raw arg using ioredis.

[luin]

@danitseitlin Can we take a step back? What do you want with the raw mode?

The protocol used by ioredis and Redis is binary-safe so it's encoding agnostic. If you need to work with data that can't be expressed with JavaScript's string, then you should go with the approach mentioned by @TysonAndre.

[danitseitlin]

@luin Sure, The reason is because of the users of this SDK asked about this and wanted to return the data as if it's been sent back from redis-cli with --raw parameter. So I was wondering if this is possible from the ioredis so I could support his requests. Here is the original issue: https://github.com/danitseitlin/redis-modules-sdk-ts/issues/130 Thanks for responding :)

[danitseitlin]

@luin Still waiting on some info here

[luin]

Turns out it is not as easy as it should be to skip response transformation. ioredis does provide a method to opt-out but it affects all ioredis instances:

Redis.Command.setReplyTransformer("hgetall", null);
client.hgetall("foo", console.log);