slog icon indicating copy to clipboard operation
slog copied to clipboard

Best way to send record over IPC channel??

Open spease opened this issue 4 years ago • 3 comments

Working on a project where there's some concern over making things async signal safe. It looks like to do that, we'll need to send slog records to a separate process which can then handle writing etc.

It looks like the way to do this would be to create a Drain. However this requires implementing log. OwnedKVList is Sync and Send, but Record appears to be marked explicitly not Send or Sync.

What would be the most efficient way to serialize and then deserialize a Record (and OwnedKVList) across an IPC?

Thanks.

EDIT: Updated to explicitly ask about OwnedKVList as well, since just because it's Send/Sync would not preclude it from having an Arc or something that would break between processes.

spease avatar Sep 04 '19 23:09 spease

slog-async is one reference. I don't know what your performance requirements are. If you can go slow - you can just serialize everything and send over IPC. If you need to go fast, I'd go with some preallocated circular buffer queue in a shared memory between processes.

dpc avatar Sep 05 '19 02:09 dpc

Hmm ok. I’m not too worried about the performance of the IPC, and assumed I’d look into the options provided by ipc-channel, which I believe does provide a shared memory backend.

Sounds like slog-async does de/serialization that would be compatible with IPC?

spease avatar Sep 05 '19 04:09 spease

slog-async doesn't really do much except sending the data to another thread. Sooo.... am.... I guess it is not all that relevant.It might be useful if you're worried about the serialization slowing down the execution.

The actual serialization is going to probably be similar to slog-json. You might want to use something more compact/binary and faster though: bson, msgpack, protobuf or something like that.

dpc avatar Sep 05 '19 04:09 dpc