Adding an eventlog exporter

[TeofilC]

I'd like to add an eventlog exporter to this library.

I'm thinking it would allow writing traces to the eventlog, and some extra data from the eventlog can be added (eg, sleeping threads, extra GC info, why threads are being blocked), and then a separate process can re-export these traces using the existing exporters.

I wanted to open an issue before starting work on this in case anyone had any thoughts about it.

[parsonsmatt]

@wavewave I think we were talking about something like this recently

[iand675]

I think this is a very very good idea. A sketch for how we might implement this now:

Use eventlog-socket along with ghc-events to support this. I think that it would be interesting to support a few modes of operation:

• As a sidecar process that connects to the provided eventlog socket or file. This is probably more operationally robust, but maybe more annoying to use outside of server-type contexts.
• As a library that can be baked into an executable that processes its own eventlog. I'd envision this being useful for e.g. sending telemetry for CLI tools at the expense of the executable polluting its own eventlog slightly.

[wavewave]

I just noticed this thread. Exactly! In fact, I did some experiments with eventlog-socket and ghc-events on a middle-sized app (hoodle) for which real-time performance is pretty important, and the eventlog-socket overhead was very negligible. On target app side, the necessary modification is really minimal. just one line addition at the top-level main. https://github.com/wavewave/hoodle/blob/19bf2dd111fa521e4d256533a91b89f8266bad57/hoodle/app/hoodle/Main.hs#L18

On monitoring app side, as the first case in the above comment by @iand675, the external monitoring app continuously monitors the connected socket and parses bytestring to ghc-event type. My experiment is here: https://github.com/MercuryTechnologies/ghc-specter/blob/main/logcat/app/logcat/Log.hs#L88

The blocking step on target app is the initialization (to start the socket), and afterwards, even if the monitoring app is killed, the target app continuously working and can reconnect the monitoring app when rerun the app. The reconnection is quite robust in my experiment.

[TeofilC]

Nice! These are the things I've been experimenting with as well. Great to see other folks are on the same page!

As a library that can be baked into an executable that processes its own eventlog. I'd envision this being useful for e.g. sending telemetry for CLI tools at the expense of the executable polluting its own eventlog slightly.

I've wondered about this as well. I wonder if having the processing happen in the same process would interfere with statistics, eg, extra GCs.