posthog
posthog copied to clipboard
refactor(plugin-server): Unify event types
Problem
When working on #10572, I realized how messy our typing for events is. Five types were exposed, and it was really hard to say which is for what:
Event– an obsolete shape based on the old Postgresposthog_eventtableClickHouseEvent– a relevant shape, but built by extending ofEventand lacking persons-on-events columnsPreIngestionEvent– a shape used for some intra-plugin-service processingIngestionEvent– weirdly an alias ofPreIngestionEventClickhouseEventKafka– basicallyClickHouseEventbut slightly different for no good reason
Additionally there are two @posthog/plugin-scaffold types: PluginEvent and ProcessedPluginEvent.
Both used a lot internally, which is hampering #10572 – because internal code is so closely intertwined with the representation exposed to plugin developers, changing the external API is now painful.
Changes
This reworks the typing structure to something more closely resembling the real world. Work in progress, but at the moment the scheme looks like this:
RawEvent– raw ClickHouse event row (i.e. datetimes and JSON are strings), in sync with the current schemaEvent – same as above, but processed for JavaScript (i.e. datetimes and JSON are parsed)PreIngestionEvent – mostly the same, except can't contain serialized timestamps anymorePostIngestionEvent – clearer alias ofIngestionEvent
How did you test this code?
Relying on TypeScript and existing tests to catch errors.
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.
This PR was closed due to 2 weeks of inactivity. Feel free to reopen it if still relevant.
Sorry this is taking a while to get in - other work took priority.
Note that the work you did rebasing/updating this before was kind of unneeded - as mentioned in the original slack thread, taking over this PR and will try get it merged :)
Got everything green and added most of the missing testing. See comments above for what changed.
I also introduced branded types to disambiguate between ISO and Clickhouse timestamps as I removed the clonable concept.
Will merge on Monday as on holiday for next 2 days and want to monitor for any unintended regressions.