add posthog for product analytics

[andrewm4894]

This PR adds PostHog for product analytics telemetry. I set it up as per docs here.

End result is events (pageviews and click events) coming into the PostHog project like this:

You can then define "Actions" based on events - so for example this is the "Submit" action, based on the data attributes of what was clicked.

Anyway - anyone can feel free to play with this - maybe just dm me on discord and i can add you to the PostHog project so you can then click around locally and see how the events flow though in PostHog.

Really i just wanted to do this as a quick POC to show what i mean by using a tool like PostHog for product analytics which could be useful for insights on usage and even sharing public dashboards back with the user community etc. Obviously is a bigger decision or discussion around tools and things like that.

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[andrewm4894]

Here is a quick example of making a sharable public dashboard https://app.posthog.com/shared/ky7upoXLDdXYhVFkyEFRCaX0DQ-C6A

[andreaskoepf]

Why would we want to collect this data? What about privacy?

[andrewm4894]

Why would we want to collect this data? What about privacy?

So really it would be to understand how people are using the app and so more easily spot issues etc. If things break or usage trends change in some way that can be a sign of some issues.

In terms of privacy you can from the frontend remove things like ip addresses etc fairly easily or you could have telemetry itself disabled by default and allow users to opt in if they so wished.

It really depends and each project or company is going to be different.

Idea here is if you want to understand how people are actually using the app in order to understand and come up with potential ideas to improve and make it better then tools like this can be useful.

One of main reasons I was proposing posthog is that you can customize it to do as little or as much tracking as suits your own use case and it's all fairly easy and painless as opposed to just using something more common like google analytics.

It can also be much easier to get insights like this than having to do lots of custom instrumentation on the backend with tools like Grafana and Prometheus etc which while also cool can sometimes need more overhead to maintain if you want anything custom that correlates to how users are actually using the app (they usually more focused on backend services and metrics etc).

Of course for sure it would also be a totally fine decision to decide something like this is just not needed or useful yet, or that even as a project we don't want to do any analytics on users (it's always a big and tricky area, especially for open source projects).

[fozziethebeat]

We chatted a bit about this in Discord. The current opinion between me and Andreas is that we should wait on this. We're currently not too worried about diagnosing the kinds of problems PostHog would provide. Further, I'd rather not include any more cloud dependencies that we can't self host (since that makes it harder for anyone else to deploy this for themselves). Andreas also generally doesn't like these kinds of analytics and what they track.

When the time comes, we maybe will include this but I probably want to prefer an option that can be totally self hosted. PostHog self-hosting looks really hard and handles a very small load given how complex it is.

[andrewm4894]

no worries, mostly i just wanted to make a POC to show how easy it would be to add - deciding whether should or not for sure a whole different decision.

i'd say maybe should be able to get by with backend stats and metrics for stuff like task completion volume and things like that if and when someone looks to measure that stuff without needing the more fine grained and granular FE telemetry that for sure can rub some people the wrong way.

I was also going to try help on BE observability side too in here: https://github.com/LAION-AI/Open-Assistant/issues/1121 so should be ways to count anything custom on BE if we wanted above standard BE infra metrics and things like that.

Closing this.