Self-host GlitchTip

[0dm]

We are reaching our monthly event quota on GlitchTip, and it would be beneficial for us to host it ourselves.

[eLRuLL]

I would like to contribute working on this issue 👍

Basically GlitchTip offers multiple ways to host it yourself as specified here: https://glitchtip.com/documentation/install so it depends on the infrastructure that you guys have at the moment (do you guys have your own servers? are you using kubernetes? any cloud provider?).

I could also setup a docker-compose file on the repository so this can be deployed on different local environments, what do you guys suggest?

[abrichr]

@eLRuLL thank you!

Currently everything is locally hosted (except https://github.com/microsoft/SoM which will be deployed via deploy.py to EC2 (and eventually to ECS) as per https://github.com/OpenAdaptAI/OpenAdapt/issues/543.

Perhaps this should probably be deployed via a similar mechanism. What do you think?

[eLRuLL]

I checked about SoM and the discussion in #543 and I don't think we need so much control for this service (and even for SoM).

To make GlitchTip work with the application, we only have to setup the ERROR_REPORTING_DSN env variable at the start of the commands, and the application is also already configured to get that from an env variable or the .env file.

This means that we should make GlitchTip start independently, it shouldn't be part of the start of our normal commands or application. People that start this locally don't actually need GlitchTip (so we should actually make ERROR_REPORTING_ENABLED=False by default), or do we want to give their own Error Tracking to everybody starting this locally? I don't see the value on that.

The quota error you are getting is because everybody running the application is reporting to the GlitchTip you guys own, which shouldn't be the case, the main developers should be the only ones using this GlitchTip instance. To make this cost 0 or even less you can "self-host" a GlitchTip instance on your own servers, but it's still a unique GlitchTip instance hosted on your servers (not one GlitchTip instance for each person running the app).

So the changes I would recommend would be:

• Deploy a GlitchTip instance on your own servers (this can be done as specified here https://glitchtip.com/documentation/install)
• Set ERROR_REPORTING_ENABLED to False by default
• Set ERROR_REPORTING_DSN to an empty string by default
• Delete the GlitchTip app that https://[email protected]/3798 belongs to (or cancel its subscription at least)
• The self-hosted instance offers a new GlitchTip DSN, which should be shared secretly to the developers that want to be tracked

what do you think @abrichr ?

[abrichr]

Thanks @eLRuLL !

the main developers should be the only ones using this GlitchTip instance

Can you please clarify? Our goal here is to prioritize bugs, which we can do most effectively if we are collecting errors from as many users as possible.

I believe you are correct that most users won't want to host their own, but our goal is to build the infrastructure so that those who wish to do so, can.

We can then also use this infrastructure ourselves.

Please let me know if anything is unclear! 🙏

[eLRuLL]

I totally agree, so here are my recommendations taking several things into account:

For transparency, privacy, and data protection regulations we should always inform users if we are somehow tracking actions on their commands. This is a pretty common pattern when dealing with command-line applications, where the application informs the user when an error happens if he would like to report that error to a regulatory external company. We could start doing this by setting 2 types of environments, maybe something like DEBUG and PRODUCTION. If the user selects DEBUG, we could show a message indicating that we'll be tracking some errors, or we could also set some prompts to accept this tracking. In this case, we set our own GlitchTip DSN, and if not (or in PRODUCTION environment) we set ERROR_REPORTING_ENABLED=False. (FYI this should be their own Issue, as it's divering from the main topic of this Issue)

On the part of users wanting to host their own infrastructure, I totally agree with you but we have to differentiate between what are some "useful tools" and what are some "dependencies" of the application. GlitchTip or any Error Reporting is a "useful tool" not a dependency of the application. The only thing that we need to do is to set the ERROR_REPORTING_DSN and the errors would go into that specified GlitchTip instance.

Coming back to the reason for opening this issue, this is because you guys are using GlitchTip's hosted instances for error reporting, for which the GlitchTip company is charging you or setting this event quota to restrict abuse. What we have to do is, as specified in this issue's description, host our own GlitchTip instance, which is completely external to the code of this repository, and then update our ERROR_REPORTING_DSN with the new variable gotten from this self-hosted instance. This means that the only code changes respective to this repo for this issue would be the update of the ERROR_REPORTING_DSN

If you are planning on hosting several things on EC2, take into account that GlitchTip has its own dependencies (a Postgres database, Redis, etc) which can also be hosted on the EC2 instance, or we could deploy those resources independently. The part about showing users to "host their own GlitchTip instance" shouldn't be OpenAdaptAI's problem, but this GlitchTip's problem and that's why they are giving us instructions like https://glitchtip.com/documentation/install

I hope this gave a better explanation of the path going forward, please let me know what you think 👍