jitsu

jitsu copied to clipboard

Problem

Most of the changes of the configuration within Jitsu requires server restart. This is not very convenient in some cases. One of the canonical examples is storage (Redis) migration. Jitsu uses Redis for multiple purposes:

• Configuration storage (if configurator is used)
• Retroactive User Recognition cache
• Event Queues
• Event Cache used for Live Events in UI (non-critical)
• Coordination (distributed locks)

Jitsu could work well with Redis down for a long period of time. One can take advantage of that for Redis migration by following the pattern:

1. Shutdown Redis
2. Move Redis DB to a new location (via BGSAVE / restore)
3. Configure Jitsu to use a new Redis

However, it will break on step 3. Jitsu should be restarted to apply config changes, and with Redis down all systems rely on RAM, so all data will be lost.

Use-case above is not the only one when user needs hot reload. Other cases might be:

• Turn on/off debug logging
• Changes in destination / sources / api_keys — that part is already partially implemented, if the configuration is externalized

Solution

Jitsu server should accept a command to reload configuration on the fly. The command can either be sent as Admin HTTP end-point call or as SIGHUP signal.

Once the command is reсeived, Jitsu should get a new config (including file, external JSON, env vars), apply the changes in the sections we support (redis) and respond with a short report of what happened (and write the report to logs, useful for SIGHUP handler).

Also, in the scope of this issue we need to allow Jitsu start without Redis (but put a very clear log message).