Does pingora support dynamically adding services after startup?

[machine3]

For example, dynamically add listener, router, upstream, etc.

[vicanso]

After modifying the configuration, stop the old process and start a new one.

https://github.com/cloudflare/pingora/blob/main/docs/quick_start.md#gracefully-upgrade-the-service

[eaufavor]

Right now it does not support adding new listeners or service without a (graceful) restart as mentioned above. But you can have dynamic routing and upstream without restarting.

[sandys]

@eaufavor a followup question on this - what is the mental model of a large scale deployment. for e.g. 1000 users have deployed 2000 services and i want pingora to be the front for these services. Users will be constantly changing their configuration.

does this mean - each user has a pingora instance for them ? so whenever they update the config, it gets updated and redeployed ? or is there a smaller pool of pingora instances shared between all users ? in which case, a config update would mean all instances would get redeployed.

pingora pulling config from a database on each request would add too much latency i suppose.

[eaufavor]

pingora pulling config from a database on each request would add too much latency i suppose.

Not necessarily if done with a caching layer. There are two ways to make this fast

1. With https://docs.rs/pingora-memory-cache/latest/pingora_memory_cache/ you can pull and cache the user conf (either the entire conf or per user conf if your database supports it) out of the database and cache it. You only pay the latency every TTL seconds.
2. Put the user conf in a ArcSwap and refresh it in the background. In this case no request would pay for the latency. The background refresh logic can be either active (refresh every TTL seconds) or reactive (the external system will signal it to refresh when the conf is changed).

[hikaricai]

@eaufavor a followup question on this - what is the mental model of a large scale deployment. for e.g. 1000 users have deployed 2000 services and i want pingora to be the front for these services. Users will be constantly changing their configuration.

does this mean - each user has a pingora instance for them ? so whenever they update the config, it gets updated and redeployed ? or is there a smaller pool of pingora instances shared between all users ? in which case, a config update would mean all instances would get redeployed.

pingora pulling config from a database on each request would add too much latency i suppose.

You can add L4 nat proxy before the pingora and read both proxy info and service config at the begining of the pingora lifecycle.

[github-actions[bot]]

This question has been stale for a week. It will be closed in an additional day if not updated.

[machine3]

Right now it does not support adding new listeners or service without a (graceful) restart as mentioned above. But you can have dynamic routing and upstream without restarting.

Is there a plan to support this in the future? @eaufavor

[github-actions[bot]]

This question has been stale for a week. It will be closed in an additional day if not updated.

[github-actions[bot]]

This issue has been closed because it has been stalled with no activity.