superduper
superduper copied to clipboard
[DistEnv] Circulate configuration using Ray.
For the moment, we need to define the env variables of the configuration at every services we instantiate on the kubernetes stack.
That includes the data backend credentials.
However, for the native snowflake packaging this won't be possible since we don't know the credentials of the user who will install the package.
To mitigate this issue, we need to use the ray capabilities for circulating the configuration.
https://docs.ray.io/en/latest/ray-core/handling-dependencies.html