superduper icon indicating copy to clipboard operation
superduper copied to clipboard

[DistEnv] Circulate configuration using Ray.

Open fnikolai opened this issue 1 year ago • 0 comments

For the moment, we need to define the env variables of the configuration at every services we instantiate on the kubernetes stack.

That includes the data backend credentials.

However, for the native snowflake packaging this won't be possible since we don't know the credentials of the user who will install the package.

To mitigate this issue, we need to use the ray capabilities for circulating the configuration.

https://docs.ray.io/en/latest/ray-core/handling-dependencies.html

fnikolai avatar May 21 '24 18:05 fnikolai