nebari
nebari copied to clipboard
[ENH] - Scheduled Nebari Backups
Feature description
We want to expand the existing configuration to allow it to specify backup parameters. Backing up Nebari includes the following:
- KeyCloak user details.
- User data
- Conda environments
Interface
backup
location: <BACKUP LOCATION>>
schedule: <CRON EXPRESSION>
retention_period: <NUMBER OF DAYS>
Implementation
There are several ways this can be achieved:
- Kubernetes CronJob
- Pros:
- Native to K8
- Good for simple disconnected tasks
- Cons: 2. Will need to custom build and maintain status, monitoring, and alerting.
- Pros:
- Argo Workflows
- Pros:
- Fully featured UI
- Native artifact support
- Workflow archive
- Rest API
- Cons:
- Argo becomes a core component in Nebari.
- Most of our workflows might not be DAGs; it's possible that all the tasks can be independently executed.
- Pros:
Value and/or benefit
- It will save the state from accidental deletes.
Anything else?
No response
### Tasks
- [x] Add the pros and cons of each of the available options.
- [ ] Discuss available options
BTW, I think you would use k8s Jobs instead of k8S Cronjobs. Also, one option to run things sequentially with k8s Jobs - https://stackoverflow.com/questions/40713573/how-to-run-containers-sequentially-as-a-kubernetes-job (haven't tried it myself, but worth a try)
Closing in favor of #2518