onyx
onyx copied to clipboard
onyx-dot-app
Feature: Support danswer releases through helm charts
There seems to have been an attempt to do this in https://github.com/danswer-ai/danswer/issues/870 and https://github.com/danswer-ai/danswer/pull/871. But both Issue and PR where closed without merging for some reason
@sebracs understood. let me give it a shot and first release a helm chart without disturbing current github actions.
Hey i tried a draft of helm charts. Some problems with vespa stateful set solved some. Some are pending. Right now at an Indian wedding which will last for 4-5 days post that shud be able to complete within 2-3 days. After that regarding github actions for helm chart will trigger it on any of existing github workflow based on docker run to update the image for corresponding modules for helm charts. Please provide feedback for the approach.
@sebracs so current issue is both the deployments - api-server-deployment and background-deployment are referring to same persistent volume. so and persistence type is readwriteonce which is causing race condition. the error is -
PreBind plugin "VolumeBinding": Operation cannot be fulfilled on persistentvolumeclaims "dynamic-pvc": the object has been modified; please apply your changes to the latest version and try again
So right now to enable we gotta have right storage class which supports readWriteMany. so doing that right now so that local experience does not break. most of kubernetes cloud providers should have storage class that supports read write many
right now it will only work if multiple Pods that mount the same volume are running on the same node. thats a limitation that we need to get rid of. leme know your thoughts.
@JayGhiya Do they need to share storage? If not they could have their own PVCs. Of course some storage providers do allow RWX (ReadWriteMany) even across multiple nodes, but it's not available everywhere.
Yea that is doable if they do not need sharing but right now I am unsure regarding roles and responsibilities of both the components. If I can get clarity on that we could make a decision.
@yuhongsun96 do api server deployment and background deployment need to share pvcs or they can use separate ones?
Sorry for delay, ya unfortunately there are some communications that are done through the file system so currently they must have some shared volumes. This does limit the auto-scaling as it's much easier to run things in a single pod. A more complicated approach is needed to share the pvcs across different pods.
This is not a limitation in the design, only the implementation. We haven't gotten around to fixing it, but will do so hopefully soon
I was able to get it to work following this guide to set up an NFS server and changing the PVCs to ReadWriteMany
accessModes:
- ReadWriteMany
Yeah, if you can use ReadWriteMany, it's probably the easiest solution. But of course, it's only supported by specific network file systems and not block devices. At least all the large cloud providers should have it I think
yep. understood. apologies for delayed response. will release the charts today.
will be releasing through rook ceph storage mechanism so that it works locally and also on bare metal kubernetes clusters.
Rook ceph gave a lot of trouble regarding creation of ceph cluster locally on rancher desktop. So switched to openebs based nfs storage. It worked. Final tweaks and our charts Wil be ready to work on local as well as bare metal kubernetes plarforms.
https://github.com/danswer-ai/danswer/pull/1186 . Here is where we are working towards helm chart.
@JayGhiya I see you've merged #1186 and #1177 , is there a PR for the documentation on how to use these helm charts? Or are they still a work in progress?
Thanks for your work on this!
Should the two separate helm charts (https://github.com/danswer-ai/danswer/tree/main/deployment/helm and https://github.com/danswer-ai/danswer/tree/main/deployment/helm) be folded into a single chart? Or is there a reason to keep them separate?
Need to have a pr for documentation. Yes if there are multiple charts they need to be merged into one. Cc: @yuhongsun96 need GitHub actions to host helm charts and then need to add documentation. Do you have some time to connect this week?
I plan to do some cleaning and merging of the two version of the Charts. Will aim to do it this week hopefully! Thanks everyone for the contributions, insights, and interest!
I am trying to deploy danswer on AKS and am confused about which helm chart we are supposed to use, one is 2 months old and the other is 2 weeks old . Can you please suggest which folder we are supposed to use
https://github.com/danswer-ai/danswer/tree/main/deployment/helm or https://github.com/danswer-ai/danswer/tree/main/deployment/kubernetes
So currently if we want to use them, we'd just clone locally them helm install from the local file path? There's no helm repo to add?
ie:
git clone [email protected]:danswer-ai/danswer.git
cd danswer/deployment/kubernetes/charts/danswer-release
helm install danswer .
So currently if we want to use them, we'd just clone locally them helm install from the local file path? There's no helm repo to add?
ie:
git clone [email protected]:danswer-ai/danswer.git cd danswer/deployment/kubernetes/charts/danswer-release helm install danswer .
Yea there is no helm repo as of now. Need to work on github actions for the same. CC: @yuhongsun9
Something like this could work if the index.yaml file for the helm repo was added to the charts folder:
helm repo add danswer https://raw.githubusercontent.com/danswer-ai/danswer/main/deployment/kubernetes/charts/
helm repo update
helm install danswer/danswer-stack