Sam Stoelinga

@alpe if you have some ideas on how to fix this, feel free to just take my test and submit a PR. I haven't started looking into how to fix...

I tried to reproduce in go but it ended up being too much work. I do agree that we should test this kind of thing in a unit test case...

Hmm but we also set a lot of TPU environment variables in launch.py without any if statements. I don't think there is a better place since it needs to happen...

I will take a look in more detail later (likely tomorrow). In the meantime, here is an example that uses ollama with PVC using kind and local storage: https://github.com/substratusai/kubeai/tree/main/test/e2e/engine-ollama-pvc Hope...

Note that you should use a storage solution that provides ReadWriteMany or ReadOnlyMany if you want to run on more than 1 node/replica. You can use both hostPath or local...

Yeah you could probably load the model on the hostPath using some other method. As long as the every hostPath on all nodes have the required files, you should be...

Thanks! That's very helpful to know. I will try and do a build to see if the compatibility issue is still there. Ideally both build and openai image use 22.04...

My laptop crapped out during local builds for some reason. I will try to pick this up by spinning up a cloud VM to do builds.

The github action CI job runs out of disk space lol. We should optimize the Docker image so they're smaller.

Seems @lukebaumann encountered an issue when not using `create-pathways`. A potential fix is to remove the `create-pathways` command all together since it doesn't seem needed. We may be able to...