Martin Treml
Martin Treml
The copy task to get the evaluations after all models are run on kubernetes is currently failing with the following Stacktrace: ``` 2024/08/05 09:51:15 $ kubectl cp --namespace eval-dev-quality eval-dev-quality/eval-storage-access-688fcdd98d-9th4q:/var/evaluations/....
I tried to combine all the reports of the last run into one csv with the following command: ``` eval-dev-quality report --evaluation-path ./**/evaluation.csv ``` The problem is that the `/**/`...
This image shows the uptime of the `RWKV v5 World 3B` model:  This model is so long down that the retry logic we are currently using does not really...
It seems that Openrouter now has a few provider which provide a lower quantization size than others, we need to ensure with our calls to openrouter that we are not...
When running the evaluation and specifying the same model multiple times currently the evaluation is run X amount of times for the model. Example: ``` eval-dev-quality evaluate --runtime docker --result-path...
When we are running multiple models within the kubernetes runner we are only watching for the `completed` type on the job. When an model is not available anymore the job...
For those who didn't know about it: https://github.com/bitnami/containers/issues/75671 With January 6th 2025 Bitnami only releases their `latest` branch for free (PostgreSQL 17) updates to older versions are going to vanish...
### Chart Name and Version n/a ### Feature Description The update script should check if the new semver version is higher then the current one ### Motivation Currently if the...
### Feature Description Usually in Kubernetes deployments are used to deploy stateless applications, if an PVC is involved it has to be an "ReadWriteMany" one so all deployments can access...