Aaron Batilo

I did NOT have the time when I had hoped to. Please feel free to assign to @petersondmg! I appreciate it though @gmlewis

Hi @petersondmg. If you configure the GitHub App with the `Contents` permissions, then there aren't any problems with downloading from the root. However, if you configure single files only like...

This is a must have feature for anyone that wants to hermetically run terraform with any kind of automation. I should not have to install the OCI cli to every...

I've escalated this to the account team for my employer. No updates from them on this but they're at least aware of the request.

@OlivierDehaene Yes! Here's the full output from adding `--env` as well: ``` root@mixtral:/# text-generation-launcher --model-id mistralai/Mixtral-8x7B-Instruct-v0.1 --dtype bfloat16 --sharded true --num-shard 2 --env 2023-12-11T17:33:30.628861Z INFO text_generation_launcher: Runtime environment: Target: x86_64-unknown-linux-gnu...

Yeah, something fishy is going on here. The container is built with triton 2.1.0 ``` root@mixtral:~# pip freeze | grep "triton" triton==2.1.0 ``` So then I can go look at...

I found a solution. If you're running in Kubernetes, you need to re-run `ldconfig` before `text-generation-inference` runs. ``` - command: - bash - -c - | ldconfig text-generation-launcher --model-id mistralai/Mixtral-8x7B-Instruct-v0.1...

It looks like in the latest version of `triton`, that there's a different env variable that you can set to specify where to find `libcuda.so`: https://github.com/openai/triton/blob/720d12c1433e5b13eb3a18d1e065c1d3d25a32f1/python/triton/common/build.py#L21-L23

@OlivierDehaene could you share a typical manifest that's used?

Hey @sdgluck, so sorry on the delay on this. I've been meaning to respond for weeks. Anyways, the changes would be minimal overall, I think? I don't know if there's...