Ettore Di Giacinto

@dependabot recreate

we don't filter by model category yet - but this is something in our radar and that should be addressed at some point

we should definetly enhance logging but also the logic here around loading. Going to treat this as a feature request. We should at least not try at all the model...

> @mudler @sozercan Some more context now i have a working prototype for parsing gguf models. > > Using GGUF-Parser i have following output for model: `Meta Llama Meta Llama...

With https://github.com/mudler/LocalAI/pull/3729 should cover most of the models and add also video understanding. Model configuration files needs to be specify placeholders used by models for image/video tags in the text...

another way would be to use the gpu images which comes with python and the backends already installed. See also: https://localai.io/basics/container/#standard-container-images

@Fr0d0Beutl1n python dependencies are available only in extra images as documented in https://localai.io/basics/container/#standard-container-images

@chris-hatton looks something behind the scenes is happening - care to share dmesg when that happens? Also, why using `REBUILD`? The image should have already binaries for the CPU flagset...

@chris-hatton gotcha, I did not payed attention to the messages that you are actually using hipblas images. Could you try https://github.com/mudler/LocalAI/pull/4167 if solves your issues once merged in master?