Christian Otto Stelter

icon company PlusServer GmbH icon location Verl, Germany icon location Senior DevOps Engineer @plusserver, dog person. Currently building clouds based on @openstack and the tools to deploy and maintain these clouds.

Results 3 issues of Christian Otto Stelter

inference-workers exits when trying other models than distilgpt2 (on a non-GPU system)

2 comment

Is it possible to run the worker with other models than distilgpt2 on a non GPU-system? After successfully launching the services (profiles ci + inference) with the distilgpt2 model, I...

question
inference

Generic endpoint support for OpenAI API compatible providers

5 comment

**Is your feature request related to a problem? Please describe.** I would like to see generic support for OpenAI API compatible providers as there are already quite a lot of...

enhancement

Add support for Idefics 3

3 comment

### Model description Please add support for HuggingFaceM4/Idefics3-8B-Llama3 in tgi: _Idefics3 is an open multimodal model that accepts arbitrary sequences of image and text inputs and produces text outputs. The...

new model