Usman Ahmad

I think it is the problem with that specific deepspeed version (i.e., 0.4.0) in requirements. In my case, it was solved by upgrading deepspeed. You can upgrade it by using...

Hi @PyxAI . Which ubuntu version did you run this code on?

Any updates on the 4bit inference kernel? @JianbangZ I am also experiencing issues with inference speed.

@xiaofan-luan any update on this? I am also looking to store list of strings (as metadata).

The issue hasn't been fixed yet. Better to implement your own retriever: https://dspy-docs.vercel.app/docs/deep-dive/retrieval_models_clients/custom-rm-client#the-dspythonic-way

Getting same error when trying to run a [gpt2](https://huggingface.co/KennethTM/gpt2-small-danish) model on the latest 2.11.0. Any leads on this issue?

Apologies @xenova . My bad. I forgot to mention that i have already converted weights using the given instructions. The converted weights does have a file called `onnx/decoder_model_merged.onnx` alongwith others.

Thank you @xenova . I have tried that already following the thread but unfortunately it does not work. It is strange that [gpt](https://huggingface.co/gpt2) works but not the danish variant that...

Sure. Let me give another try and I can add a separate issue in case it does not work. Thanks for the prompt response :)