Cohere Chat Model Integration

[habanoz]

I added Cohere chat model support e.g. command-r. I followed the way ChatOpenAI model is currently integrated.

I have provided unit tests, similar to that provided for nemollm.

I also created config file cohere-commandr-config.yml for prompts customized for the command-r model.

I am not affiliated with Cohere. I just needed this for my tests. And sharing for the broader benefit of the community.

[Pouyanpi]

Hi @habanoz Thanks for your contribution! Please check your email asap; there’s an issue with your PR that needs your attention.

[habanoz]

Hi @habanoz Thanks for your contribution! Please check your email asap; there’s an issue with your PR that needs your attention.

@Pouyanpi Thanks for pointing out.

It is not a real API key (e.g. arbitrary leading x chars). ChatCohere does not initialize without it.

[habanoz]

My latest changes about cohere embeddings support also ended up in this PR. This was unintentional. Sorry about the long PR. Please review the PR as is.

Updates include CohereEmbeddingModel class which allows using Cohere embeddings API. Unit tests are added.

[Pouyanpi]

Hi @habanoz Thanks for your contribution! Please check your email asap; there’s an issue with your PR that needs your attention.

@Pouyanpi Thanks for pointing out.

It is not a real API key (e.g. arbitrary leading x chars). ChatCohere does not initialize without it.

Great! I'll let you know how we can address it in the review.

[drazvan]

Hi @habanoz! I see you've closed this. We'd still like to merge this for 0.10.

[habanoz]

Hi @habanoz! I see you've closed this. We'd still like to merge this for 0.10.

Hi @drazvan I can imagine how busy you are. I just interpreted the long silence as a lack of interest for the PR. Let's proceed as you see fit.

[habanoz]

During the review process, I realized command-r does not align with the sample intents given. I tried to modify the prompts but it did not help. I feel command-r is either not useful or hard to make use of for the intent extraction tasks.

I tried generate endpoint (as opposed to chat endpoint) with command-r but it does not accept command-r.

[trebedea]

Hi, @habanoz ! Unfortunately, if the command-r model does not work for generating intents, it does not really seem useful for Guardrails for now. I am sure that with a different prompt, command-r will work, but we don't have the bandwidth to do prompt engineering.

If you are able to find a good prompt, the contribution was good otherwise.

[habanoz]

Hi, @habanoz ! Unfortunately, if the command-r model does not work for generating intents, it does not really seem useful for Guardrails for now. I am sure that with a different prompt, command-r will work, but we don't have the bandwidth to do prompt engineering.

If you are able to find a good prompt, the contribution was good otherwise.

I agree. The prompt was working when I first pushed it, but probably command-r is updated in the meantime which made the prompt useless. I tried variations without any success. Let's leave it.

BTW I deleted my fork without noticing it will affect this PR.

Anyways; I will continue contributing in the future if I see a gap.