IBM
Updating RAG example to use Granite model
Why are these changes needed?
Using Granite as our LLM
Related issue number (if any).
Thanks, @sujee I tested this with the Granite model you have set up as: MY_CONFIG.LLM_MODEL = "ibm-granite/granite-20b-code-instruct-8k" And it is fine. Doesn't it make sense to change the name of the Notebook from: rag_1D_query_llama_replicate.ipynb to rag_1D_query_granite_replicate.ipynb?
@sujee If you are ok with changing the name of the notebook, please remember to change the README file, too: https://github.com/IBM/data-prep-kit/blob/dev/examples/notebooks/rag/README.md
@sujee If you are ok with changing the name of the notebook, please remember to change the README file, too: https://github.com/IBM/data-prep-kit/blob/dev/examples/notebooks/rag/README.md
DONE. thx
@sujee In the meeting with Hima this morning, she said that IBM will open-source the Granite "language" models in a week or so and using the "code" model "ibm-granite/granite-20b-code-instruct-8k" (I am sure you used it because it is already open-sourced), doesn't make sense in the RAG example. @touma-I This means holding back on merging this PR, until the Granite language model is open-sourced and we change the code model with that.
can we merge this so we can run examples from the main repo? once the language model drops and available on replicate, I will submit another PR to update the model
@sujee I see that the Granite language models are now available on replicate.com (see: https://replicate.com/search?query=granite). Please change to these models (2b and 8b) and tag me for testing.