NeMo-Guardrails
NeMo-Guardrails copied to clipboard
NVIDIA
What blocks the multi-language use case?
Failed to use Nemo guardrail with other languages, the responses are always like I don't understand <language>, so it does recognize the language of input, but it still provides no answer.
I am wondering what stops multi-language use cases. Is it because the Colang only support English? That means even if I define a greeting flow define user express greeting in Franch define utilisateur exprimer salutation wouldn't work?
In theory, nothing should block the multi-language use case. I'll follow up on this in a few days. Someone was able to have a bot respond in Japanese if I remember correctly.
We're not actively developing this yet. We have done some internal testing in two directions:
- Use translation in and out, and keep everything in English.
- Define the canonical forms, examples, bot messages in a different language.
Both approaches worked reasonably well. The second one is highly dependent on the capabilities of the underlying model to understand a non-English language. If you are developing a non-English / multi-language bot, and have a specific issue with either of the two approaches, let us know and we can help.
Hi! Any progress on this? I would like the bot to be able to respond in whatever language the user is utilising. Even if I set this in the model's instructions or if I provide examples in the rails, it tends to default back to English as the conversation evolves. Any guide on how to go about this would be appreciated!
@hww-rociorey: what model are you testing with?
I'm using gpt-3.5-turbo-instruct
@patriciapampanelli: can you provide more guidance on how to keep the LLM always responding in a different language? Was the "return to English" an issue in the tests you did as well?
@drazvan I didn't see this type of problem in the tests. The most promising results were obtained with the manual translation of the model's instructions, as well as canonical forms, examples, and bot messages.
@hww-rociorey Are you evaluating one language at a time? Have the model instructions also been translated?
