NeMo-Guardrails icon indicating copy to clipboard operation
NeMo-Guardrails copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

Execution of action in parallel with input guardrails

Open maneeshv8 opened this issue 1 year ago • 5 comments

I'm trying to make my bot fast in responding to the user. So, instead of waiting for the input guardrail to finish, I want to use colang logic to determine by action and fetch the context; meanwhile, the input guardrails decide whether to allow the input or reject.

This was context/retrieved knowledge ready for inference or some other action by the time the guardrails had passed.

Is there a way to do this now? Currently, I'm utilising 2 Nemo apps to make this happen (Hacky way)

maneeshv8 avatar Jan 16 '24 10:01 maneeshv8

Hi @maneeshv8! Currently, this is not possible. We'll think of a way to enable this through a flag, and it will be part of #256. Thanks for bringing this up. Good timing.

drazvan avatar Jan 16 '24 11:01 drazvan

Ok. Thanks for the response @drazvan. I am looking forward to that change.

maneeshv8 avatar Jan 17 '24 07:01 maneeshv8

Hi @drazvan I am looking for the same feature but couldn't find in merged PR. If it is added is there any chance to give an example usage?

curefatih avatar May 07 '24 07:05 curefatih

Hi @curefatih! That feature did not make it, unfortunately. It should make it to 0.10.0, though. As a workaround, you can achieve this by making to parallel calls from your code. One that only triggers the input rails and one that only triggers the others (https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/docs/user_guides/advanced/generation-options.md#disabling-rails).

drazvan avatar May 07 '24 07:05 drazvan

Thanks a lot for the quick response @drazvan Make sense to use by switching the rail in this case as you suggested

curefatih avatar May 07 '24 07:05 curefatih