NeMo-Guardrails
NeMo-Guardrails copied to clipboard
bug: Optimize the time taken while loading LLMRails / initializing dispatchers
Did you check docs and existing issues?
- [x] I have read all the NeMo-Guardrails docs
- [x] I have updated the package to the latest version before submitting this issue
- [x] (optional) I have used the develop branch
- [x] I have searched the existing issues of NeMo-Guardrails
Python version (python --version)
Python 3.11.6
Operating system/version
Windows
NeMo-Guardrails version (if you must use a specific version and not the latest
0.12.0
Describe the bug
For the project, we want to run selective flow . for example , self_check_input. Ideally self_check_input is talking negligible time like 1.25 seconds. however, before calling any rails, dispatchers are getting loaded, which is taking almost 15 seconds, which is huge. is there any way to load and run necessary dispatchers or minimize loading time of dispatchers.
attaching screenshot for reference
Steps To Reproduce
Just run any rails , it will populate all dispatchers, calculate duration for loading the dispatchers
Expected Behavior
Should minimize the dispatchers initialization time or initialize just necessary dispatchers
Actual Behavior
taking too much time while dispatchers initialization