NeMo-Guardrails icon indicating copy to clipboard operation
NeMo-Guardrails copied to clipboard

bug: Optimize the time taken while loading LLMRails / initializing dispatchers

Open ashvmore opened this issue 7 months ago • 3 comments

Did you check docs and existing issues?

  • [x] I have read all the NeMo-Guardrails docs
  • [x] I have updated the package to the latest version before submitting this issue
  • [x] (optional) I have used the develop branch
  • [x] I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

Python 3.11.6

Operating system/version

Windows

NeMo-Guardrails version (if you must use a specific version and not the latest

0.12.0

Describe the bug

For the project, we want to run selective flow . for example , self_check_input. Ideally self_check_input is talking negligible time like 1.25 seconds. however, before calling any rails, dispatchers are getting loaded, which is taking almost 15 seconds, which is huge. is there any way to load and run necessary dispatchers or minimize loading time of dispatchers.

attaching screenshot for reference

Image

Steps To Reproduce

Just run any rails , it will populate all dispatchers, calculate duration for loading the dispatchers

Expected Behavior

Should minimize the dispatchers initialization time or initialize just necessary dispatchers

Actual Behavior

taking too much time while dispatchers initialization

ashvmore avatar Apr 28 '25 15:04 ashvmore