NeMo-Guardrails icon indicating copy to clipboard operation
NeMo-Guardrails copied to clipboard

Integrating AutoAlign's guardrail library with NeMo Guardrails

Open rahm-decypher opened this issue 1 year ago • 3 comments

This PR demonstrate how to config and use AutoAlign's guardrail library.

The readme at nemoguardrails/library/autoalign/README.md describes AutoAlign's guardrail configuration and api descriptions.

The following folders have example running setup on how to use AutoAlign:

/examples/configs/autoalign/autoalign_config/ examples/configs/autoalign/autoalign_factcheck_config

And, these AutoAlign setups can be executed from a terminal with nemoguardrails chat --config=./examples/configs/autoalign/autoalign_config and nemoguardrails chat --config=./examples/configs/autoalign/autoalign_factcheck_config.

In order to use the AutoAlign, The AUTOALIGN_API_KEY (and relevant model api keys, when needed) has to be setup as environment variables, and user can obtain an AutoAlign key contacting us at [email protected].

One example session with AutoAlign could look like: image

ps: @drazvan i dont seem to be able to add you as a reviewer; could you please take a look?

rahm-decypher avatar Mar 12 '24 16:03 rahm-decypher

@rahm-decypher / @abhijitpal1247 : is this ready for another review? Feel free to add comments on the discussion items from the first review, or mark as resolved.

drazvan avatar Apr 26 '24 12:04 drazvan

@drazvan it is good to review; @abhijitpal1247 had marked issues as resolved. Could you please refer to may last email on how to test locally? Please let us know if you face issues while testing. Thanks very much!

rahm-decypher avatar Apr 26 '24 15:04 rahm-decypher

@drazvan wondering, were you able to test the AutoAlign integration? im copying the testing instruction from our earlier conversations below. Please let us know if you face issues while testing. Thanks!

I have tested the autoalign key (J4b98ja7KaJh) locally, and it seems to work. Btw, you will be required to set your open ai key - the example config was set to work with the user's open ai key and model. I followed the sample steps below to start a autoalign guarded chat session:

---------------------------
git clone https://github.com/NVIDIA/NeMo-Guardrails.git
cd NeMo-Guardrails/
git fetch origin pull/395/head:pull_395
git checkout pull_395
pip install -e .
export OPENAI_API_KEY=<your open ai key>
export AUTOALIGN_API_KEY=<your AutoAlign api key, sent via email>
nemoguardrails chat --config examples/configs/autoalign/autoalign_config/
---------------------------

rahm-decypher avatar May 09 '24 04:05 rahm-decypher

@drazvan no worries; please take a look at the latest update. Thanks.

rahm-decypher avatar Jun 03 '24 21:06 rahm-decypher

@rahm-decypher @abhijitpal1247: Thanks for the changes. The only blocking issue now is the fact that not all commits are signed. One way to solve this is :

  1. Configure automatic signing of commits (see the Contributing guide).

  2. Check out the current develop branch in a separate branch

git remote add nvidia [email protected]:NVIDIA/NeMo-Guardrails.git
git checkout -b nvidia-develop nvidia/develop
  1. Rebase against that branch. The rebase will automatically re-sign all the commits.
git checkout develop
git rebase nvidia-develop

Because of the rebase you'll have to force-push the branch, or create another PR against a new branch.

Hopefully the above works. Let me know if you need more guidance.

Oh, and also, include the following fix: https://github.com/NVIDIA/NeMo-Guardrails/commit/0cc75f4eddfc8a885e1c14aff37a6a140eeec58f (I could not push directly to your repo, and didn't create another fork).

drazvan avatar Jun 05 '24 14:06 drazvan

@drazvan @rahm-decypher new PR is open #543

Please take a look

abhijitpal1247 avatar Jun 05 '24 18:06 abhijitpal1247

This work was merged as part of #543.

drazvan avatar Jun 06 '24 09:06 drazvan