Sagemaker endpoint not integrated

[maheshk1022]

While using nemo guardrail it supports sagemaker endpoint in provider list but when try to do inference then it generating complete garbage responce.

This is the code snippet I am using.

config = RailsConfig.from_content( colang_content=colang_content, yaml_content=yaml_content) rails = LLMRails(config=config,llm = sagemaker_endpoint)

await rails.generate_async(messages = [{"role": "user", "content": "How do I cook spaghetti?"}])

Also I have tried different prompt twiking but it did not works well.

[Pouyanpi]

Hi @maheshk1022, thanks for sharing it with us. Would it be possible for you to share more info, like what config you are using, so that we can reproduce it? Thanks!