Send "GraphCypherQAChain" context to fact checking

[analyticalpicasso]

Hi,

Working on implementing Fact-check solution. I need to understand how can I send context from GraphCypherQAChain.

def _get_qa_chain_graph(llm):
    """Initializes a QA chain using the jobs report.

    It uses OpenAI embeddings.
    """
    graph.refresh_schema()
    cypher_chain = GraphCypherQAChain.from_llm(
        cypher_llm = ChatOpenAI(temperature=0, model_name='gpt-4'),
        qa_llm = ChatOpenAI(temperature=0, openai_api_key=openai.api_key), graph=graph, verbose=True,
    )

    return cypher_chain

define flow
  user ...
  $contexts = execute qa_chain_context(query=$last_user_message)
  $answer = execute qa_chain_graph(query=$last_user_message)
  $accurate = execute check_facts(evidence_list = $contexts)
  bot $answer
  bot facts checked

Additionally, I am unable to send multiple bot messages at once to the solution. Kindly help.

[drazvan]

The self check facts action fetches the data directly from context variables: https://github.com/NVIDIA/NeMo-Guardrails/blob/main/nemoguardrails/library/self_check/facts/actions.py#L36

evidence = context.get("relevant_chunks", [])
response = context.get("bot_message")

So, if you write the content into those variables, instead of $answer and $contexts and set the $check_facts to True, it should work (see also this for more context https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/docs/user_guides/guardrails-library.md#usage-2) :

define flow
  user ...
  $relevant_chunks = execute qa_chain_context(query=$user_message)
  $bot_message = execute qa_chain_graph(query=$user_message)

  $check_facts = True
  bot $answer

  # I assume this is just a confirmation message
  bot facts checked

Also, you should use $user_message rather than $last_user_message.

Let me know if this works.

[analyticalpicasso]

self check facts is not working for me here. It doesn't pass anything to evidence. It does pass generated answer to hypothesis. But evidence i completely empty.