passing in relevant_chunks as a message's context

[johnyoonh]

In the Rag example, passing in the relevant_chunks as a message is possible

response = rails.generate(messages=[{
    "role": "context",
    "content": {
        "relevant_chunks": """
            Employees are eligible for the following time off:
              * Vacation: 20 days per year, accrued monthly.
              * Sick leave: 15 days per year, accrued monthly.
              * Personal days: 5 days per year, accrued monthly.
              * Paid holidays: New Year's Day, Memorial Day, Independence Day, Thanksgiving Day, Christmas Day.
              * Bereavement leave: 3 days paid leave for immediate family members, 1 day for non-immediate family members. """
    }
}]

But when trying this as a JSON input, the relevant_chunks will trigger ContextUpdate and make the state update to stop, ending the event chain to be incomplete and input request failing silently with empty response.

Here is my current workaround, where I try to treat context update for relevant_chunks as other types of event, so that new_state is created and event chain is continued: nemoguardrails/colang/v1_0/runtime/flows.py

    # We don't need to decide any next step on context updates.
    if event["type"] == "ContextUpdate":
        # TODO: add support to also remove keys from the context.
        #  maybe with a special context key e.g. "__remove__": ["key1", "key2"]
        state.context.update({"context": event["data"]})
        if not 'relevant_chunks' in event["data"]:
            state.context.update(event["data"])
            state.context_updates = {}
            state.next_step = None
            return state

My proposed solution is to allow for custom role and update the context with consistent name pattern: "user_message", "context_message", "assistant_message", "custom_message", etc

This would allow me to pass in context specific message like this:

[{
    "role": "context",
    "content": "I am context content"
    },
{
    "role": "custom",
    "content": "I am content for custom role"
    }
    ]

In an action, I would be able to retrieve them like this: context_message = context.get("context_message")

[drazvan]

Hi @johnyoonh! I'm not sure I understand the issue. If you run the generation only with a message that contains the context updates (as in the first snippet you shared), then nothing happens because there's no user message to act on. In the RAG example notebook that you pointed to, the generation call includes both the context (with the relevant chunks) and the user query.

Can you provide an example to reproduce the issue?

[johnyoonh]

Apologize for the delayed response. I only included context for the brevity of an example, but the behavior is the same even when I include messages with roles: user and assitant to get the proper RAG action. Try the following example. It will not trigger actions properly because the events don't continue to get chained once ContextUpdate happens:

response = rails.generate(messages=[, { "role": "user", "content": "What's the policy on vacation time off?" }, { "role": "assistant", "content": "Employees are eligible for vacation of 20 days per year, accrued monthly" }, "role": "context", "content": { "relevant_chunks": """ Employees are eligible for the following time off: * Vacation: 20 days per year, accrued monthly. * Sick leave: 15 days per year, accrued monthly. * Personal days: 5 days per year, accrued monthly. * Paid holidays: New Year's Day, Memorial Day, Independence Day, Thanksgiving Day, Christmas Day. * Bereavement leave: 3 days paid leave for immediate family members, 1 day for non-immediate family members. """ }, { "role": "context", "content": { "relevant_chunks": """ Employees are eligible for the following time off: * Vacation: 20 days per year, accrued monthly. * Sick leave: 15 days per year, accrued monthly. * Personal days: 5 days per year, accrued monthly. * Paid holidays: New Year's Day, Memorial Day, Independence Day, Thanksgiving Day, Christmas Day. * Bereavement leave: 3 days paid leave for immediate family members, 1 day for non-immediate family members. """ } }]