Add new attribute 'citations' to ConversationMessage type

[hghandri]

Issue number:

Summary

Update the type "ConversationMessage" to include a "citations" property, allowing the possibility to reference the sources used by the LLM to generate its responses. This is the first step towards enabling everyone to conveniently use citations for each agent, with built-in source and citation capabilities.

Changes

Please provide a summary of what's being changed

User experience

Please share what the user experience looks like before and after this change

For certain LLMs that use the web or knowledge base systems, it might be beneficial to track the source of the response, such as a link to a website or a document in the knowledge base.

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• [ ] I have performed a self-review of this change
• [ ] Changes have been tested
• [ ] Changes are documented

Is this a breaking change?

RFC issue number:

Checklist:

• [ ] Migration process documented
• [ ] Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[brnaba-aws]

Hi @hghandri Thanks for your contribution. I like the idea of adding citations to the ConversationMessage. However, I think it might be a good opportunity to make it more flexible. Citations can be text, but they could also include document name, pages etc... Also, as part of extending ConversationMessage, we could also think about adding usage (tokens) to it. I'd suggest to make it a metadata type that could include much more than just citations.

[hghandri]

Great idea ! I will work on it. What kind of metadata do you think we should implement exactly? An exhaustive list is required to define an appropriate subtype of ConversationMessage.

By the way, I’ve made some changes to integrate the usage of the new version of ConversationMessage.

[brnaba-aws]

This is a good start: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-runtime/client/converse_stream.html

usage and latency

[hghandri]

@brnaba-aws : I’ve made some changes to the Bedrock LLM agent. What do you think? Is this the right approach expected ?

[hghandri]

@brnaba-aws : I've just pushed a last update of source, could you check ?

[hghandri]

Hello @brnaba-aws , @cornelcroi

Any updates on this PR ? We need it for our internal project.

[brnaba-aws]

@hghandri I'll review it when I have the time. But the citations should also include the file location, file name, the chunck and the page in the doc.

[Ciaran0]

Any update on this? Showing citations is extremely important for us. Thanks

[suraj143rosy]

Any update on this? Citations will be crucial in agentic frameworks.

[brnaba-aws]

we will resume the work on this. Thanks for your patience

[sabpap]

Hi @hghandri Thanks for your contribution. I like the idea of adding citations to the ConversationMessage. However, I think it might be a good opportunity to make it more flexible. Citations can be text, but they could also include document name, pages etc... Also, as part of extending ConversationMessage, we could also think about adding usage (tokens) to it. I'd suggest to make it a metadata type that could include much more than just citations.

Token usage and citations would be a great addition indeed. Would you consider adding tool_use and tool_response sequences to the ConversationMessage as well?