Lex v2 multiple response cards are not working with full page loader

[AbhishekDesai99]

This is the response, I am sending from my lambda function, it has 3 messages, but UI is only showing 1st and 3rd message only. When sending multiple response cards, UI is just showing the last from the messages. { "interpretations": [ { "intent": { "confirmationState": "None", "name": "WelcomeIntent", "slots": {}, "state": "Fulfilled" }, "nluConfidence": { "score": 1 } }, { "intent": { "name": "FallbackIntent", "slots": {} } }, { "intent": { "name": "categoryIntent", "slots": { "category": null } }, "nluConfidence": { "score": 0.47 } }, { "intent": { "name": "ExampleIntent", "slots": {} }, "nluConfidence": { "score": 0.44 } } ], "messages": [ { "content": "<div>Welcome to Mitzon. I am an AI based agent and work 24 by 7. The more you interact with me the better I would be able help you to find the right course for your career growth.<br>Select the category or type a category</div>", "contentType": "CustomPayload" }, { "contentType": "ImageResponseCard", "imageResponseCard": { "buttons": [ { "text": "Computer Science", "value": "Computer Science" }, { "text": "Communication", "value": "Communication" }, { "text": "Humanities", "value": "Humanities" }, { "text": "Social Sciences", "value": "Social Sciences" }, { "text": "Business Management", "value": "Business & Management" } ], "title": "Course Categories" } }, { "contentType": "ImageResponseCard", "imageResponseCard": { "buttons": [ { "text": "Select All", "value": "show courses for all" }, { "text": "Show more", "value": "show more categories" } ], "title": "Course Categories" } } ], "requestAttributes": {}, "sessionId": "us-east-1:a5a90f32-5b15-4947-bfdd-95a09595eb1a", "sessionState": { "dialogAction": { "type": "Close" }, "intent": { "confirmationState": "None", "name": "WelcomeIntent", "slots": {}, "state": "Fulfilled" }, "originatingRequestId": "f662f6a1-deb3-4707-83b0-665c42387d7a", "sessionAttributes": { "category": "[\"Computer Science\",\"Communication\",\"Humanities\",\"Social Sciences\",\"Business & Management\"]", "welcomeIntent_lastRoundIndex": "5" } } }

When testing from the Lex V2 console, it is perfectly showing both the response cards. But when using with web UI, it is the issue with multiple response cards.

Also there is very less documentaion available for integrating Lex v2 bots with web UI, also the response format shown on the official documentaion https://docs.aws.amazon.com/lexv2/latest/dg/lambda.html is bit different than the one actually working.

The below code is shown on official docs for sending response with response cards "messages": [ { "contentType": "CustomPayload | ImageResponseCard | PlainText | SSML", "content": "string", "imageResponseCard": { "title": "string", "subtitle": "string", "imageUrl": "string", "buttons": [ { "text": "string", "value": "string" } ] } } ],

This is the exact message, that my lambda function is sending back "messages": [ { "content": "<div>Welcome to Mitzon. I am an AI based agent and work 24 by 7. The more you interact with me the better I would be able help you to find the right course for your career growth.<br>Select the category or type a category</div>", "contentType": "CustomPayload" }, { "contentType": "ImageResponseCard", "imageResponseCard": { "buttons": [ { "text": "Computer Science", "value": "Computer Science" }, { "text": "Communication", "value": "Communication" }, { "text": "Humanities", "value": "Humanities" }, { "text": "Social Sciences", "value": "Social Sciences" }, { "text": "Business Management", "value": "Business & Management" } ], "title": "Course Categories" } }, { "contentType": "ImageResponseCard", "imageResponseCard": { "buttons": [ { "text": "Select All", "value": "show courses for all" }, { "text": "Show more", "value": "show more categories" } ], "title": "Course Categories" } } ]

But when processing this response, web UI is not rendering the first image response card, I can see only two buttons for Select All and Show more

how Lex V2 console is properly showing the message, with all the response cards

how the web UI is rendering this response

[bobpskier]

Lex V1 supported a single response card as part of the return structure from a Lambda. LexWebUi was originally implemented supporting this single response card. Lex V2 changed the response and added support for multiple response cards. The changes in V2 were not backward compatible with Lex V1. LexWebUi was reworked to support calling a V2 Lex Bot implementation however the UI / Vue components / client side processing still support only a single response card. Its a fairly large undertaking to support the entire native V2 response. I suspect handling multiple response cards will be addressed in the future.

[AbhishekDesai99]

Lex V1 supported a single response card as part of the return structure from a Lambda. LexWebUi was originally implemented supporting this single response card. Lex V2 changed the response and added support for multiple response cards. The changes in V2 were not backward compatible with Lex V1. LexWebUi was reworked to support calling a V2 Lex Bot implementation however the UI / Vue components / client side processing still support only a single response card. Its a fairly large undertaking to support the entire native V2 response. I suspect handling multiple response cards will be addressed in the future.

Thanks for the response, is there any other option available to replace this web UI and still use Lex V2 only? We are also looking for Amplify JS INTERACTIONS, but there is no documentation available for integrating it with Lex V2. Right now we are shifting to Lex V1, but our main purpose was to explore Lex v2 capabilities.

[bobpskier]

Other than Amplify, I'm not ware of other pre-created web interfaces for Lex. I'd recommend to stick with Lex V2 as V1 is not the preferred path going forward especially for new features/development on the AWS side. New AWS regions do not support V1. Depending on where you want to deploy, V1 may not be available.

I'm going to take a look today to see what its going to take to support multiple ResponseCards in the base Vue implementation. Hopefully not significant complexity. I'll post a response late in the day Colorado time.

[bobpskier]

I've implemented a feature change that allows multiple ImageResopnseCards in Lex V2 to be handled in LexWebUi. The change was moderately complex. Lambda's that return multiple ImageResponseCards will now be properly displayed in LexWebUi. Its interesting to note that if you manually configure multiple ImageResponseCards in the Lex V2 Console, only one will be selected and returned to the client say for slot prompt. The only way I could get multiple ImageResponseCards to be returned to LexWebUi was via a Lambda code handler. Its not entirely clear from the V2 spec exactly what Lex V2 is designed to handle.

That said, if multiple ImageResponseCards are now part of a single message array from a Lambda response, the LexWebUi with this change should handle it.

I've submitted pull requests to the aws-samples/aws-lex-web-ui repo. It may a bit of time for the AWS team to incorporate into a new release.

If you want to build yourself I think you might be able to grab this pull request and perform a build of your own. I could make my build available in the region you are working in if that would help your testing.

[AbhishekDesai99]

I've implemented a feature change that allows multiple ImageResopnseCards in Lex V2 to be handled in LexWebUi. The change was moderately complex. Lambda's that return multiple ImageResponseCards will now be properly displayed in LexWebUi. Its interesting to note that if you manually configure multiple ImageResponseCards in the Lex V2 Console, only one will be selected and returned to the client say for slot prompt. The only way I could get multiple ImageResponseCards to be returned to LexWebUi was via a Lambda code handler. Its not entirely clear from the V2 spec exactly what Lex V2 is designed to handle.

That said, if multiple ImageResponseCards are now part of a single message array from a Lambda response, the LexWebUi with this change should handle it.

I've submitted pull requests to the aws-samples/aws-lex-web-ui repo. It may a bit of time for the AWS team to incorporate into a new release.

If you want to build yourself I think you might be able to grab this pull request and perform a build of your own. I could make my build available in the region you are working in if that would help your testing.

Thanks for the quick help, I will grab the pull request and make the build. And yes I am using lambda functions only to send response cards. Right now for development, I am using Full Page mode, later on in production, I will shift to IFrame mode. I appreciate your quick help, thanks.

[hongNianYS]

I've implemented a feature change that allows multiple ImageResopnseCards in Lex V2 to be handled in LexWebUi. The change was moderately complex. Lambda's that return multiple ImageResponseCards will now be properly displayed in LexWebUi. Its interesting to note that if you manually configure multiple ImageResponseCards in the Lex V2 Console, only one will be selected and returned to the client say for slot prompt. The only way I could get multiple ImageResponseCards to be returned to LexWebUi was via a Lambda code handler. Its not entirely clear from the V2 spec exactly what Lex V2 is designed to handle.

That said, if multiple ImageResponseCards are now part of a single message array from a Lambda response, the LexWebUi with this change should handle it.

I've submitted pull requests to the aws-samples/aws-lex-web-ui repo. It may a bit of time for the AWS team to incorporate into a new release.

If you want to build yourself I think you might be able to grab this pull request and perform a build of your own. I could make my build available in the region you are working in if that would help your testing.

Is that already applied to the ifram version of lex-web-ui? I still can not get the title of the image card if there is multiple response

[bobpskier]

@hongNianYS Version 0.19.5 contains this fix and has been merged into the master branch. The iframe version will contain this fix. By default, titles of image responsecards are ignored and not displayed. Only sub-titles will be displayed. You can change this behavior by setting the Cloud Formation template parameter 'ShowResponseCardTitle' to true. ResponseCards always require the 'title' property as defined by the lex api however in many use cases, displaying a title is not useful. Hence by default the ShowResponseCardTitle is set to false. Instead the optional subTitle parameter is used to display additional text.