aws-samples
I am getting a model timeout
After 8 minutes (more or less) I am getting a model timeout like this one.
Error processing response stream for session j7WimTYOfJYKQw4VAAAD: ModelTimeoutException: Model has timed out in processing the request. Try your request again. at de_ModelTimeoutExceptionRes (/home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1632:21) at de_ModelTimeoutException_event (/home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1940:10) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async /home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1819:32 at async Object.deserializer (/home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/node_modules/@smithy/eventstream-serde-universal/dist-cjs/index.js:127:37) at async SmithyMessageDecoderStream.asyncIterator (/home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/node_modules/@smithy/eventstream-codec/dist-cjs/index.js:438:28) at async NovaSonicBidirectionalStreamClient.processResponseStream (/home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/dist/client.js:403:30) at async NovaSonicBidirectionalStreamClient.initiateSession (/home/ubuntu/PROJECTS/AI/VOICE/websocket-nodejs/dist/client.js:270:13) { '$fault': 'client', '$metadata': { httpStatusCode: undefined, requestId: undefined, extendedRequestId: undefined, cfId: undefined } } Error in session: { source: 'responseStream', message: 'Error processing response stream', details: 'Model has timed out in processing the request. Try your request again.' }
How can I overcome this? My application will need like minimum one hour.
Here's example on bedrock session restart with nova-sonic : https://github.com/aws-samples/amazon-nova-samples/tree/main/speech-to-speech/repeatable-patterns/resume-conversation You can customize it to have a fresh session or pass in previous session details and data for session continuity.
Here's example on bedrock session restart with nova-sonic : https://github.com/aws-samples/amazon-nova-samples/tree/main/speech-to-speech/repeatable-patterns/resume-conversation You can customize it to have a fresh session or pass in previous session details and data for session continuity.
I've tested this and is working fine, I was thinking how to make this totally transparent to the user.
@tokarev-artem by transparent I mean to start a timer for 8 minute and before expire create a new socket connection and pass the history in setupHistoryForConversationResumtion and move to the new connection, without the user to notify it
Any update on this? I have similar scenario, where after more or less 8 minutes of conversation the model timeout error comes. I am storing the chathistory in dynamodb and able to retrieve it in correct sequence. I want to handles this error in the backend and without disconnecting the session want to continue with the conversation. Right now I am passing the chat history but it is not consistent always.
@anikm1987 It is design intend like also @tokarev-artem mentioned. My plan is to start a timer for each session and when is about to expire to start another one and pass the current content to it, dropping the old one. If I will have time I will do it this week.
@bcatalin, but what would be if the "restart" timer were when Sonic "speaks" ? Will it be interrupted?
@tokarev-artem Good question, will see. We know when there is activity and when not, so it is about finding the right moment.
@tokarev-artem Good question, will see. We know when there is activity and when not, so it is about finding the right moment.
Did you manage to test it? I'm still struggling with adding and it stops responding after 8 minutes