Test with client and server having different frame size limits

[tigrannajaryan]

I was advised that different websocket implementations may have different frame size limits (typically 32K or64K bytes) which may result in interoperability problems. This needs testing and confirming that either it is not an issue or it needs a solution/workaround.

[andykellr]

While investigating this issue, many of the references I found regarding frame size limits refer to the 32K limit in AWS API Gateway.

Because of the WebSocket frame-size quota of 32 KB, a message larger than 32 KB must be split into multiple frames, each 32 KB or smaller. If a larger message (or larger frame size) is received, the connection is closed with code 1009.

https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html

This particular problem would present itself if an OpAMP server were hosted in AWS behind API Gateway and Agents were connecting to the server using an OpAMP client implementation with a WebSocket library that sent frames larger than 32K.

[tigrannajaryan]

Thanks for the research.

What do you think if we add a (SHOULD) recommendation that Client implementations must limit outgoing frame size to 32KB?

I wonder what frame size our current opamp-go implementation uses.

[tigrannajaryan]

Apparently it is not me who thinks 32KB frame/128KB message is unreasonably low limit: https://repost.aws/questions/QUpz6uWU0SSnunjzPrbmLtRg/api-gateway-websockets-size-limit-feedback

[tigrannajaryan]

I don't think we want to do message fragmentation and reassembly ourselves. That would dramatically complicate OpAMP just to accommodate one particular implementation that has unusually low limits.

I think we can probably do is this:

• Add a warning in the spec that some implementations have low frame/message size limits that may cause delivery failures and recommend to use plain HTTP transport if that happens.
• In our opamp-go implementation try to fragment large messages into small frames. Perhaps the frame size can be a client/server config option. For Gorilla Websockets using NextWriter should result in slicing into frames of at most WriteBufferSize.
• We should probably also file a feature request to Gorilla Websockets to add frame size limit as a proper option so that we don't rely on an undocumented behavior.

Any other ideas?

[tigrannajaryan]

Filed a feature request to gorilla websocket for now: https://github.com/gorilla/websocket/issues/814

[tigrannajaryan]

We have added an ability to introduce message fragmentation in the future, which is how we will probably solve this problem.

I think there is nothing else to do at the moment on this issue. Closing.